Labsco
MCP SERVER

Kafka MCP Server

by tuannvm

Produce and consume Kafka messages, inspect topics and consumer groups, and check cluster health from the conversation.

Data Platform: Pipelines, Warehousing, BI & Governance
Summary
Reports, not just metadata.

The difference between this and a thin admin-API wrapper is the prompt layer: kafka_consumer_lag_report comes back as a table of groups with their lag and a list of things to check, and kafka_under_replicated_partitions names the affected topics and the likely causes. That is the part an on-call engineer actually wants at 3am. The producer and consumer tools mean the same session can also reproduce the problem, so weigh that against who you give access to.

What it is

A Go MCP server that talks to a Kafka cluster as an admin and a client at the same time. Alongside the tools it publishes diagnostic resources and prompts, so "is the cluster healthy" comes back as a written report rather than as raw metadata for the model to interpret.

What you get
  • Move data — produce_message writes one record with an optional key, consume_messages reads a batch from one or more topics at the latest offsets
  • See what exists — list_topics with partition counts and replication factors, list_brokers, and list_consumer_groups with their states
  • Look at one thing closely — describe_topic returns partitions, leaders, replicas and ISRs; describe_consumer_group returns members, partition assignments and, on request, commit offsets and lag
  • Read configuration — describe_configs for a topic or broker, including retention and cleanup settings and where each value came from
  • cluster_overview — brokers, controller, topic and partition counts, under-replicated and offline partitions in one call
  • Four resources the client can read directly: kafka-mcp://overview, kafka-mcp://health-check, kafka-mcp://under-replicated-partitions and kafka-mcp://consumer-lag-report
  • Four prompts that write the diagnosis rather than dumping the data: kafka_cluster_overview, kafka_health_check, kafka_under_replicated_partitions and kafka_consumer_lag_report
Requirements

Reachable brokers, listed in KAFKA_BROKERS, with KAFKA_CLIENT_ID naming the connection. MCP_TRANSPORT chooses stdio or http. For a secured cluster, KAFKA_SASL_MECHANISM takes plain, scram-sha-256 or scram-sha-512 with KAFKA_SASL_USER and KAFKA_SASL_PASSWORD, and KAFKA_TLS_ENABLE turns on TLS. Over HTTP transport you can additionally put OAuth 2.1 in front of the server: OAUTH_ENABLED with OAUTH_PROVIDER set to hmac, okta, google or azuread, in either a native client-managed or proxy server-managed mode. A container image is published to GHCR.

Setup effort

One command plus a key — brew install kafka-mcp-server, then supply credentials