Labsco
MCP SERVER

Dataproc MCP Server

by dipseth

Create Dataproc clusters from a named profile and run Spark or Hive jobs without composing the API call.

Data Platform: Pipelines, Warehousing, BI & Governance
Summary
Profiles carry the parameters, so most calls take a fraction of what the Dataproc API wants.

Project and region are injected from config and cluster shape comes from a profile, which is why creating a cluster or listing them is a one-line request rather than a wall of arguments. Job cancellation is worth knowing about on its own — a runaway Spark job needs only its job ID to stop.

What it is

An MCP server for Google Cloud Dataproc that injects project and region defaults, ships cluster profiles, and covers cluster lifecycle, job submission and analytics.

What you get
  • Cluster lifecycle: start_dataproc_cluster, create_cluster_from_yaml, create_cluster_from_profile, list_clusters, get_cluster, delete_cluster
  • list_tracked_clusters for clusters this server created, and get_zeppelin_url for the notebook UI
  • Jobs: submit_dataproc_job for Spark, PySpark and Presto, plus submit_hive_query, cancel_dataproc_job, get_job_status, get_job_results
  • Hive queries tracked separately: get_query_status and get_query_results, with pagination
  • Profiles and stored data: list_profiles, get_profile, query_cluster_data
  • Analytics: check_active_jobs, get_cluster_insights, get_job_analytics, query_knowledge
  • 8 built-in cluster profiles, so a cluster comes up without hand-writing the config
Requirements

Run it with npx @dipseth/dataproc-mcp-server@latest, or install globally as dataproc-mcp-server. DATAPROC_CONFIG_PATH points at your config. Authentication is service-account impersonation, a direct service-account key, application default credentials, or a hybrid with fallbacks. Version 4.7.0.

Setup effort

One command plus a key — npx @dipseth/dataproc-mcp-server@latest, then supply credentials