eventstream-consumption-cli

by microsoft

Update Check — ONCE PER SESSION (mandatory) The first time this skill is used in a session, run the check-updates skill before proceeding. npx skills add https://github.com/microsoft/skills-for-fabric --skill eventstream-consumption-cli Download ZIPGitHub716

Update Check — ONCE PER SESSION (mandatory) The first time this skill is used in a session, run the check-updates skill before proceeding.

• GitHub Copilot CLI / VS Code: invoke the check-updates skill.

• Claude Code / Cowork / Cursor / Windsurf / Codex: compare local vs remote package.json version.

• Skip if the check was already performed earlier in this session.

CRITICAL NOTES

• To find the workspace details (including its ID) from workspace name: list all workspaces and, then, use JMESPath filtering

• To find the item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace and, then, use JMESPath filtering

• Eventstream ≠ Eventhouse. Eventstream is a real-time event ingestion and routing pipeline. For KQL queries, use eventhouse-consumption-cli.

Eventstream Consumption — CLI Skill

Table of Contents

Task Reference Notes Finding Workspaces and Items in Fabric COMMON-CLI.md § Finding Workspaces and Items in Fabric Mandatory — READ link first [needed for finding workspace id by its name or item id by its name, item type, and workspace id] Fabric Topology & Key Concepts COMMON-CORE.md § Fabric Topology & Key Concepts Environment URLs COMMON-CORE.md § Environment URLs Authentication & Token Acquisition COMMON-CORE.md § Authentication & Token Acquisition Wrong audience = 401; read before any auth issue Core Control-Plane REST APIs COMMON-CORE.md § Core Control-Plane REST APIs Includes pagination, LRO polling, and rate-limiting patterns Gotchas, Best Practices & Troubleshooting COMMON-CORE.md § Gotchas, Best Practices & Troubleshooting Tool Selection Rationale COMMON-CLI.md § Tool Selection Rationale Authentication Recipes COMMON-CLI.md § Authentication Recipes az login flows and token acquisition Fabric Control-Plane API via az rest COMMON-CLI.md § Fabric Control-Plane API via az rest Always pass --resource; includes pagination and LRO helpers Gotchas & Troubleshooting (CLI-Specific) COMMON-CLI.md § Gotchas & Troubleshooting (CLI-Specific) az rest audience, shell escaping, token expiry Quick Reference COMMON-CLI.md § Quick Reference az rest template + token audience/tool matrix Listing and Discovering Eventstreams EVENTSTREAM-CONSUMPTION-CORE.md § Listing and Discovering Eventstreams List, Get, Search across workspaces Inspecting Eventstream Topology EVENTSTREAM-CONSUMPTION-CORE.md § Inspecting Eventstream Topology Decode base64 definition → trace graph flow Monitoring Eventstream Health EVENTSTREAM-CONSUMPTION-CORE.md § Monitoring Eventstream Health Retention and throughput checks Source and Destination Status EVENTSTREAM-CONSUMPTION-CORE.md § Source and Destination Status Validation checklist for sources and destinations Integration with Downstream Analytics EVENTSTREAM-CONSUMPTION-CORE.md § Integration with Downstream Analytics Eventhouse, Lakehouse, Activator, Real-Time Hub Gotchas and Troubleshooting Reference EVENTSTREAM-CONSUMPTION-CORE.md § Gotchas and Troubleshooting Reference 10 common issues with causes and fixes List Eventstreams SKILL.md § List Eventstreams Inspect Eventstream Topology SKILL.md § Inspect Eventstream Topology Decode and explore the graph Get Custom Endpoint Connection String SKILL.md § Get Custom Endpoint Connection String Retrieve Kafka/EH connection via Topology API Validate Eventstream Configuration SKILL.md § Validate Eventstream Configuration Gotchas, Rules, Troubleshooting SKILL.md § Gotchas, Rules, Troubleshooting MUST DO / AVOID / PREFER checklists

List Eventstreams

List All Eventstreams in a Workspace

az rest --method GET \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams" \
 --resource "https://api.fabric.microsoft.com"

Returns an array of Eventstream items. Use JMESPath to filter by name:

az rest --method GET \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams" \
 --resource "https://api.fabric.microsoft.com" \
 --query "value[?displayName=='my-eventstream']"

Get Eventstream Details

az rest --method GET \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}" \
 --resource "https://api.fabric.microsoft.com"

Inspect Eventstream Topology

Tip: The Topology API (GET .../eventstreams/{id}/topology) returns runtime status, error info, and node IDs without base64 decoding. Prefer it for operational inspection (health checks, connection retrieval). Use POST .../getDefinition (below) when you need the full authoring-time graph structure for topology modification.

Retrieve the Eventstream definition and decode it to inspect the full graph topology.

Step 1: Get the Definition

API Note: The Eventstream Definition API uses POST .../getDefinition, not GET .../definition. This follows the Fabric Items Definition pattern. See official docs.

az rest --method POST \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/getDefinition" \
 --resource "https://api.fabric.microsoft.com" \
 --body '{}'

Step 2: Decode the Topology

Extract the eventstream.json part's payload field and base64-decode it:

# Using jq + base64 (Linux; on macOS use base64 -D instead of -d)
az rest --method POST \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/getDefinition" \
 --resource "https://api.fabric.microsoft.com" \
 --body '{}' \
 | jq -r '.definition.parts[] | select(.path=="eventstream.json") | .payload' \
 | base64 -d | jq .

# PowerShell (Windows)
$def = az rest --method POST `
 --url "https://api.fabric.microsoft.com/v1/workspaces/$WORKSPACE_ID/eventstreams/$EVENTSTREAM_ID/getDefinition" `
 --resource "https://api.fabric.microsoft.com" `
 --body '{}' | ConvertFrom-Json
$payload = ($def.definition.parts | Where-Object { $_.path -eq 'eventstream.json' }).payload
[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($payload)) | ConvertFrom-Json | ConvertTo-Json -Depth 10

Step 3: Summarize the Topology

After decoding, count and list each node type:

Metric Path in decoded JSON Sources .sources[] | .name, .type Destinations .destinations[] | .name, .type Operators .operators[] | .name, .type Streams .streams[] | .name, .type

Get Custom Endpoint Connection String

The POST .../getDefinition endpoint returns empty properties for Custom Endpoint sources. To retrieve the Kafka/Event Hub connection info, use the Topology API /connection endpoint.

Important: This endpoint requires Eventstream.ReadWrite.All permission scope (not just Read).

Step 1: Get the Topology to Find the Source ID

az rest --method GET \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/topology" \
 --resource "https://api.fabric.microsoft.com"

From the response, find the Custom Endpoint source node and extract its id:

# Extract the sourceId for a Custom Endpoint source (use name filter if multiple exist)
SOURCE_ID=$(az rest --method GET \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/topology" \
 --resource "https://api.fabric.microsoft.com" \
 | jq -r '[.sources[] | select(.type=="CustomEndpoint")] | if length == 0 then error("No Custom Endpoint sources found in this Eventstream") elif length > 1 then error("Multiple Custom Endpoint sources found — filter by .name") else .[0].id end') \
 || { echo "Failed to resolve Custom Endpoint source ID"; exit 1; }

if [ -z "$SOURCE_ID" ]; then echo "SOURCE_ID is empty — check topology output"; exit 1; fi

# PowerShell — extract sourceId for Custom Endpoint (fails clearly if multiple exist)
$topology = az rest --method GET `
 --url "https://api.fabric.microsoft.com/v1/workspaces/$WORKSPACE_ID/eventstreams/$EVENTSTREAM_ID/topology" `
 --resource "https://api.fabric.microsoft.com" | ConvertFrom-Json
$customSources = @($topology.sources | Where-Object { $_.type -eq 'CustomEndpoint' })
if ($customSources.Count -eq 0) { throw "No Custom Endpoint sources found in this Eventstream" }
if ($customSources.Count -gt 1) { throw "Multiple Custom Endpoint sources found. Filter by name: $($customSources.name -join ', ')" }
$sourceId = $customSources[0].id

Step 2: Get the Connection Details

⚠️ Security: This endpoint returns access keys and connection strings. Get explicit user confirmation before calling it. Redact primaryKey, secondaryKey, primaryConnectionString, and secondaryConnectionString from any displayed output unless the user explicitly asks for secret values in a secure context. Avoid logging raw credentials; store securely and rotate as needed.

az rest --method GET \
 --url "https://api.fabric.microsoft.com/v1/workspaces/${WORKSPACE_ID}/eventstreams/${EVENTSTREAM_ID}/sources/${SOURCE_ID}/connection" \
 --resource "https://api.fabric.microsoft.com"

az rest --method GET `
 --url "https://api.fabric.microsoft.com/v1/workspaces/$WORKSPACE_ID/eventstreams/$EVENTSTREAM_ID/sources/$sourceId/connection" `
 --resource "https://api.fabric.microsoft.com" | ConvertFrom-Json

Expected Response

{
 "fullyQualifiedNamespace": "namespace.servicebus.windows.net",
 "eventHubName": "es_ ",
 "accessKeys": {
 "primaryKey": "...",
 "secondaryKey": "...",
 "primaryConnectionString": "Endpoint=sb://namespace.servicebus.windows.net/;...",
 "secondaryConnectionString": "..."
 }
}

Kafka Producer Configuration

Use the response to configure a Kafka producer:

Setting Value bootstrap_servers {fullyQualifiedNamespace}:9093 topic {eventHubName} security_protocol SASL_SSL sasl_mechanism PLAIN sasl_plain_username $ConnectionString (fixed literal — not a variable) sasl_plain_password {primaryConnectionString}

Limitation: The /connection endpoint is only supported for Custom Endpoint sources (returns Kafka/Event Hub credentials). Other source types (Event Hub, IoT Hub, etc.) store their connection configuration (e.g., dataConnectionId, consumerGroup) directly in the decoded definition properties.