Labsco
microsoft logo

azure-aigateway

✓ Official★ 1,300

by microsoft · part of microsoft/azure-skills

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway.

🔥🔥🔥✓ VerifiedFreeQuick setup
#cloud#devops#git#monitoring
🧩 One of 7 skills in the microsoft/azure-skills package — works on its own, and pairs well with its siblings.

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway.

Inspect the full instructions your agent will receiveExpand

This is the exact playbook injected into your agent when the skill activates — shown here so you can audit it before installing. You don't need to read it to use the skill.

by microsoft

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway. npx skills add https://github.com/microsoft/azure-skills --skill azure-aigateway Download ZIPGitHub1.3k

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

Category Triggers Model Governance "semantic caching", "token limits", "load balance AI", "track token usage" Tool Governance "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" Agent Governance "content safety", "jailbreak detection", "filter harmful content" Configuration "add Azure OpenAI backend", "configure my model", "add AI Foundry model" Testing "test AI gateway", "call OpenAI through gateway"

Quick Reference

Policy Purpose Details azure-openai-token-limit Cost control Model Policies azure-openai-semantic-cache-lookup/store 60-80% cost savings Model Policies azure-openai-emit-token-metric Observability Model Policies llm-content-safety Safety & compliance Agent Policies rate-limit-by-key MCP/tool protection Tool Policies

Get Gateway Details

Copy & paste — that's it
# Get gateway URL
az apim show --name --resource-group --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name --resource-group \
 --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
 --service-name --resource-group --subscription-id

Test AI Endpoint

Copy & paste — that's it
GATEWAY_URL=$(az apim show --name --resource-group --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments/ /chat/completions?api-version=2024-02-01" \
 -H "Content-Type: application/json" \
 -H "Ocp-Apim-Subscription-Key: " \
 -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

Common Tasks

Add AI Backend

See references/patterns.md for full steps.

Copy & paste — that's it
# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name --resource-group \
 --backend-id openai-backend --protocol http --url "https:// .openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee \
 --role "Cognitive Services User" --scope

Apply AI Governance Policy

Recommended policy order in <inbound>:

  • Authentication - Managed identity to backend

  • Semantic Cache Lookup - Check cache before calling AI

  • Token Limits - Cost control

  • Content Safety - Filter harmful content

  • Backend Selection - Load balancing

  • Metrics - Token usage tracking

See references/policies.md for complete example.

References

SDK Quick References