Ai-Gateway

How Envoy’s AI Protocol Manager handles LLM traffic today, and a set of tested configs: observing native APIs, translating between OpenAI, Claude and Gemini (Vertex AI included), retries and failover across APIs, a forward-proxy egress, and model routing with ext_proc.

MCP’s 2026-07-28 revision makes the protocol stateless, so MCP traffic now looks a lot like ordinary HTTP. Envoy 1.40 builds on that: route MCP requests by tool name, stop malformed or spoofed requests at the proxy, control who can call which tool, and turn any REST API into an MCP server using only config.