Teams supplying models to coding agents have traditionally had to either self-host LiteLLM or hand-roll a proxy. Google has now absorbed this layer into a single serverless gateway URL.
How it works
The August 3 API Gateway Release Notes (widely republished by Chinese tech media on August 4–5) introduced model routing, marked preview & v1. The mechanism is straightforward:
1. The client sends an OpenAI-compatible request (POST /chat/completions) to the gateway.
2. The gateway reads the model field in the JSON body.
3. It matches routing rules defined in the OpenAPI spec.
4. It transcodes the request in-flight into Vertex AI's prediction schema.
5. It dispatches to a Model Garden endpoint; unmatched requests fall back to defaultModel.
Google's own positioning: "a managed alternative to client-side proxies such as LiteLLM."
Configuration is declarative, using two OpenAPI 3.x extension fields. The documentation's example target models are google/gemini-3.5-flash-lite, anthropic/claude-opus-4-7, and openai/gpt-oss-120b-maas—note the third is the open-weight gpt-oss (a MaaS open model), not OpenAI's closed-source GPT.
What it is not
Chinese coverage has framed this as "cross-provider routing," which is inaccurate. All three models are hosted via Vertex AI Model Garden; the gateway never connects directly to api.anthropic.com or api.openai.com. It also does not perform intelligent scheduling by cost, latency, or quality—during Preview it "routes based exclusively on the model tag or name in the payload," i.e., static dispatch by model name plus a default fallback. Private deployments and self-hosted models are unsupported, and all models in a single router must share the same hostname.
Why it matters
The significance for AI coding harnesses is operational, not capability. Previously, enterprises supplying models to agents had to run containers, handle scaling and secrets themselves. Now one edge gateway URL replaces that layer: zero client changes (OpenAI SDK connects directly), with authentication, quotas, and usage observability centralized at the edge. For mixes like "Gemini for long context, Claude for code, gpt-oss for cheap batch processing," configuration cost drops noticeably.
However, it is not a replacement for Apigee AI Gateway—true cross-vendor direct connections, semantic caching, and token budgets still live on Apigee's heavier product line. This is the lightweight version, locked to Google-hosted models.
Pricing and limitations
Billing follows standard API Gateway pricing: 0–2M calls free, 2M–1B calls at $3.00/million, above 1B at $1.50/million; model tokens are billed separately via Vertex AI.
Hard limits: request timeout capped at 3600 seconds; SSE response streaming is supported, but request-side streaming, gRPC, WebSockets, Gemini Live, and VPC Service Controls are not. A known Preview defect acknowledged by Google: when the request body lacks a model field, the gateway fails silently into error handling rather than reporting a clear error.
There is no standalone official blog post—just Release Notes and documentation, a documentation-level low-key launch. Chinese reports claiming a chained flow with the Gemini enterprise agent platform (egress through Agent Gateway, then API Gateway dispatch) have no corresponding wording in the two official documents and should be treated as speculation.
Sources
- API Gateway Release Notes (2026-08-03): https://cloud.google.com/api-gateway/docs/release-notes
- Model routing overview: https://docs.cloud.google.com/api-gateway/docs/model-routing-overview
- Configure model routing: https://docs.cloud.google.com/api-gateway/docs/model-routing-configure
- Pricing: https://docs.cloud.google.com/api-gateway/pricing