Appearance
AI Gateway
The AI Gateway is the central execution layer. It decides which provider to use and applies your routing and safety rules.
Why it matters
- You can change providers without changing your app.
- You can control cost, speed, and quality with policies.
- You get consistent logging and analytics.
Typical workflow
- Define a routing policy.
- Send a prompt through the gateway.
- Review cost, latency, and provider results.
Automatic provider failover
If an AI provider can't deliver a usable answer, the gateway doesn't dead-end — it automatically fails over to the next provider in the pool and keeps your conversation going. Failover triggers when a provider:
- returns an empty response,
- times out, or
- fails with an auth or billing error.
You're told why the switch happened, in plain language. For example:
The AI provider OpenAI appears unresponsive because it returned an empty response — switching to Anthropic to keep your conversation going.
Empty responses now count as failures. A provider can return a "successful" 200 reply with no actual content — the classic symptom of an expired key, an exhausted balance, or an over-aggressive content filter. Previously a blank reply like this dead-ended as an empty or error message. Now it's treated as a failure that triggers failover, so you get a real answer from another provider instead of a blank screen.