Customize timeouts for faster automatic failover on Vercel AI Gateway | Endigest
Vercel
Get the latest tech trends every morning
Receive daily AI-curated summaries of engineering articles from top tech companies worldwide.
Vercel AI Gateway now supports per-inference provider timeouts to enable faster automatic failover between AI providers.
- •Timeouts are configured per provider in milliseconds using the `providerTimeouts` field inside `providerOptions.gateway`
- •If a provider doesn't respond within the configured timeout, AI Gateway aborts the request and falls back to the next available provider
- •Currently in beta and only available for BYOK (Bring Your Own Key) credentials; system provider timeout support is coming soon
- •The `order` field can be combined with `providerTimeouts` to control both provider sequence and failover speed
- •Some providers don't support stream cancellation, so timed-out requests may still incur charges depending on the provider
This summary was automatically generated by AI based on the original article and may not be fully accurate.
Google Cloud
Databricks
Stripe