AI APIs fail differently than CRUD databases. Models slow down under load, gateways return 429s, streams stall mid-sentence, and clients retry the same user message three times because the UI did not clear loading state. Robust error handling is what separates a demo from a product.
This playbook complements Building production-ready AI SaaS features and applies regardless of whether you bill per token or on a flat-rate plan.
Error categories
| Signal | Typical cause | User-facing behavior |
|---|---|---|
| 429 | Rate limit (yours or upstream) | Short retry + "busy" copy |
| 408 / timeout | Long generation or network | Offer shorter answer or retry |
| 400 | Bad model name or schema | Fix server validation; never leak raw upstream body |
| 5xx | Provider outage | Graceful degradation, status link |
| Stream abort | Client navigated away | Stop upstream reader; do not retry automatically |
Retry policy
Retry: idempotent server-side failures (502/503) with exponential backoff and jitter. Cap at 2–3 attempts.
Do not blindly retry: 400s, most 401/403, or completed partial streams unless you dedupe user messages.
async function withBackoff<T>(fn: () => Promise<T>, attempts = 3): Promise<T> {
let delay = 500;
for (let i = 0; i < attempts; i++) {
try {
return await fn();
} catch (e) {
if (i === attempts - 1) throw e;
await new Promise((r) => setTimeout(r, delay + Math.random() * 200));
delay *= 2;
}
}
throw new Error("unreachable");
}Timeouts
Set two timeouts:
1. Connect / first byte — fail fast if the API is wedged
2. Total generation — cancel streams that exceed product limits
On cancel, call AbortController on fetch and close the client stream.
User experience
- Show actionable errors ("Try a shorter question") vs raw JSON
- Preserve the user's message when retrying
- Log a correlation id across proxy → upstream for support tickets
Rate limits and your budget
Retries multiply token usage on metered APIs. A triple retry on a 3k-token prompt is 9k billed tokens. Flat-rate APIs remove invoice panic but you should still cap retries to protect latency. See true cost of AI APIs.
Observability minimum
Track: ai.request.count, ai.request.duration_ms, ai.error.count by status, ai.tokens.in/out if exposed. Alert on error rate spikes, not every 429.
Security on errors
Do not forward provider error bodies to browsers — they may mention account ids or internal routing. Map to stable error codes in your API.
Summary
Treat AI calls like unreliable external dependencies: classify errors, retry only what is safe, bound timeouts, and design UI that fails calmly. Your users will not remember a perfect first token; they will remember whether the product recovered.