How to test AI models without paying for API calls is essential once AI sits in your CI pipeline. Uncontrolled tests can burn more tokens than production.
Layer 1: Unit tests with mocks
Replace the HTTP client in tests:
- Return fixed JSON for chat completions
- Simulate streaming with chunked strings
- Assert your parsing, not model creativity
You validate wiring without invoking a model.
Layer 2: Record/replay fixtures
Capture golden responses once, replay in CI:
- Refresh fixtures when prompts change intentionally
- Store fixtures without secrets or PII
Layer 3: Local models for integration tests
Run Ollama or llama.cpp in CI (self-hosted runners with GPU optional). Slower, but $0 API. See Run AI locally.
Layer 4: Staging keys with hard budgets
If you must hit cloud:
- Separate low-quota keys
- Daily spend alerts
- Feature flags to disable AI in preview branches
Layer 5: Shadow mode before cutover
When migrating providers, dual-write and compare outputs offline (migration guide).
What not to do
- Point CI at production keys without limits
- Retry aggressively in tests (multiplies cost)
- Log full prompts to third-party analytics
Security: Managing API keys.
When paid testing is worth it
Before launch, run one paid eval window on real models — especially if you sell accuracy. Free mocks will not catch frontier-only failures.
Flat-rate plans like Daymora ($25/month) remove invoice panic during heavy manual QA — see best API for dev and testing.
Summary
Testing without API calls means mocks in CI, local models for integration, cloud only for eval windows. Your wallet and your on-call engineer both win.