Single unified API for 100+ LLM providers, also deployable as a team-wide proxy with virtual keys, spend tracking, and 8ms P95 latency at 1k RPS.
Before you start
pip install litellm • API keys for the providers you want to use
LiteLLM's completion() function has the same signature as the OpenAI SDK. Change the model string (e.g. 'anthropic/claude-sonnet-5-5', 'gpt-4o', 'groq/llama-3.3-70b') and it routes to the right provider automatically.
For multi-agent systems, run LiteLLM as a proxy server. All agents hit the same endpoint; LiteLLM handles routing, fallbacks, rate limiting, and spend tracking.
LiteLLM tracks spend per model and per virtual key. Configure budget thresholds to alert or block requests when spend exceeds a limit. Use router_strategy: cost-based to automatically route requests to the cheapest model that meets a quality threshold.
LiteLLM's SDK call signatures, proxy server commands, and routing/budget configuration.
| Command / Skill | What it does | When to use it |
|---|---|---|
litellm.completion(model=...) | Single call for all providers, change the model string to route to a different provider | Replace every direct provider SDK call with this, interface is identical to the OpenAI SDK |
Model string format | Provider-prefixed strings: 'anthropic/claude-sonnet-5-5', 'gpt-4o', 'groq/llama-3.3-70b', 'gemini/gemini-2.0-flash' | Switching providers, change only the model string, nothing else in your code changes |
fallbacks config | List of fallback models to try if the primary fails, configured in model_list or per-call | Any production agent, provider outages happen; automatic fallbacks prevent total failure |
litellm proxy startadvanced | Start the LiteLLM proxy, all agents call one endpoint, proxy handles routing and tracking | Multi-agent systems or teams needing centralized LLM access and per-project spend tracking |
Virtual keysadvanced | Per-project API keys with spend limits, issued by the proxy, not the provider | Team deployments: give each service its own key with a budget cap instead of sharing provider keys |
router_strategy: cost-basedadvanced | Auto-route requests to cheapest model that meets quality threshold | Mixed workloads with tasks of varying complexity, cut costs without degrading quality |