Skip to content
Agentic Levels
Start hereLevelsTracksPromptsAbout
Get my levelGet my level

  • Start hereYour first useful task with AI.
  • Level quiz8 questions. No AI needed.
  • Fluency ScoreYour Level and a score, from your own AI.

  • LevelsThe 11 levels, in order.
  • LessonsShort lessons, filter by level.
  • TracksGuided paths by tool, role or goal.
  • Set up your AI tool
  • Starter packFree files that help AI remember.

  • PromptsReady prompts to copy and adapt.
  • ChatGPT vs Claude vs GeminiPick the right one for the job.
  • How-tos
  • ReferenceCommands for Claude Code, Codex, Antigravity.
  • ResourcesHand-picked reading and tools.

  • Your progress
  • Your next moveAn interview that picks your next skill.
  • Your next lessonsThe next lessons at your level.
  • Updates

  • Pricing
  • About
  • FAQ & Feedback
  • Work with us
  • How we handle your data
Sign inPreferences
AboutPricingHow-tosPromptsStarter packMy stackHow we handle your data

© 2026 Fuentes Studio

PrivacyTermsRefundsAccessibilityAI content policyCredits
yourCouncil
Ready to help
✦

What do you want to understand?

Ask anything about what you're learning.

←How-to guides
  1. Resources
  2. ›How-to guides

How to use LiteLLM

Single unified API for 100+ LLM providers, also deployable as a team-wide proxy with virtual keys, spend tracking, and 8ms P95 latency at 1k RPS.

Levels 5 to 7On GitHub

Your orchestrator is coupled to one provider. You want to swap models, add fallbacks, or route cheaper tasks to cheaper models, without rewriting your LLM call layer. LiteLLM solves this as both a Python SDK (single litellm.completion() call for all providers) and a proxy server (centralized LLM gateway for teams with virtual key management, per-project spend tracking, load balancing, and admin dashboard). Achieves 8ms P95 latency at 1k RPS. Used in production by Stripe, Google, and Netflix.

Before you start

pip install litellm • API keys for the providers you want to use

  1. 1Replace your direct provider call with litellm.completion()
  2. 2Set up the LiteLLM proxy for a team or daemon
  3. 3Add cost-based routing and budget alerts

Steps

01

Replace your direct provider call with litellm.completion()

LiteLLM's completion() function has the same signature as the OpenAI SDK. Change the model string (e.g. 'anthropic/claude-sonnet-5-5', 'gpt-4o', 'groq/llama-3.3-70b') and it routes to the right provider automatically.

Set up your model routing config
I'm currently using [PRIMARY_MODEL] for [USE_CASE]. I want to add: (1) a fallback to [FALLBACK_MODEL] if the primary fails, (2) routing cheaper tasks to [CHEAP_MODEL]. Show me the LiteLLM configuration for this setup including fallback logic and cost-based routing.
[PRIMARY_MODEL]Your primary model e.g. anthropic/claude-sonnet-5-5
[USE_CASE]What you're using it for e.g. multi-step code generation with tool use
[FALLBACK_MODEL]Fallback model e.g. gpt-4o
[CHEAP_MODEL]Cheap model for simple tasks e.g. groq/llama-3.3-70b
02

Set up the LiteLLM proxy for a team or daemon

For multi-agent systems, run LiteLLM as a proxy server. All agents hit the same endpoint; LiteLLM handles routing, fallbacks, rate limiting, and spend tracking.

Configure the proxy
I want to run LiteLLM as a proxy for my multi-agent system. My agents need: [REQUIREMENTS]. Write the config.yaml for the proxy with: (1) model routing for [MODELS], (2) fallback logic if the primary fails, (3) per-project virtual keys with spend limits. Include the docker or CLI command to start it.
[REQUIREMENTS]Your proxy requirements e.g. 3 services sharing the same LLM endpoint, each with a $50/month budget cap
[MODELS]Models to route across e.g. anthropic/claude-sonnet-5-5 (primary), gpt-4o (fallback), groq/llama-3.3-70b (cheap tasks)
03

Add cost-based routing and budget alerts

LiteLLM tracks spend per model and per virtual key. Configure budget thresholds to alert or block requests when spend exceeds a limit. Use router_strategy: cost-based to automatically route requests to the cheapest model that meets a quality threshold.

Set up spend tracking and alerts
I have a LiteLLM proxy running with these virtual keys: [VIRTUAL_KEYS]. Set up: (1) budget limits of [BUDGET] per key per month, (2) an alert (email or webhook to [ALERT_ENDPOINT]) at 80% of budget, (3) automatic fallback to a cheaper model when the budget limit is hit. Show the config.yaml changes.
[VIRTUAL_KEYS]Your virtual key names e.g. api-service, background-workers, research-agent
[BUDGET]Budget per key e.g. $50
[ALERT_ENDPOINT]Alert endpoint e.g. https://hooks.slack.com/...

How do you know it worked?

  • At least two providers working through the same LiteLLM call
  • Fallback fires correctly when the primary provider fails
  • Cost tracking visible in LiteLLM logs

If something looks off

Test provider-specific params
Model parameter names differ across providers, LiteLLM normalizes most but not all; always test your specific params.
Run the proxy close to compute
The proxy adds a network hop. Run it locally for dev, collocated with your compute for production.

Skills & Commands Reference

LiteLLM's SDK call signatures, proxy server commands, and routing/budget configuration.

Command / SkillWhat it doesWhen to use it
litellm.completion(model=...)Single call for all providers, change the model string to route to a different providerReplace every direct provider SDK call with this, interface is identical to the OpenAI SDK
Model string formatProvider-prefixed strings: 'anthropic/claude-sonnet-5-5', 'gpt-4o', 'groq/llama-3.3-70b', 'gemini/gemini-2.0-flash'Switching providers, change only the model string, nothing else in your code changes
fallbacks configList of fallback models to try if the primary fails, configured in model_list or per-callAny production agent, provider outages happen; automatic fallbacks prevent total failure
litellm proxy startadvancedStart the LiteLLM proxy, all agents call one endpoint, proxy handles routing and trackingMulti-agent systems or teams needing centralized LLM access and per-project spend tracking
Virtual keysadvancedPer-project API keys with spend limits, issued by the proxy, not the providerTeam deployments: give each service its own key with a budget cap instead of sharing provider keys
router_strategy: cost-basedadvancedAuto-route requests to cheapest model that meets quality thresholdMixed workloads with tasks of varying complexity, cut costs without degrading quality

See also

  • Dispatch: multi-model task routing
  • Ruflo: swarm orchestration

What you'll do, 3 steps

  1. 1Replace your direct provider call with litellm.completion()
  2. 2Set up the LiteLLM proxy for a team or daemon
  3. 3Add cost-based routing and budget alerts

Was this helpful?