Gateways & routers
LiteLLM
Python SDK and self-hostable proxy that puts a consistent interface over 100+ model APIs, with budgets and rate limits per key.
- Category
- Gateways & routers
- Pricing
- Open source
- Runs
- Hosted or self-hosted
- Languages
- Python
- Interface
- Library, Proxy server
- Source
- Open source
What LiteLLM is
LiteLLM is both a Python library and a standalone proxy you run yourself. As a library it normalises calls across providers; as a proxy it becomes a gateway for any language, issuing virtual keys with their own budgets and rate limits. Because you host it, your provider keys and your uptime stay inside your infrastructure.
Best for
Teams that want gateway behaviour — budgets, key management, failover — without adding a third-party service to the request path.
Consider something else if
Self-hosting means you operate it, monitor it, and keep it current with provider API changes.
LiteLLM alternatives
The closest options in gateways & routers, on the axes that actually separate them.
| Tool | Best for | Pricing | Runs |
|---|---|---|---|
| LiteLLMopen source | Teams that want gateway behaviour — budgets, key management, failover — without adding a third-party service to the request path. | Open source | Hosted or self-hosted |
| OpenRouter | Trying many models quickly, or shipping something that must keep working when a single provider degrades. | Usage-based | Hosted |
| Portkeyopen source | Teams that want routing and request-level tracing from one system instead of integrating a gateway and an observability tool separately. | Free tier | Hosted or self-hosted |
| Cloudflare AI Gateway | Adding caching and cost visibility to calls you already make, especially if you are on Cloudflare already. | Free tier | Hosted |
Choosing within gateways & routers
Hosted or self-hosted
A hosted gateway is another service in your critical path — when it is down, you are down. A self-hosted proxy keeps the failure domain yours but is now yours to run. Some options offer both; pick before you build around one.
What it costs you in latency
Every gateway adds a network hop. That is usually single-digit milliseconds and irrelevant next to generation time, but it matters for high-volume classification where the model call itself is short.
Questions
What is LiteLLM?
LiteLLM is both a Python library and a standalone proxy you run yourself. As a library it normalises calls across providers; as a proxy it becomes a gateway for any language, issuing virtual keys with their own budgets and rate limits. Because you host it, your provider keys and your uptime stay inside your infrastructure. It is open source and hosted or self-hosted.
What are the alternatives to LiteLLM?
The closest alternatives are OpenRouter, Portkey, Cloudflare AI Gateway. They sit in the same category — gateways & routers — and differ mainly on hosting model, pricing shape, and how much they abstract away.
Is LiteLLM the right choice?
Teams that want gateway behaviour — budgets, key management, failover — without adding a third-party service to the request path. The main caveat: Self-hosting means you operate it, monitor it, and keep it current with provider API changes.
Whatever you build on, the model is the line item that scales. See what each one costs per million tokens, or price your own workload.