Best LLM Gateways in 2026: 8 AI Gateways Compared

Calling LLM providers directly works until it doesn't. The moment you're managing multiple API keys, tracking costs across providers, or scrambling during a provider outage, you need an LLM gateway: one API in front of every provider that handles routing, failover, keys, caching and cost tracking.

Not all LLM gateways are equal. Some are pure proxies. Some lock you into their ecosystem. Some charge you for features that should be free.

We evaluated eight LLM gateways (the vendors who say "AI gateway" mean the same thing) on what matters: provider coverage, pricing transparency, self-hosting, observability, and how fast you can get started. Updated October 11, 2026.

Best LLM gateways: the short version

The rest of this post goes through each one.

1. LLM Gateway

Best overall. Open source. Self-hostable.

LLM Gateway is an open-source API gateway that routes requests to 250+ models across 40+ providers through a single OpenAI-compatible endpoint. Change your base URL, keep your existing code.

What sets it apart:

• Open source (AGPLv3) — inspect the code, self-host on your own infrastructure, no vendor lock-in
• Zero markup on BYOK — bring your own provider keys and pay nothing extra on top of provider costs
• Built-in caching — Redis-powered response caching cuts repeat request costs to zero
• Real-time analytics — cost tracking, latency monitoring, and usage breakdowns per model, project, and API key
• Automatic failover — requests reroute to backup providers when the primary goes down
• Guardrails (Enterprise) — content safety rules with configurable filters
• AI SDK provider — first-class Vercel AI SDK integration via @llmgateway/ai-sdk-provider
• Lounge — chat, compare independent answers, bring models into a shared council debate, and generate images

Pricing: Free to sign up, with a rate-limited free model to start. Pay-as-you-go with no hidden fees. No markup when using your own keys.

Best for: Teams that want full control over their AI infrastructure without building it from scratch.

1import OpenAI from "openai";2
3const client = new OpenAI({4  apiKey: "YOUR_GATEWAY_KEY",5  baseURL: "https://api.llmgateway.io/v1",6});7
8// Works with any of 250+ models9const response = await client.chat.completions.create({10  model: "claude-sonnet-5-5",11  messages: [{ role: "user", content: "Hello!" }],12});

1import OpenAI from "openai";2
3const client = new OpenAI({4  apiKey: "YOUR_GATEWAY_KEY",5  baseURL: "https://api.llmgateway.io/v1",6});7
8// Works with any of 250+ models9const response = await client.chat.completions.create({10  model: "claude-sonnet-5-5",11  messages: [{ role: "user", content: "Hello!" }],12});

2. OpenRouter

Large model catalog, community-driven.

OpenRouter aggregates models from many providers and offers a unified API. It has a wide selection of models including open-source and fine-tuned variants.

Strengths:

• Large model catalog including community and fine-tuned models
• OpenAI-compatible API
• Usage tracking dashboard

Weaknesses:

• Not open source — you can't self-host or audit the code
• Charges a 5.5% fee on credit purchases (no per-token markup)
• BYOK is free up to $25,000 of list-price usage per month on pay-as-you-go, then a 5% fee
• Observability has improved (request logs, exports, OpenTelemetry) but lives behind its own dashboard
• Response caching is newer and still in beta

Pricing: Pay-as-you-go with a 5.5% credit-purchase fee; provider token rates pass through with no markup.

Best for: Developers who want quick access to a wide variety of models and don't need self-hosting. Looking to switch? See the 10 best OpenRouter alternatives in 2026.

3. Portkey

Enterprise-focused with governance features.

Portkey positions itself as an AI gateway for enterprises, with emphasis on observability, guardrails, and compliance features.

Strengths:

• Detailed request logging and traces
• Guardrails and content moderation
• Multi-provider routing with fallbacks
• Enterprise compliance features

Weaknesses:

• The gateway and much of the platform are open source (MIT), but persistent observability storage and compliance stay in the managed cloud
• Enterprise pricing can be opaque
• Heavier setup compared to simpler gateways
• Now part of Palo Alto Networks (acquired May 2026) — a consideration if you prefer an independent vendor

Pricing: Free tier with limited requests. Paid plans for higher volume and enterprise features.

Best for: Large enterprises with strict compliance and governance requirements. Re-evaluating after the Palo Alto Networks acquisition? See the 8 best Portkey alternatives in 2026.

4. LiteLLM

Open-source proxy with broad provider support.

LiteLLM is an open-source Python proxy that translates OpenAI-compatible requests to 100+ providers. It's popular as a self-hosted solution.

Strengths:

• Open source (MIT license)
• Wide provider support
• Active community
• Python-native

Weaknesses:

• Python-only — heavier runtime for deployment
• Built-in usage dashboard and spend logs, though deeper observability still leans on integrations
• No managed hosting — you handle infrastructure yourself
• Caching is built in, but you configure and run the backend yourself
• Less polished developer experience

Pricing: Free and open source. You pay for your own infrastructure.

Best for: Python teams comfortable with self-hosting and managing their own infrastructure. Outgrowing it? See the best LiteLLM alternatives.

5. Helicone

Observability-first with proxy capabilities.

Helicone started as an LLM observability platform and added gateway features. It excels at logging, monitoring, and cost tracking. Note: Helicone was acquired by Mintlify in 2026 and is now in maintenance mode rather than active development.

Strengths:

• Excellent request logging and analytics
• Easy integration (one-line header change)
• Cost tracking and alerting
• Prompt management features

Weaknesses:

• In maintenance mode since the 2026 Mintlify acquisition — not actively developed
• Gateway routing has historically been secondary to observability
• Not designed as a full gateway replacement
• Can become expensive at high request volumes

Pricing: Free tier with limited requests. Usage-based pricing beyond that.

Best for: Teams whose primary need is visibility into LLM usage rather than routing and failover. With the project in maintenance mode, see the 8 best Helicone alternatives in 2026.

6. Vercel AI Gateway

Zero markup, deep AI SDK integration.

Vercel AI Gateway routes to hundreds of models across 45+ providers through one endpoint, with both OpenAI- and Anthropic-compatible APIs. It went GA in 2025 and is the default provider for the Vercel AI SDK.

Strengths:

• Zero markup on tokens — including with your own keys (BYOK)
• First-class Vercel AI SDK integration (@ai-sdk/gateway)
• Automatic failover, provider routing, and automatic caching
• Observability and spend monitoring built in

Weaknesses:

• Not open source or self-hostable — managed cloud only
• Credits and BYOK are tied to a Vercel team account
• Some governance features (custom reporting, team-wide ZDR/allowlists) cost extra
• Strongest when you're already in the Vercel/Next.js ecosystem

Pricing: Pay-as-you-go credits with no token markup. Paid add-ons for custom reporting and team-wide governance.

Best for: Teams building on the Vercel AI SDK who want zero markup and tight ecosystem integration.

7. Cloudflare AI Gateway

Edge-based with Cloudflare ecosystem integration.

Cloudflare AI Gateway leverages their edge network to proxy and cache LLM requests. Tight integration with the Cloudflare ecosystem.

Strengths:

• Edge caching for low-latency responses
• Rate limiting and cost controls
• Simple setup if already on Cloudflare
• No per-request fees

Weaknesses:

• Limited to Cloudflare's supported providers
• Basic analytics compared to dedicated solutions
• Tightly coupled to Cloudflare ecosystem
• No BYOK — limited provider key management
• Fewer advanced routing features

Pricing: Free tier included with Cloudflare account. Paid plans for higher limits.

Best for: Teams already invested in the Cloudflare ecosystem who want basic gateway features.

8. AWS Bedrock

Cloud-native for AWS shops.

AWS Bedrock provides access to foundation models through AWS infrastructure. It's less of a traditional gateway and more of a managed model access layer within AWS.

Strengths:

• Deep AWS integration (IAM, VPC, CloudWatch)
• Enterprise security and compliance
• No infrastructure to manage
• Access to exclusive models (Amazon Nova)

Weaknesses:

• AWS lock-in — tightly coupled to the AWS ecosystem
• Limited model selection compared to dedicated gateways
• Complex pricing with multiple dimensions
• No OpenAI-compatible API — requires AWS SDK
• Slower to add new models from third-party providers

Pricing: Pay-per-token with AWS pricing. No upfront costs.

Best for: Organizations fully committed to AWS that need models within their existing cloud infrastructure.

Comparison Table

How to choose an LLM gateway

You want full control and no lock-in: LLM Gateway is the only option that's open source, self-hostable, and charges zero markup on your own keys. You get enterprise features without enterprise pricing.

You want the widest model selection: LLM Gateway and OpenRouter both offer extensive catalogs. LLM Gateway covers 250+ models with the added benefit of self-hosting and BYOK.

You want observability above all: Helicone is purpose-built for logging and analytics, though LLM Gateway and Portkey offer comparable dashboards with full gateway capabilities included.

You're locked into a cloud provider: AWS Bedrock makes sense if your entire stack is AWS and you need models within that boundary. Just know you're trading flexibility for integration.

You're on the Vercel AI SDK: Vercel AI Gateway is the natural fit with zero token markup — though you trade away self-hosting and bring-your-own-infrastructure options.

You're cost-conscious: Gateways that support BYOK (LLM Gateway, LiteLLM, Vercel) let you avoid middleman markup entirely. Over thousands of daily requests, the savings compound fast.

Getting Started with LLM Gateway

If you want to try the top pick, you can be running in under two minutes:

1. Sign up free — no credit card required
2. Create a project and copy your API key
3. Point your existing OpenAI SDK to https://api.llmgateway.io/v1

That's it. Your existing code works. Every request gets logged, cached, and tracked automatically.

Create a free account | Browse 250+ models | Read the docs