The real math of LLM API fees

Every LLM gateway charges something on top of provider list price. That’s not a scandal — routing, uptime, and a single OpenAI-compatible endpoint across dozens of providers costs money to run. The problem is that most gateways don’t tell you the number, and the ones that do bury it behind a credit-purchase flow that changes the effective rate depending on how you pay.

This post walks through what a “gateway fee” actually is, shows the math on a real model, and lays out what’s publicly documented about how competitors structure theirs.

What a gateway fee actually is

When you call a model through a gateway instead of the provider directly, you’re paying for three things: the routing layer (picking a provider, failing over if one is down), the API surface (one schema instead of 56 different ones), and margin. The fee is usually expressed as a percentage markup on the provider’s per-token price, sometimes with a flat fee stacked on top.

The number that matters isn’t the percentage in isolation — it’s the percentage applied to your actual token mix, at your actual volume, compared honestly against provider list price. That’s the only apples-to-apples comparison, and it’s the one most pricing pages make hardest to do.

Worked example: Claude Sonnet 4.5

Anthropic’s list price for Claude Sonnet 4.5 is $3.00 per million input tokens and $15.00 per million output tokens. Through OpenKey, at a flat 3% fee, that becomes:

• Input: $3.00 → $3.09 per million tokens (+$0.09)
• Output: $15.00 → $15.45 per million tokens (+$0.45)

That’s it. No credit-purchase surcharge, no separate rate for “prepaid” vs. “pay as you go,” no markup that changes based on how much you’ve deposited. 3% of provider price, on every token, every time.

Scale that to a real workload — say 10M input tokens and 2M output tokens in a month, a reasonable size for a mid-volume agent or batch job:

• Provider cost: (10 × $3.00) + (2 × $15.00) = $60.00
• OpenKey cost: (10 × $3.09) + (2 × $15.45) = $61.80
• Fee paid: $1.80, exactly 3% of $60.00

You can run this same calculation for any of the 338 models we route — the per-model price with the fee applied is printed on every model page, and the full table is on /pricing.

What competitors document

We don’t have a rate card for every aggregator — most don’t publish one. Here’s what’s actually documented, with sources:

OpenRouter. OpenRouter’s own knowledge base describes a flat 5% markup on inference spend passed through at or near provider list pricing. A separate developer analysis (costbench.com, cited in industry research) puts the real number at “5% markup plus $0.35” on credit purchases — meaning the effective rate depends on how you top up, not just what you spend. Route through a cloud marketplace like AWS Bedrock, Azure, or Vertex on top of that and you can add another 10-20%. There’s no single canonical fee sheet that reconciles these numbers; OpenRouter’s pricing page itself has shown a “Platform Fees” row rendering “N/A” with the real figure injected client-side rather than printed as static text.

Run our $60.00 Sonnet 4.5 workload through a flat 5% markup and you get $63.00 — $1.20 more than OpenKey’s $61.80, before any credit-purchase surcharge or marketplace pass-through is added.

Together AI and Fireworks AI. These aren’t fee-on-top-of-a-single-provider gateways — they’re infrastructure providers hosting open-weight models on their own hardware, which means the same model can carry a different price on each platform depending on the host’s infrastructure costs and margin. The clearest documented example: identical Llama 4 Maverick weights price roughly 20% higher on Together AI than on Fireworks AI for the same model, per an analysis from tokenmix.ai. That’s not a routing fee in the OpenRouter sense — it’s price layering baked into how each infra provider sets its own rate card. The practical effect for a developer is the same, though: the same model name doesn’t mean the same price, and you have to check both to know which one you’re actually paying.

LiteLLM. Worth naming because it’s the zero-fee option: it’s an open-source proxy you self-host, so you pay providers directly at list price with no markup at all. The cost you take on instead is operational — you own deployment, scaling, monitoring, and patching, with no managed uptime SLA unless you pay for the enterprise tier.

The comparison that matters

None of this makes 5% wrong or 3% automatically right — a gateway is worth paying for if it saves you more in integration and reliability than the fee costs. But you can’t evaluate that trade-off if the fee itself is a moving target based on how you pay, or hidden behind cloud-marketplace pass-through, or simply undocumented.

Our position is narrow: provider price plus 3%, computed on every request, shown on every model page and every comparison page, with no separate rate for credit purchases. If you want to check the math yourself before committing volume, the full price table is at /pricing and free models with no fee at all are at /models/free.