Cutting your inference bill

Two teams run the same feature on the same model. One's bill is ten times the other's. Neither is doing anything clever — one just turned a few knobs the other didn't.

Cost is a formula, not a mystery

Whether you call an API or self-host, the bill comes from the same place: how many requests, how many tokens in and out of each, and how expensive the model is per token. That's a formula you can estimate up front — and every term in it is a lever. Change the model tier or the token counts and the bill moves in a way you can predict before you ship.

The levers that actually move it

A few habits cut the bill hard. Right-size the model: use the smallest one that passes your quality bar, not the flagship by default. Trim tokens: shorter prompts and capped outputs cost less on every single call. Batch non-urgent work — batch processing trades speed for a lower rate. And cache: prompt caching reuses a repeated context instead of paying for it again. Stacked, these routinely cut a bill by most of its size.

The shape of it

• —The bill = requests × (input + output tokens) × the model's per-token price.
• —Right-size the model — the smallest one that passes your quality bar, not the biggest.
• —Trim tokens, batch non-urgent jobs, and cache repeated context.
• —Estimate the formula first, then pull whichever term dominates your bill.