6 Best Platforms for LLM Cost Management and Optimization 

TL;DR 

• Annual AI budgets now run out in months. Uber had used up its 2026 AI budget by April, so cost control has to happen per call. 
• Most companies have AI cost policies; very few can see the spend those policies are meant to cover. 
• Grade every platform on three jobs: see who spent it, stop spend before the call, shrink the cost of each finished task. 
• Lyzr OpenController does all three per agent, inside your network, and pays frontier prices only when a cheaper answer fails. 
• Portkey and Cloudflare AI Gateway put hard budgets on the gateway. Datadog and Langfuse show cost on every trace. CloudZero translates spend for the CFO. 
• Start at the gateway. Add a FinOps tool when finance asks what each customer costs to serve. 

In December 2025, Uber gave its engineers Claude Code. In mid-April, its CTO told The Information that the full-year AI budget was already blown. On June 2, Uber capped spend at $1,500 a month per employee for each agentic coding tool, Bloomberg reported. 

The cap made sense, but it arrived four months late. A number set in 2025 planning met a workload whose appetite changes weekly. LLM cost management and optimization is the work of closing that gap: knowing what each call costs, refusing the calls you didn’t budget for, and paying less for the ones you did. 

Why the forecast keeps losing 

Uber isn’t an outlier. In Harness’s 2026 State of AI in FinOps, a survey of 700 engineering leaders, 72% had hit a surprise AI bill in the past year, and 73% have AI cost policies while only 13% have basic visibility into AI spend. Over half say no one clearly owns the bill, and respondents estimate 26% of AI spend is wasted. 

Everyone is on the hook: the FinOps Foundation’s State of FinOps 2026 found 98% of practitioners now manage AI spend, against 31% in 2024. Few of them can forecast it: Mavvrik’s 2026 State of AI Cost Governance found 89% of organizations miss AI forecasts by more than 10%. 

Agents make prediction harder still. A Stanford Digital Economy Lab study found that runs of the same coding task can differ by up to 30x in tokens, that input tokens drive most of the cost, and that models underestimate what their own runs will cost. An average cost per agent is a guess with a decimal point. 

Three jobs, and most tools do one 

See: Every call carries a name. Otherwise cost attribution is a monthly argument. Demo test: pick yesterday’s most expensive request and ask who made it. 

Stop: A budget that blocks the call before it reaches the provider. An alert afterwards is just a report. Demo test: set a $1 cap, fire twenty concurrent requests, count what got through. 

Shrink: Lower cost per finished task: smaller models first, cached repeats, leaner context. Input tokens carry most of an agent’s bill, so context is where the fat sits. Demo test: show cost per task before and after one change. 

Gateways can stop and shrink because every request passes through them. Observability and FinOps tools only see. Buy for the job you’re failing at. 

The 6 best platforms for LLM cost management and optimization 

1. Lyzr OpenController: a budget per agent, enforced on the request path 

OpenController launched in September as a control plane for enterprise agents, with one view of agent usage, model and infrastructure costs, and spend across developers, projects and agents. Agents stay on their current runtimes, and OpenController checks identity, policy and budget before each call lands. The LLM gateway issues virtual keys with team budgets and rate limits, sets which models each agent may call, and handles routing, fallback and caching across providers, locally hosted models included. 

The shrink lever is small-model routing. A task goes first to an open-weight model, and the gateway checks the answer against a quality bar you define: rules, a classifier or other validation. Failures go, unmodified, to a frontier model, and the trace records each model tried and the token cost of every step. You pay frontier prices only after a cheaper answer fails. 

It deploys in your VPC, on-prem or air-gapped, and provider credentials never need to reach Lyzr. 

Buy it for: agent programs where spend has to follow agents and teams across several providers.  

Think twice if: you need revenue-per-customer reporting; pair it with a FinOps tool). 

2. Portkey (Prisma AIRS AI Gateway): hard caps on every key 

Palo Alto Networks closed its acquisition of Portkey on May 29, making it the AI gateway inside Prisma AIRS, Palo Alto announced. Portkey’s docs describe budget limits on API keys and provider integrations, in dollars or tokens, with weekly or monthly resets and threshold emails; workspace budgets cover teams. Two details: a limit counts only requests made after it’s set, and changing a provider’s limit means duplicating the provider. In March, Portkey moved budget limits and semantic caching into its open-source gateway, the company announced. 

Buy it for: teams that want a mature gateway with caps per key, workspace and provider.  

Think twice if: you’re on a lower tier. Hosted budget limits are Enterprise-only, plus select Pro customers. 

3. Cloudflare AI Gateway: dollar caps at the edge 

Cloudflare shipped spend limits on June 5, in open beta on every plan, the company announced. Per Cloudflare’s docs, budgets are set in dollars and scoped by model, provider or custom metadata such as user or team; once a limit is hit, requests get a 429 until the window resets, with up to 20 rules per gateway. Limits are eventually consistent, so a concurrent burst can briefly overshoot. Since August, Cloudflare Access can attach a verified identity to each request, and Cloudflare says identity budgets can downgrade a user to a cheaper model instead of blocking. 

Buy it for: the fastest hard cap you can turn on, if Cloudflare already fronts your traffic.  

Think twice if: prompts must stay inside your network. It’s a hosted service. 

4. Datadog Agent Observability: cost beside the APM trace 

Formerly LLM Observability, Datadog’s product estimates the cost of each LLM request from public pricing and token counts for more than 800 models, tagged by model, provider and application, per Datadog’s docs. Its pricing page lists a free plan with 40,000 LLM spans a month and Pro from $160 for 100,000. Unsupported providers need costs sent manually, and every figure is a list-price estimate, not your invoice. 

Buy it for: Datadog shops that want token cost on the same screen as latency and errors.  

Think twice if: you need something to refuse a call. It watches; it doesn’t block. 

5. Langfuse: open-source cost per trace 

ClickHouse bought Langfuse in January, alongside its own $400 million raise, ClickHouse announced, and Langfuse says the core stays MIT-licensed and free to self-host. Cost sits on the same trace as evaluations and prompt versions, so when a prompt change halves cost, you can check whether quality halved too. 

Buy it for: engineering teams that want cost data next to evals, on their own servers.  

Think twice if: you need enforcement. Langfuse sits beside the request path, not in it. 

6. CloudZero: AI spend in the CFO’s language 

CloudZero launched its “financial control plane for AI” on May 28, tying spend to customers, features and outcomes. CloudZero lists native Anthropic and OpenAI integrations plus Bedrock, Vertex AI and Azure OpenAI support, and its September AI Signals release attributes spend by provider, model, person, team and work activity. 

Buy it for: finance teams that want AI spend in the same ledger as cloud, priced per customer.  

Think twice if: you expect a “control plane” to stop a request. This one can’t; put a gateway in front. 

Six platforms on one grid 

✓ built in · ◐ partial, paid tier or app-dependent · ✗ not offered. Portkey and Cloudflare route without a quality check. Cloudflare’s per-user attribution needs Access. Editorial assessment, November 2026. 

Gateways own the middle three columns; the finance tool owns the sixth. Our view: the gateway comes first, because a saving you can’t enforce is just a forecast. 

One invoice agent at month-end, priced call by call 

Take an accounts-payable agent whose traffic triples in the last week of each month: 

1. Name it. Its own virtual key and monthly budget, owned by a person in finance. 
2. Try small. An open-weight model extracts the fields; a validator checks that line items add up. Mismatches go to a frontier model. 
3. Skip repeats. Lookups of the same vendor payment terms come from cache. 
4. Agree the brake. Decide in advance what happens at the cap: block, or a human approves more. 
5. Report. Each trace carries the cost per invoice to the finance tool, next to invoices processed. 

Watch cost per invoice, escalation rate and cache hit rate. Month-end should triple volume, not cost per invoice; if that moves, the trace shows which prompt or model changed. That’s cutting LLM spend without changing models. 

100+ Agentic Use Cases for CFOs 106 finance use cases, each with an ROI estimate. Calculate your ROI → 

Budget per call, review per quarter 

Uber’s $1,500 rule could have been a gateway policy in January, visible from week one. Our bet for 2027 planning: annual numbers stay, enforced one request at a time, with a name on each budget. 

Start at the gateway. Then find the agents calling providers with their own keys; no budget covers traffic it never sees. Book a demo and we’ll put a per-agent budget on a slice of your own traffic through OpenController. 

FAQs 

Related reading: Open-source LLM observability platforms · Best tools for real-time AI agent monitoring · Best tools for AI policy enforcement at scale 

Book A Demo: Click Here
Join our Slack: Click Here
Link to our GitHub: Click Here