6 Best Tools for Tracking Token Usage and Cost in LLMs 

TL;DR 

• Two-thirds of large US companies have an AI cost dashboard. Only 26% see what AI costs to run in real time. 
• Tokens get counted in three places: the request path, the trace and the invoice. They rarely agree. 
• Lyzr OpenController counts every governed call by agent, developer and project, across providers and self-hosted models, inside your network. 
• Kong Konnect turns tokens into billable events. LangSmith, Langfuse and Grafana Cloud put cost on traces. Vantage reads what providers billed. 
• Pick one meter, one trace store and one invoice source, and reconcile them monthly. 

In KPMG’s Q2 2026 AI Quarterly Pulse, 66% of large US companies had AI cost dashboards. Only 26% reported full, real-time visibility into what their AI systems cost to operate. 

Both numbers hold because a dashboard shows whatever reaches it. A chart of spend by provider is easy. Saying which agent spent it, on which kind of token, at which price, is the hard part, and it’s what tracking token usage and cost in LLMs actually means. This guide compares six tools on three questions: who made the call, what it cost, and whether your figure matches the invoice. 

Why your token count is probably wrong 

Start with names. Accenture’s CIO guide to AI tokenomics, a mid-2026 survey of 750 executives at $1 billion-plus companies, found only 53% of token usage traceable to a user or team. One in three organizations runs through its token budget before year-end. And it pays to fix: in KPMG’s Global AI Pulse, firms with full cost visibility were five times as likely to report established ROI (15% against 3%). 

Then there’s the count itself. Reasoning models bill hidden thinking tokens at output rates. In a paper from Stanford, UC Berkeley, CMU and Microsoft Research, revised in May 2026, the model with the lower list price cost more in 32% of model pairs, by up to 28x, and one model used 900% more thinking tokens than another on the same query. Langfuse’s token and cost tracking docs add that it can’t infer cost for reasoning models unless token counts are sent, because those tokens never appear in the text. 

What Separates Production AI Deployments from the Pilots That Preceded Them: A measured look at task success, evaluation, model dependency and expansion economics. Read the white paper → 

What tracking token usage and cost in LLMs should record 

Every call should leave one record with five fields, written when the call happens, not pieced together at month-end. 

• Who: An agent, user, team or customer ID. Without it, cost attribution is a spreadsheet exercise. 
• Which model, and where: Provider, version and region, plus self-hosted models, which cost GPU hours instead of per-token fees. 
• Which tokens: Input, cached input, output and reasoning, as the provider reported them. Cached input is usually discounted; reasoning bills as output. 
• At what price: Your contract rate. Most tracing tools use a public price table; your invoice doesn’t. 
• For what: The ticket or workflow the call served, so spend divides by finished work. 

These records live in one of three places. A gateway counts every call it proxies and can refuse one. A trace pins cost to the step that spent it. The invoice, read through each provider’s cost API, says what you’ll pay, often a day late and with little context. Each tool below is strongest in one. 

The 6 best tools for tracking token usage and cost in LLMs 

1. Lyzr OpenController: the meter on every agent’s request path 

OpenController launched on September 17 with one view of agent usage, model and infrastructure costs, and spend by developer, project and agent, including which agents call which models and where those models run. Its discovery mode brings agents already running into that view, without rebuilding them. 

Because the count happens in the LLM gateway, the name is attached at the source. The product page lists virtual keys with team budgets, and per Lyzr’s comparison with LiteLLM, identity, policy and budget are checked before a call lands. Models on vLLM or Ollama sit behind the same proxy with a single trace, so GPU-hosted and frontier models share one report. Lyzr doesn’t resell model APIs or touch your provider billing. It deploys in your VPC, on-prem or air-gapped. 

Counts: every call through its gateways, by agent, developer and project.  

Blind spot: traffic that bypasses it, and the provider’s billed figure. Pair it with an invoice source.  

Pick it if: agents span several providers and teams, and you want a meter that can also say no. 

2. Kong Konnect Metering & Billing: tokens turned into invoices 

Kong bought OpenMeter in September 2025 and folded it into Konnect, Kong announced. Per Kong’s Metering & Billing plugin docs, metering runs in the gateway request path and emits immutable usage events, kept apart from observability data. The billable subject can be a consumer, a portal app or a header such as x-tenant-id. Kong’s AI Gateway policy emits separate input and output token events with department or project dimensions, which Konnect prices against rate cards. 

Counts: tokens per consumer, ready to invoice.  

Blind spot: the meter itself caps nothing; blocking at an allowance takes Kong’s separate Entitlement Enforcement plugin. Metering is an Enterprise-only plugin.  

Pick it if: you bill customers for AI features or run formal chargeback, and Kong already fronts your APIs. 

3. LangSmith: cost across the whole agent run, not just model calls 

Since December 2025, LangSmith records token usage and derived cost for major providers and accepts custom costs for tools and retrieval, per LangChain’s changelog. Costs roll up through the trace tree and dashboards, and May 2026 releases added agent and user cost tables. Its LLM Gateway, in public beta on Plus and Enterprise plans, adds spend caps by organization, workspace, API key or user. 

Counts: cost per step, tools and retrieval included.  

Blind spot: LangSmith’s docs say its computed spend comes from configured model prices, not provider billing.  

Pick it if: you build on LangChain or LangGraph and want cost beside debugging. 

4. Langfuse: open-source cost per generation, with pricing tiers 

Per Langfuse’s docs, it records usage and cost for every LLM call, sent from the response or inferred from a model definition, and sent values win. Pricing tiers cover models that charge more above a context threshold, and custom definitions cover fine-tuned and self-hosted models. A Metrics API returns usage and cost by user or tag. Langfuse says the core stays MIT-licensed and free to self-host after ClickHouse bought it in January 2026. 

Counts: cost per generation, user, session and tag.  

Blind spot: inferred costs are fixed at ingestion, so price updates only apply to new generations.  

Pick it if: cost data must live on your servers, next to evals. 

5. Grafana Cloud AI Observability: token cost as one more metric 

Grafana launched AI Observability in public preview at GrafanaCON on April 21, the company announced. It runs on the open-source OpenLIT SDK, which sends OpenTelemetry traces and metrics, token counts and cost included, into prebuilt dashboards beside latency and GPU metrics. Per Grafana’s pricing docs, billing for Agent Observability generations began on October 1. 

Counts: tokens and estimated cost per application and environment.  

Blind spot: attribution goes only as deep as the attributes you send, and OpenTelemetry still marks its GenAI conventions as Development.  

Pick it if: your SREs already live in Grafana. 

6. Vantage: the number the provider actually billed 

Per Vantage, it reads cost through Anthropic’s Usage and Cost API with a read-only Admin key, beside AWS, Azure, OpenAI and 20-plus other providers. In July it added Claude Enterprise: daily per-user cost across Claude Code, Cowork and Chat, by product, model and user. That catches seat-based coding tools that never touch your gateway. 

Counts: billed cost by provider, model and, where exposed, user.  

Blind spot: daily granularity, and no idea which agent or prompt made a call.  

Pick it if: finance needs an invoice-grade number to reconcile against. 

Six tools on one grid 

✓ built in and documented · ◐ partial, beta, paid tier or separate component · ✗ not offered. OpenController offers showback, not invoicing. Editorial assessment from public docs, November 2026. 

No row fills both “request path” and “billed cost.” So run a meter and an invoice source, and reconcile them. 

Reconciling one month of token spend 

Take a platform team with 30 agents on two providers and one self-hosted model, plus coding assistants for 200 developers. (Illustrative, not a customer case.) 

1. Name every caller. One virtual key per agent, each with an owner. Coding tools get per-user analytics from the provider. 
2. Keep the provider’s counts. Store usage from each response, split by token type. 
3. Load the invoice daily. Pull each provider’s cost API into your FinOps tool. 
4. Reconcile on the fifth. Compare gateway totals with billed totals per provider. A gap above a few percent means a stale price table or traffic on keys the gateway never saw. 
5. Show, then charge. Send each owner a monthly showback with cost per finished task. After a quarter, move the biggest consumers to chargeback. 

Accenture found only 7% of companies use chargeback to track AI costs, and called it the strongest predictor of cost optimization and value attribution in its research. Once owners see their own line, cutting spend without changing models is an easier sell. And expect step 4 to find agents nobody registered. 

Agentic AI Maturity Assessment: 16 questions, five minutes: see how ready you are to run and govern agents in production. Take the assessment → 

Buy the meter before the dashboard 

Dashboards are cheap. What’s scarce is a count taken at the moment of the call, with a name on it. Get that right and the trace and invoice tools have something to agree with. Get it wrong and every chart inherits the gap. 

If one framework and one provider carry nearly all your traffic, your tracing tool may be enough. Once agents span teams, providers and your own GPUs, put a meter on the request path first. Book a demo and we’ll run a slice of your traffic through OpenController and show you who spent it. 

FAQs 

Related reading: Open-source LLM observability platforms · Best tools for real-time AI agent monitoring · Best tools for AI policy enforcement at scale 

Book A Demo: Click Here
Join our Slack: Click Here
Link to our GitHub: Click Here