A customer asks for a refund. A router agent passes it to a billing agent, which calls a payments tool on a separate MCP server, and a compliance agent in another service checks the result. The answer arrives 40 seconds late and costs six times what it should. Your logs show four healthy services. None of them shows the run.
Distributed tracing closes that gap, and it’s harder with agents because the path isn’t fixed: three hops today, nine tomorrow. Anthropic’s engineering team, after building its own multi-agent research system, put it plainly in its write-up: “Adding full production tracing let us diagnose why agents failed and fix issues systematically.”
This guide compares six tools on what breaks first with agents: carrying context across boundaries, showing who handed what to whom, and acting once a trace exposes a problem.
Get the short answer
- Need traces tied to agent identity, with policy enforced on what they reveal? Pick Lyzr.
- Want open-source tracing you host yourself, over OTLP? Pick Langfuse.
- Running LangChain or LangGraph across several services? Pick LangSmith.
- Want traces and evals in one self-hosted tool? Pick Arize Phoenix.
- Already trace microservices with OpenTelemetry and want a neutral backend? Pick Jaeger.
- Need agent traces next to your existing APM data? Pick Datadog LLM Observability.
What is distributed tracing in multi-agent systems?
Distributed tracing follows one request across every process it touches by passing a trace ID along. In a multi-agent system, each agent invocation, model call and tool call becomes a span, and the boundaries between them are often real network hops: one agent calling another over HTTP, or a tool behind an MCP server.
The usual carrier for that ID is the traceparent header from the W3C Trace Context standard. If any hop drops it, the trace splits in two and the downstream agent’s work shows up as an orphan with no parent.
OpenTelemetry now has a vocabulary for agents. Its GenAI semantic conventions define span operations for create_agent, invoke_agent, execute_tool, invoke_workflow and plan, plus attributes such as gen_ai.agent.id and gen_ai.conversation.id that tie each span to a specific agent and session. It’s all still marked Development, but instrument against these conventions anyway. Proprietary span formats are how teams get stuck with their first tool.
What it costs to run agents without tracing
In LangChain’s State of Agent Engineering survey of 1,340 practitioners, 89% had some form of observability, but only 62% had detailed tracing that let them inspect individual agent steps and tool calls. Among teams already in production, those figures rose to 94% and 71.5%.
The cost side is sharper: the same Anthropic write-up reports that multi-agent systems use about 15 times more tokens than chat, and that on the BrowseComp evaluation, token usage alone explained 80% of the variance in performance. A trace without token counts per span hides the biggest driver of both your bill and your quality.
Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. Traces are the evidence in all three arguments.
How we assessed each tool
We scored every tool on the same seven dimensions, using public documentation as of October 2026. Lyzr publishes this guide and is listed first; we say where a leaner tool is the better call. A dependable
website backup strategy helps teams protect website data and recover important files when unexpected failures occur.
| Dimension | Why it matters |
|---|---|
| Token and cost per trace | Token use explained 80% of performance variance in Anthropic’s tests |
| OpenTelemetry support | Keeps your instrumentation portable |
| Filtering by agent, user and session | Finding one bad run among thousands starts here |
| Agent discovery | You can’t trace agents nobody knows exist |
| Identity attribution | A span named “agent_2” won’t satisfy an auditor |
| Policy enforcement | A trace that can’t change anything is a report |
| Free self-hosting | Traces often contain customer data |
The 6 distributed tracing tools compared
1. Opencontroller by Lyzr: the governance layer that sits above your traces
Lyzr Agent Studio gives every execution a root trace with its trace ID, duration, and token and credit consumption, plus a span waterfall that runs from session start through agent orchestration to the generated response. You can filter by date range, agent name, user or session ID, and watch average and P95 latency. The telemetry is standardized on OpenTelemetry.
The five tools after this one are built to show you the run. Opencontroller is built for what happens next. It discovers agents, models, tools, data and workflows across clouds and frameworks, attributes every action to an identity, and enforces policy in the request path. If a trace shows an agent reaching for data it shouldn’t, the next call can be refused, and the agent stopped, restricted or isolated, with every decision kept on the record.
| Capability | Tracing tools provide this | Lyzr adds this |
|---|---|---|
| Span waterfall | Yes | Yes, with credits and tokens per trace |
| Agent identity | Usually a name tag | Every action attributed to an identity |
| Enforcement | Rarely | Refuse, stop, restrict or isolate in the request path |
Pick it if you are
- Running agents across several frameworks or clouds and losing count of them.
- Expected to show who acted, under which policy, and what stopped it.
Skip it if you are
- After a free trace viewer for one framework. Langfuse or Phoenix gets you there faster.
- Looking for a general APM to cover non-AI services too.
Not sure how many of your agents are even emitting traces? The AI Agent Sprawl Audit helps you find the ones nobody is watching.
2. Langfuse: open-source tracing you host yourself
Langfuse is MIT-licensed (apart from its ee folder), with about 32,000 GitHub stars. It accepts OTLP at /api/public/otel, and its SDK v4 is a thin layer over the official OpenTelemetry client. Sessions group multi-turn runs, and agent graphs switch between an aggregated view of the workflow and an expanded, step-by-step one.
ClickHouse acquired Langfuse in January 2026; the founders say self-hosting stays first-class and no licensing changes are planned.
The catch: graphs are inferred from observation types, and generic spans don’t become nodes. Type your agent and tool observations or the graph will look emptier than your system.
License: MIT core; managed cloud available.
Best for: teams that want open-source tracing on their own infrastructure.
Not ideal if: you need traces joined to non-AI service data.
3. LangSmith: cross-service tracing for LangChain stacks
LangSmith’s distributed tracing passes a langsmith-trace header, plus an optional baggage header for metadata, between services. Python middleware covers FastAPI, Flask, Django and Starlette, and TypeScript helpers cover Express and Hono, so an orchestrator in one service and a subagent in another land in one trace.
Two cautions. The docs say to accept these headers only from trusted internal services, or outsiders can attach calls to your traces. And the native header isn’t W3C traceparent; LangSmith also accepts OpenTelemetry traces, so mixed stacks should pick one propagation scheme.
License: Commercial; self-hosting is an Enterprise-plan add-on.
Best for: LangChain and LangGraph teams splitting agents across services.
Not ideal if: most of your agents run on other frameworks.
4. Arize Phoenix: traces and evals on your own servers
Phoenix builds on OpenTelemetry through OpenInference and instruments LangGraph, LlamaIndex, CrewAI and DSPy. It runs locally or on Docker and Kubernetes, and puts evaluations beside the traces they score, so a low-quality run and its spans share one screen.
Ownership is shifting: Dynatrace announced on August 13, 2026 that it’s acquiring Arize for about $915 million. Ask about the Phoenix roadmap first.
License: Elastic License 2.0 (source-available); Arize AX is the managed product.
Best for: teams that want tracing and evaluation together, self-hosted.
Not ideal if: you need policy enforcement or an OSI-approved license.
5. Jaeger: the vendor-neutral backend
Jaeger is a CNCF graduated project, and Jaeger v2 (November 2024) was rebuilt on the OpenTelemetry Collector, so it ingests OTLP natively. If your platform team already uses it, agent spans can land in the same backend for free.
It doesn’t understand agents yet. GenAI convention visualization is open work (issue #8401), alongside an in-app assistant built on ACP and AG-UI. Today you get generic spans, with no token roll-ups or agent graph.
License: Apache 2.0.
Best for: OpenTelemetry shops that want one backend for services and agents.
Not ideal if: you need AI-specific views this quarter.
6. Datadog LLM Observability: agents next to the rest of your stack
Datadog natively ingests OpenTelemetry GenAI spans (version 1.37 and up) and maps model, tokens, latency and cost. Its agent monitoring adds an execution flow graph of agent interactions, tool usage, handoffs, retries and errors for the OpenAI Agents SDK, LangGraph, CrewAI and Bedrock Agents.
The real reason to pick it is correlation: when an agent is slow because the database behind its tool is slow, both appear in one trace. Datadog’s AI Guard can also block risky prompts, responses and tool calls inline, though it’s still in limited access and preview.
License: Commercial, usage-based.
Best for: teams already running Datadog APM.
Not ideal if: span volume is high and budgets are tight; model the bill first.
Compare all six distributed tracing tools side by side
Key: • full support · ◐ partial support · ○ not the focus. Editorial assessment, October 2026, based on public documentation.
| Capability | Lyzr | Langfuse | LangSmith | Arize Phoenix | Jaeger | Datadog |
|---|---|---|---|---|---|---|
| Token and cost per trace | • | • | • | • | ○ | • |
| OpenTelemetry support | • | • | ◐ | • | • | • |
| Filtering by agent, user and session | • | • | • | ◐ | ◐ | • |
| Agent discovery across the estate | • | ○ | ○ | ○ | ○ | ◐ |
| Identity attribution per action | • | ○ | ○ | ○ | ○ | ○ |
| Policy enforcement in the request path | • | ○ | ○ | ○ | ○ | ◐ |
| Free self-hosting | ○ | • | ○ | • | • | ○ |
| License | Commercial | MIT (core) | Commercial | ELv2 | Apache 2.0 | Commercial |
Still torn between two? Start from the one question that matters most to you.
Run three tests on any shortlist
- The broken-hop test. Put one agent behind a separate service and send a request. If its spans arrive as their own trace, propagation is broken and every dashboard will undercount.
- The handoff test. Ask which agent passed what context to which. If you rebuild the answer from timestamps, the tool shows spans, not handoffs.
- The expensive-trace test. Open last week’s costliest run and find the agent that burned the tokens. If that takes more than a minute, cost reviews will hurt.
Tracing is one item on a longer pre-launch list; the playbook on how to take agents to production covers the rest.
Turn your traces into controls
If your agents only summarize documents, any tool above will do. Once they issue refunds or touch customer data, a trace that explains the damage afterwards isn’t enough. Opencontroller turns what your traces reveal into decisions made before the next call, for agents built in Lyzr or any other framework.
Book a demo of Opencontroller →
FAQ
Opencontroller by Lyzr, Maxim AI, LangSmith, Langfuse, and Jaeger. Opencontroller acts on traces and identity rather than generating them; the other five trace multi-agent runs directly, spanning open-source and commercial options.
It’s propagating one trace context across every agent, sub-agent and tool call in a run, so the full path reconstructs as a single tree, the same job distributed tracing does for microservices, applied to agent boundaries.
A session is a whole multi-turn interaction. A trace is one request-response cycle within it. A span is one step inside that trace: a model call, tool call or database query. Most tools here use some version of this hierarchy.
Usually yes, for Langfuse, LangSmith, and Jaeger, which need spans emitted from your code. Opencontroller differs: it governs agents that already run and consumes signals from existing tracing rather than requiring new instrumentation.
Book A Demo: Click Here
Join our Slack: Click Here
Link to our GitHub: Click Here
