Executive summary
Every enterprise is piloting agents. Almost none can show you what is running in production. The independent data is unambiguous about the scale of the gap: MIT’s Project NANDA found that 95% of enterprise generative-AI pilots deliver zero measurable return, and Gartner reports that only 48% of AI projects reach production at all, taking, on average, eight months to get there.1,2 The share of enterprises abandoning most of their AI initiatives rose from 17% to 42% in a single year.3
That gap is the subject of this paper. It is not an argument about where agentic AI is headed; it is an account of where it already is, drawn from a live base of agents doing real work inside customer environments. Task-success rates, time-to-live, the model mix behind the agents, how the eval gate works, where deployments stall, and how a single use case becomes a fleet.
The central claim is simple and testable: if a vendor cannot state their production task-success rate, their pre-production eval pass rate, and the model mix their agents depend on, they do not have agents in production. They have demos. This paper states all three, shows how the machinery behind them works, and closes with five questions any buyer can take into any vendor meeting. Questions a vendor with agents in production can answer without leaving the room.
For the technical owner of AI outcomes, this is a vocabulary for judging your own vendors and a realistic picture of time-to-value. For the executive accountable for ROI, it is evidence that “production” can mean measured business outcomes rather than another well-rehearsed pilot.
The state of agents in production
Gartner finds that fewer than half of enterprise AI projects reach production at all. Those that do average eight months to get there.2 MIT’s Project NANDA reports that 95% of generative-AI pilots deliver zero measurable P&L return.1 S&P Global found the share of enterprises abandoning most of their AI initiatives more than doubled in a single year.3 That is the baseline. What follows is the counterpoint.
Across the production base, agents complete their defined tasks at a 95% success rate, against an internal acceptance target of ≥90%. They clear the pre-production evaluation gate at a 95% pass rate before any agent touches live traffic, and they run at 99.9% uptime once live.6
Two structural facts matter more than any single rate. First, 100% of production workflows are multi-step or multi-agent (up from 90%), which means this is genuine agentic work: agents planning, calling tools, and handing off across steps, not single-prompt completions dressed up as autonomy.6 Second, customer-validated outcome coverage stands at 60%, up from 40%: a majority of production customers now have a documented, measurable business outcome attached to their deployments.6
These are not static snapshots. The numbers are live, collected via OTel-based end-to-end trace logging on every agent interaction (latency, output quality, token usage, cost per interaction, task success, uptime), surfaced through a real-time gateway dashboard that shows per-agent spend, request counts, and version status.6 The observability layer is what makes a scorecard trustworthy: these rates are continuously measured and acted on, not periodically sampled and reported.
What is actually going live
Production is not concentrated in one clever use case. It is broad, and it skews toward the functions where a wrong answer has a cost and a right answer has a number attached.
By function, live agents span search and retrieval, coding and SDLC, conversational and content generation, SDR and sales development, email and claim triage, competitor intelligence, contract intelligence, document extraction, voice and calling, procurement, compliance and regulatory assessment, HR workflows, and research and investment-memo generation.7
By industry, the fleet is heavily weighted toward BFSI (insurance, retail banking, and wealth and private banking), followed by HR, research, marketing, customer support, and legal.7
Deployment walkthrough: Accenture Startup of the Week
Accenture’s Startup of the Week program, a recurring initiative that scouts, evaluates, and profiles emerging startups, was rebuilt end-to-end on Lyzr agents.11,13 The previous workflow was manual at every handoff: outreach drafted individually, responses arriving in mixed formats, data extracted by hand from pitch decks, and newsletters written from scratch under weekly deadlines.
Lyzr deployed a coordinated multi-agent system: an Email Agent handling personalized outreach with automated follow-ups, fine-tuned GPT-4o-mini parsing agents extracting structured data from documents and emails, and a content engine generating editorial-ready newsletter drafts, all running inside Accenture’s own AWS environment with Okta and SAML authentication, Secrets Manager, KMS, and CloudWatch.
Named deployments include Saksoft and Movate (coding/SDLC), Yugabyte (GTM intelligence), MSP Corp (support architecture), eMudhra (lead generation), Verifone (50% resolution-time reduction),7,11,12 AirAsia MOVE (content pipeline), HFS Research (enterprise research), LionAIs (diagnostics), SNS Global (resource allocation), and WTW (retirement advisory). Behind the anonymized tier sit names of comparable weight: a leading Japanese bank’s customer-onboarding agent, a global bank’s multi-agent customer-experience system, an enterprise procurement OS. Described by pattern because that is the clearance, not the limit of the work. Several are firms whose business is delivery: Accenture, Movate, Saksoft, HFS Research, and Prophet,7,11 running Lyzr agents in their own production environments.
The models behind the agents
Production is multi-model by necessity. In a16z’s survey of enterprise CIOs, the share running five or more models in production rose to 37% in a year.5 The difference is that most enterprises reach multi-model by accident; a production fleet reaches it by design.
No single model exceeds 60% of workloads. Not as an ideology, but because concentration is a measurable operational risk. A fleet that runs on one provider is a fleet that has not yet met a job that provider loses.
How model selection works in practice
Model assignment is a build-time decision per agent, not a dynamic runtime router. Models can be swapped with a single config change without touching agent logic. ShadowLM distills frontier-model behavior into smaller, cheaper models per use case, validated by the Simulation Engine. The LLM Gateway sits between every agent and its model providers, enforcing budgets, virtual keys, and policy per call. This is the layer that makes the 60%-max rule enforceable rather than aspirational.
Where the agents run
Customer data stays inside the customer’s boundary. Agents run in the customer’s own environment, and model calls are routed through in-boundary endpoints (AWS Bedrock, GCP Vertex AI, Azure OpenAI) rather than to public provider APIs. Lyzr retains no customer data and no prompt or completion content outside that boundary.
Underneath the certifications sit the operational controls a security review actually tests: encryption in transit and at rest, zero-trust networking, role-based data access, continuous access monitoring, audit logging of agent and user actions, third-party penetration testing, and business-continuity and disaster-recovery provisions.15 Every agent and user action is logged, and access to data and agents is role-scoped and monitored.
How agents reach production
The median time to a first agent in production is 8 weeks.8 Set that against Gartner’s eight-month industry average.2 An 8-week median is not a marginally faster version of that timeline; it is a different order of magnitude.
The eval gate: nothing ships unevaluated
Every agent clears the Simulation Engine, a pre-deployment evaluation gate that generates 500+ synthetic test cases per run, compared to roughly 20 in manual QA.6 Test suites are domain-specific (BFSI, healthcare, HR), not generic public benchmarks, and include adversarial inputs and edge cases. Hallucination detection and scoring run inside the simulation loop, not only at inference time, so an agent that hallucinates in pre-deployment testing is caught and iterated before it ever touches live traffic.
On failure, the system diagnoses which persona, which task, and which part of the response failed, and attaches a suggested fix. The agent is iterated in place (not rebuilt from scratch) until it clears the pass threshold. Average: four rounds to reach approximately 98% failure detection.
Staged promotion: Dev → Staging → Prod
Each push creates a version tag. Branches map to environments. Promotion to production requires explicit approval. Rollback is a pointer move on an immutable artifact: instant, no rebuild or redeploy. What was tested in staging is exactly what runs in production.
Where agents stall
Autonomy is the hardest thing to measure. Escalation-to-human, how often an agent hands a case back to a person, is the metric buyers most want and the industry least reliably has. On our own base, clean escalation-rate telemetry is not yet available. Rather than publish a number we cannot stand behind, we state the difficulty plainly: measuring autonomy well is an open problem across the field.
Inference cost is a production discipline, not a footnote. An agent that works but whose unit cost does not trend down is not yet a production asset. The LLM Gateway enforces per-agent budgets in real time: calls that exceed budget or fail policy checks are refused at the gateway, not logged after the fact.
What happens when an agent fails
The Observe → Act → Evolve loop
Production operations follow a continuous cycle. Observe: OTel traces, gateway dashboard, per-agent metrics. Act: rollback, quarantine, gateway refusal. Evolve: production failures become test cases, the agent is iterated through the eval loop, and redeployed through the pipeline. Every production incident strengthens the eval suite for the next deploy.
Gartner projects that more than 40% of agentic-AI projects will be cancelled by the end of 2027.4 Deployments that do fail tend to fail for organizational reasons (product gap, sponsor change, budget, M&A), not technical ones. Each stalled pilot erodes the institutional appetite for the next.
One use case becomes many
The economics of agents in production are not made in the first deployment. They are made in the second, third, and thirtieth.
The expansion telemetry is the clearest signal in the dataset. A production fleet of more than 1,200 agents, with the typical customer fleet more than doubling over the trailing period.10 The time to a second use case is 45 days, down from 60. Expansion now moves faster than the initial build.10 And logo retention sits at 97%, up from 89%: once a fleet is in production, it stays.10
Why the second agent is cheaper than the first
The governance layer is built once and shared. The Simulation Engine’s test-scenario library persists and compounds: domain-specific test suites built for agent #1 become the regression suite for agents #2 through #30. Each new agent registers into existing infrastructure rather than standing up its own. The Control Plane manages the fleet as one entity (one identity system, one policy engine, one gateway, one audit trail) while each agent retains its own identity, budget, and version history within that shared system. The first agent is expensive to earn. Everything after it compounds.
The clearest example is Accenture, whose corporate venture arm runs 200+ agents on Lyzr, the ceiling of the block-buildout. A second, anonymized example, a global marketing-services group running a multi-track agent “lab,” shows the same shape at a different scale. WTW’s governed retirement-advisory agent has run in production for over a year, the difference between a deployment that survives its first renewal and a pilot that never had one.14
Five questions to demand from any agent vendor
Close
Everyone can show you a pilot. The measurable facts (95% task success, 95% eval pass, 99.9% uptime, a genuinely multi-step and multi-model fleet, an 8-week median to production, and a 97% logo-retention expansion motion) are what separate agents that ship from agents that demo. Behind those numbers sits the machinery that produces them: an eval gate that generates 500+ test cases per agent and iterates until the failure-detection rate reaches 98%, a staged promotion pipeline that version-tags every deployment and rolls back in seconds, a gateway that enforces model policy and budgets per call, and a shared infrastructure layer that makes every agent after the first cheaper to build, test, and govern.
The pilot era proved agents could be built. Production is where they are measured. This is what the measured ones look like.
Move beyond demos. Actually ship agents.
Bring us a workflow. See it in production.
Book a Demo Explore Agent Studio
A note on methodology
“In production” means a distinct agent running in a customer environment against real workloads, invoked within the preceding 30 days.
Task-success rate: successful end-to-end task completions ÷ total invocations, weighted by customer.
Eval pass rate: agents passing the evaluation suite ÷ agents submitted to the pre-production gate.
Uptime: uptime minutes ÷ scheduled minutes, weighted by customer.
Multi-step share: workflow invocations involving >1 step or agent ÷ total invocations.
Outcome coverage: production customers with a documented business outcome ÷ total production customers.
Median TTV: median of (go-live date − contract date) across deployments closed in the period.
Logo retention: logos retained ÷ logos at period start, trailing twelve months.
Escalation-to-human rate is not published here because clean fleet-wide telemetry does not yet exist. A number will appear when it can be stood behind, and not before.
Sources
1. MIT Sloan / Project NANDA, The GenAI Divide (2025): 95% of enterprise AI pilots delivered zero measurable P&L return. 300+ deployments, 150 leader interviews, 350 employee surveys.
2. Gartner (May 2024): 48% of AI projects reach production; 8-month average prototype-to-production. Survey of 644, Q4 2023.
3. S&P Global / 451 Research (Oct 2025): company-level AI initiative abandonment rose from 17% to 42%. Survey of 1,006, Oct–Nov 2024.
4. Gartner: 40%+ of agentic-AI projects projected cancelled by 2027. McKinsey: 62% of organizations experimenting with AI agents.
5. a16z enterprise AI survey: 37% of enterprises run 5+ models in production, up from 29%.
6–10. Lyzr production telemetry, (task success, eval pass, uptime, workflow share, outcome coverage, TTV, deployment reliability, expansion, retention).
11. Published Lyzr customer case studies: WTW, Accenture, AirAsia MOVE, eMudhra, SNS Global, Saksoft, Movate, HFS Research, LionAIs, MSP Corp, Yugabyte, Verifone, Prophet.
12. Accenture venture arm: 200+ agents, 15+ VC functions, ~80% time reduction. lyzr.ai homepage + press (PYMNTS, citybiz).
13. WTW: retirement-advisory agent, 1+ year in production. lyzr.ai homepage.
14. Lyzr Trust Center: SOC 2 Type II, ISO 27001, ISO 42001, GDPR, HIPAA. security.lyzr.ai.
