AI in hardware · October 2026 · A 30 minute read
Agents at
the edge.
Running agents on devices and appliances. When owning AI hardware beats renting cloud APIs, how to size it, and what it takes to run agents on infrastructure you control.
The short version.
- The argumentTraining is a short burst. Inference runs every day, so inference is the hardware worth owning.
- The meterAgents read far more than they write, and loops multiply it. Anthropic measured about 4 times the tokens of chat for an agent and about 15 times for a multi-agent system.
- The testOwn when the work is steady, an open-weight model is good enough, and the price you would otherwise pay is high. Rent when it is bursty, small, or needs a frontier model.
- The mathA $750K appliance costs about $23K a month to own, at four years and 12% a year to run. It has to replace more than that in rent.
- The sizeMemory sets what fits, bandwidth sets how fast, and context length is what fills the memory.
- The workTwelve jobs sit between a GPU and an agent. An appliance takes most of them off your plate.
- The startMeasure two weeks of tokens, find the baseload, and size one box against it.
The five parts
Five parts, from the bill to the box.
Part 1
The daily bill
Why inference, not training, is the cost you pay every day.
Part 2
Rent or own
The math, the break-even, and where each side wins.
Part 3
Size it
Memory, bandwidth and context, with a sizing sketch.
Part 4
Run it
Twelve jobs between a GPU and an agent, and questions for vendors.
Part 5
The reference
Optimus as a reference architecture, and a 30 day pilot.
Deloitte, 2026 TMT Predictions
Inference is becoming the bill.
Deloitte predicts that running models will take about two thirds of AI compute in 2026, up from one in three in 2023. It expects most of it to stay in data centers.
Deloitte, Technology, Media and Telecommunications Predictions 2026, November 2025. A prediction, not a measurement. Here the edge means infrastructure you control.
The daily bill
Agents multiply the meter.
An agent reads its whole context at every step and loops until the job is done. Anthropic measured agents at about 4 times the tokens of chat, and multi-agent systems at about 15 times.
Source is Anthropic Engineering, June 2025.
The daily bill
Every step re-reads the whole story.
In a 20 step run, an agent can read 555,000 tokens to write 10,000. Caching softens the bill, but the hardware still has to hold the context.
Illustrative arithmetic, not a measurement. A 4,000 token start, then 2,000 tokens of tool results and 500 of output added at each step.
Deloitte, Tech Trends 2026
Prices fell 280 times and the bill still rose.
Deloitte’s reading is simple. Usage grew faster than the price fell, so cheaper tokens did not make a cheaper bill.
Deloitte, Tech Trends 2026, The AI infrastructure reckoning. The shape of the spend line is illustrative.
Rent or own
Owning is a bet on steady volume.
Hardware is paid up front, and then the meter stops. Rent starts at zero and never stops. Which line wins depends on how much work you send.
Illustrative. An appliance at $750K up front plus $7.5K a month to run, against two levels of rent.
Five numbers decide it.
Three studies, three answers.
The calculator
Rent or own, with your numbers.
The defaults describe an Optimus Mini. Change anything, including the price and the rated speed.
The verdict
Owning is cheaper by 40%
Saves about $15,625 a month.
Rent, per million output tokens$15.50
Own, per million output tokens$9.25
Break-even volume, one unit1.5B a month
Break-even utilization7.7%
Units needed1
Own assumes four years and 12% of the price a year to run. Rent uses published rate cards.
The math
The price you compare against is the whole story.
At 30% utilization an Optimus Mini costs about $4 for each million output tokens it writes. That is 95% below frontier-tier rent, and about five times above small-tier rent.
Our math, illustrative. Mini at $750K, rated 7,500 output tokens a second, four year life, 12% a year to run. Rent at published rate cards, October 2026, with 5 tokens read for each token written and half of the reads cached.
Break-even depends on the tier you replace.
Where renting wins, and where owning wins.
The pattern
Own the baseload. Rent the peaks.
Power utilities do not build plants for the hottest hour. They own the steady load and buy the spikes. Inference has the same daily shape.
Illustrative demand curve. The shares shown are computed from the curve.
Size it
Memory bandwidth sets the speed limit.
Every token a model writes needs its active weights read from memory. Speed is roughly bandwidth divided by the bytes read, and real systems reach a quarter to a third of that ceiling. Batching shares each read across many users.
Measured speeds as reported by LMSYS through Spheron, October 2026, for gpt-oss-20b. About 1.9 GB is read per token. Bandwidths are vendor specifications.
Four numbers size an appliance.
The sizing sketch
A sizing sketch.
Memory is the first gate. Pick a model, a context and a number of sessions.
Weights use 8-bit values for the Llama models and 4-bit for gpt-oss-120b. Cache sizes come from each model’s published layer and head counts, and are upper bounds. Ninety percent of memory is treated as usable.
Memory needed
607 GB
88% of it is the context cache.
Model weights70.0 GB
Context cache537 GB
One 96 GB workstation GPUNeeds 7.0 times
A 128 GB desktop applianceNeeds 5.3 times
An Optimus Mini, 768 GBFits
96 GB GPUs needed8
Memory is the first gate. Throughput comes second, and depends on bandwidth and how many sessions share it.
Where devices, workstations and appliances fit.
Size it
What agents ask that chat does not.
Memory fills with history
Agents re-read what they have done, so context rather than weights becomes the large number.
Size for sessions
Many agents wake at once, so count busy agents rather than employees.
Loops run overnight
Work that never sleeps keeps owned capacity busy, which is what makes it pay.
Variation compounds
A ten step loop multiplies delay, so the slow tail matters as much as the average.
Log your own mix before you buy. Averages hide the peak.
Our synthesis, with the token multiples from Anthropic Engineering, June 2025.
Run it
Twelve jobs sit between a GPU and an agent.
Buying GPUs is the easy part. Integrating and operating them is the work. An appliance packages the stack so one bundle and one update path carry it.
Our list of the jobs. Every team will weigh them differently.
Run it
What an appliance takes off your plate.
Parts arrive tested together
Drivers, runtime, model serving and networking ship as one tested whole.
One update path
Lyzr says Optimus ships signed updates, so patching follows a single path.
Hardening is built in
Lyzr lists secure boot, encrypted drives and no external model calls.
Power, space, network access, the data connections, and the people who own the agents.
Lyzr Optimus site, October 2026. Vendor claims, not independently verified here.
Ten questions for any appliance vendor, one to five.
Ten questions for any appliance vendor, six to ten.
The reference
Optimus as a reference architecture.
Lyzr describes Optimus as an on-prem agent factory. Hardware, models and the agent platform arrive as one machine.
- HARDWAREEight GPUs, 768 GB, two 100 GbE ports.
- SECURITYSecure boot, encrypted drives, signed updates.
- MODELSgpt-oss-120b, Mistral Small 4, Gemma 4, or yours.
- SOFTWAREStudio, Search, admin console, Open Cowork.
Lyzr Optimus site and sovereign AI page, October 2026. Vendor claims, not verified here.
The Optimus lineup, as Lyzr lists it.
The reference
What else runs on the appliance.
Agents and governance
Agentic OS and Control Plane, with Agent Studio, Architect and Cognis memory.
A workspace for everyone
A governed AI coworker for every employee, on models you run.
Models tuned on your data
Fine-tune open-weight models on your own data, and keep the result.
One view of every agent
Policies, audit trails and spend limits for agents on or off the box.
Lyzr Nitro adds production modules on AWS, Azure and Google Cloud for the work you rent.
Lyzr sovereign AI page and pricing page, October 2026. Vendor descriptions, not independently verified here.
Where Optimus fits, and where to look elsewhere.
A 30 day pilot before you buy.
About Lyzr
The full-stack agent infrastructure platform for enterprises and governments that want to own their intelligence.
Founded in New York in 2023. Agent Studio, Architect, Opencontroller, Open Cowork, Openfinetuner, Optimus and Madison cover building, governing, running and owning agents. Deploys in your AWS, Azure or GCP account, a dedicated tenant, SaaS, on bare metal, fully air-gapped, or on the Optimus appliance.
Backed by Rocketship.vc, BGV, Accenture Ventures, GFT, the Partnership Fund for New York City and Plug and Play.
Sources and notes
Where the numbers come from.
- Deloitte, Technology, Media and Telecommunications Predictions 2026, November 2025.
- Deloitte, Tech Trends 2026, The AI infrastructure reckoning.
- Anthropic Engineering, How we built our multi-agent research system, June 2025.
- Lenovo Press, On-Premise vs Cloud, Generative AI Total Cost of Ownership, 2026 Edition (LP2368).
- Pan, A Cost-Benefit Analysis of On-Premise Large Language Model Deployment, Carnegie Mellon, 2025.
- Anthropic, Claude API pricing, rate card checked October 2026.
- Epoch AI, Open models lag state-of-the-art closed models by 4 months, May 2026.
Figures were checked in October 2026. Vendor studies are labeled as such on the slides that use them.
Sources and notes
Where the numbers come from, continued.
- NVIDIA, RTX PRO 6000 Blackwell Workstation Edition specifications.
- LMSYS benchmarks of DGX Spark and RTX PRO 6000, as reported by Spheron, October 2026.
- AMD, Hybrid NPU and iGPU optimized agent on Ryzen AI PCs, 2025.
- OpenAI, gpt-oss model card, August 2025.
- Lyzr, Sovereign AI page, including the Optimus lineup, October 2026.
- Lyzr, Optimus site, specifications and pricing, October 2026.
- Getlatka, Lyzr company profile, for reported Optimus price range.
Our math uses the assumptions stated on each slide. Prices and specifications can change.
Your next step
Own the baseload. Rent the peaks.
Measure two weeks of tokens, find the baseload, then size one appliance against it. Optimus pre-orders open in Q4 2026.