Gateway Bypass in AI Infrastructure: Why Agent Traffic Goes Unseen

TL;DR:

• API gateways were designed for north-south, request-response, REST traffic hitting endpoints known at design time. Agentic traffic is none of those things.
• It runs east-west between services and agents, streams over long-lived connections, discovers its tools at runtime, and is initiated autonomously by a reasoning loop rather than a person.
• That is four architectural mismatches, not four discipline failures, and it means a meaningful share of agent traffic was never on the gateway’s path to begin with.
• The consequence is that your visibility into AI usage is not merely incomplete, it is confidently wrong, because the dashboards reporting on it look healthy.

What Is AI Gateway Bypass in AI Infrastructure?

Gateway bypass is when agents, MCP clients, or internal workloads reach model APIs, tool endpoints, or data sources without passing through the centralised gateway meant to govern those calls.

Most writing on this subject covers one half of the problem. There are two.

Outbound, or north-south. An application calls an external model provider directly instead of routing through the gateway. A service holds a raw provider key, instantiates an SDK client, and makes the call. This is the well-understood case, it is genuinely solvable, and we cover the detection methods in a companion guide on how to detect LLM calls that bypass your AI gateway.

Internal, or east-west. An agent calls another agent. An agent invokes an internal service over a private DNS name. An agent connects to a Model Context Protocol server running as a local process. In each case the gateway was never in the path. There is nothing to bypass because there was never a checkpoint.

The second case is the one almost nobody has a control story for, and it is growing faster than the first as multi-agent architectures move from prototype into production. It is also the case this article is mostly about, because it is the one that changes what “gateway coverage” actually means.

This is a distinct problem from shadow AI, which describes unapproved tools being used at all. Gateway bypass involves approved systems taking an unapproved path, which is harder to spot precisely because nobody did anything wrong.

What Does an AI Gateway Do?

Before the argument, the fair part, because a page that treats gateways as inadequate is not analysis and an architect with one deployed will stop reading.

An AI gateway solves real problems, and solves them well:

• Credential centralisation. Provider keys live in one place. Applications hold gateway-scoped credentials that are useless anywhere else. This is the single most durable control in the entire stack, and it works regardless of where the code runs.
• Cost attribution and metering. Token consumption attributed per service, per team, per model, with quota enforcement and budget alerts. Without this you have a provider invoice and a guess.
• Rate limiting. Protection against a retry loop or a runaway process consuming a month of budget over a weekend.
• Model routing and failover. A single internal endpoint that abstracts which provider and which model actually serve a request, which is what makes model flexibility practical rather than theoretical in a multi-provider environment.
• Prompt logging and boundary guardrails. Request and response capture, PII redaction, content filtering, and injection screening applied at the network boundary.

Every one of these works. Every one of them works on traffic the gateway sees.

The rest of this article is about what share of agentic traffic that actually is.

Why AI Gateways Miss Agent Traffic: 4 Architectural Mismatches

The gateway model rests on four assumptions about how traffic behaves. Agentic systems break all four.

Mismatch one: direction

Gateways sit at the perimeter. Their entire design premise is that meaningful traffic crosses a boundary, and that the boundary is where you put the checkpoint.

Agent systems generate enormous internal traffic that never crosses it. An orchestrating agent delegates to a specialist agent. An agent queries an internal vector store. An agent connects to an MCP server running as a local subprocess on the same host, communicating over stdio, never touching the network stack at all.

In a Kubernetes environment running a service mesh, this traffic routes sidecar to sidecar. The mesh does exactly what it was installed to do, the traffic satisfies mesh policy, and the north-south gateway is not consulted because it is not on the route. Nobody made a decision to bypass anything.

This is the deepest of the four mismatches by volume. As agent orchestration patterns mature, the ratio of internal to external calls moves steadily in the wrong direction for a perimeter control.

Mismatch two: shape

Gateways are optimised for a request that completes. Send, receive, close, log.

Agentic workloads stream. Tokens arrive incrementally over a connection held open for the duration of a generation. Agent protocols use gRPC and WebSockets, and increasingly JSON-RPC for tool communication. A gateway tuned for HTTP REST can add latency to a streaming path or handle it poorly enough that the user experience degrades noticeably.

The important consequence is that some teams route around the gateway deliberately and with justification. This is an engineering trade-off made by competent people, not a policy violation, and treating it as one leads organisations to the wrong response. Support varies considerably by gateway product and version, so the honest answer for any specific stack is to test rather than assume.

Mismatch three: discovery

A gateway governs endpoints someone configured. That is the mechanism. Policy is written against a known set.

An agent discovers its capabilities at runtime. Through MCP or an equivalent protocol, tools describe themselves to the model when it connects, and the set of things an agent can reach is not determined when the gateway policy was authored. It is determined when the agent starts, or when a developer adds a server, or when a tool updates its own description.

You cannot allowlist an endpoint whose existence is decided after deployment.

This is the mismatch that cannot be fixed by adding routes, because the problem is not that the routes are missing. It is that the set of routes is dynamic and the policy model assumes it is static.

Mismatch four: initiator

Traditional traffic has a requester. A user clicked something, a schedule fired, a service called a dependency. Attribution is a solved problem.

Agentic traffic is initiated by a reasoning loop. One user intent becomes a tree of calls: a retrieval, three tool invocations, a handoff to another agent, two more tool calls, a response. The gateway sees individual HTTP requests. It does not see the decision that produced them, the reasoning between them, or the relationship among them.

This means even fully routed traffic gives an incomplete picture. A gateway that captured one hundred percent of an agent’s model calls would still not be able to tell you why the agent made them, what it did with the results, or whether the action it ultimately took was one anybody approved.

How AI Agent Traffic Bypasses the Gateway

Five recognisable causes. Note that in three of them, nobody chose to bypass anything.

Raw SDK usage in application code. The documented path in every provider quickstart is a client, a key, and a call. Nothing in that documentation prompts a developer to point at an internal endpoint. This is a default, not a decision.

Hardcoded internal endpoints. Service-to-service calls written against a private DNS name or internal address for latency reasons. Fast, direct, and structurally invisible to a perimeter control.

Service mesh routing. Sidecar-to-sidecar communication satisfying the mesh’s own policy while never approaching the enterprise gateway. The mesh did what it is for.

Isolated experimentation environments. Research and data science environments running agent frameworks with their own keys, deliberately separated from production networking. Often the separation was the point.

Protocol workarounds. Streaming or gRPC traffic routed around a gateway that handles it badly. A deliberate engineering choice with a real justification.

The pattern across all five is worth sitting with. Two involve a conscious decision, and both of those decisions were defensible given the constraints. The other three are emergent properties of how the systems are built. None of this is solved by a policy memo, which is why organisations that treat bypass as a discipline problem tend to run the same programme twice.

The same friction dynamics show up across enterprise AI agent challenges generally: the governed path loses to the fast path unless someone deliberately closes the gap.

What Happens When AI Agent Traffic Bypasses the Gateway?

Data exposure without inspection. PII, PHI, source code, and proprietary data reaching third-party models with no redaction, no classification, and no record that it happened. Any DLP or data residency control configured at the gateway applies to zero percent of bypassed traffic. Our overview of the risks of public LLMs covers the exposure surface in more detail.

Unattributed spend. Consumption landing on the provider invoice with no internal owner, no quota, and no protection against a loop. You know the total. You cannot allocate it, forecast it, or cap it.

Compliance blind spots. Agent-to-agent interactions unlogged, breaking the audit trail that GDPR, HIPAA, SOC 2, and sector regulators expect. The specific problem is subtler than a missing log. It is that the organisation cannot demonstrate the gap is bounded. “We log everything through the gateway” is only an answer if you can prove nothing goes around it. Our guides to European enterprise AI governance and AI in risk and compliance cover what evidence regulators actually expect.

Blast radius on compromise. This is the one worth the most attention. If an agent is influenced through prompt injection, what it can reach next is determined by its network position and credentials, not by the gateway. An agent operating east-west with broad internal access and no checkpoint between it and downstream services is how a content problem becomes an incident. The gateway would have been the natural interception point, and it was never in the path.

Why these compound

Individually each of these is a gap. Together they produce something worse than incomplete visibility.

An organisation with no gateway knows it has no visibility and behaves accordingly. An organisation with a partially-covering gateway has dashboards that look complete: request volume, spend, policy violations, model distribution, all reported cleanly. Compliance attestations, cost forecasts, and architecture reviews all rest on numbers that describe a subset while presenting as a total.

Being wrong is more expensive than knowing you are blind.

Where Should AI Agent Traffic Be Controlled?

Four layers, from the perimeter inward. Each covers something the others do not.

Network perimeter. Deny-by-default egress policy and DNS control, so outbound provider calls must traverse the gateway. This covers the north-south case properly. It does nothing east-west, and it does not reach laptops off VPN, contractor machines, or accounts outside your landing zone. That endpoint gap is covered separately in our guide to detecting AI coding agents on endpoints.

Credentials. Applications hold gateway-scoped credentials rather than provider keys, so a direct call fails on authentication regardless of network position. More durable than network policy because the control travels with the workload rather than depending on where it runs. This is the highest-leverage control available for outbound traffic, and our detection guide covers how to audit it.

Service mesh and internal policy. For east-west traffic, identity-based mesh policy is the only enforcement point that exists. Which workload may reach which service, enforced by workload identity rather than by address. Most organisations running a mesh have not extended this thinking to agent traffic, and it is the single biggest under-used control in the stack.

The agent layer. The one the other three structurally cannot reach. Per-agent permission scoping. Approval gates on consequential actions. Decision traces recording what the agent retrieved, what it reasoned, and what it invoked.

This is where runtime-discovered tools and autonomous initiation become governable, because the control lives with the agent rather than on a path the agent may not take. It is also the only layer that can answer the question a regulator or an executive actually asks, which is not “was this call routed correctly” but “what did this system do, on what basis, and who was accountable.”

The synthesis

The gateway remains necessary. It is the right control for model access, credential centralisation, cost attribution, and boundary guardrails, and no organisation running AI at scale should be without one.

What it cannot do is govern an actor that discovers its own capabilities at runtime and operates inside your network. That requires a control that lives where the agent lives.

The practical answer is not to replace the gateway. It is to stop expecting it to be the whole control plane. Our framework for AI agent governance covers how the layers fit together, and the comparison of AI agent observability platforms covers the tooling for the agent layer specifically.

Where an AI Agent Platform Fits Beyond the Gateway

To be explicit: Lyzr is not an AI gateway and does not replace one. Run a gateway. It solves the model-access problem well and this article is not an argument against it.

What sits above it:

Agent registry and ownership. Every agent inventoried with a named human owner and a stated purpose. This is what makes every other control enforceable, because an agent nobody owns cannot be scoped, reviewed, or decommissioned. See agent types in production for how this works at scale.

Per-agent permission scoping. Capability bounded at the agent rather than at a network path the agent may not use. This is the control that addresses mismatch three, because it constrains what an agent may do with a tool regardless of when that tool was discovered.

Approval gates. Human sign-off on consequential actions, applied by consequence rather than by route. Financial commitment, external communication, and data destruction should stop for a person whether the underlying call went through the gateway or not.

Decision traces. What the agent retrieved, what reasoning it applied, what it invoked, and with whose credentials. This is the record a gateway structurally cannot produce, because it sees one request rather than the loop around it.

The OpenController provides these as platform capabilities, with Responsible AI and hallucination management controls applied at the behavioural layer. Agents built on existing frameworks can be registered without a rewrite. The Lyzr documentation covers the registration and permission model.

Where to go next: if you want to find out whether traffic is currently going around your gateway, start with the detection guide. If you are building the control framework rather than running a detection pass, start with AI agent governance or the playbook on taking agents to production. If you would rather talk it through, book a demo.

Frequently asked questions

What is gateway bypass in AI infrastructure?

When agents, MCP clients, or internal workloads reach model APIs, tools, or data sources without passing through the centralised gateway meant to govern those calls.

Why do AI agents bypass API gateways?

Often they do not bypass anything. Gateways sit at the perimeter, and much agent traffic is internal, streaming, or directed at endpoints discovered at runtime, so the gateway was never on the path.

Is an AI gateway enough on its own?

No. It governs model access well. It cannot govern an agent that discovers tools at runtime, operates east-west inside your network, and initiates its own calls autonomously.

What is east-west agent traffic?

Traffic between services or agents inside your environment, as distinct from north-south traffic crossing the perimeter. Most enterprise gateway policy governs only the latter.

Can a service mesh solve AI gateway bypass?

For internal traffic, identity-based mesh policy is the only enforcement point that exists, and it is significantly under-used for agents. It does not address runtime tool discovery or give you agent-level attribution.

What are the risks of unseen agent traffic?

Data exposure without inspection, unattributed spend, unlogged interactions that break compliance evidence, and an unbounded blast radius if an agent is compromised.

How is gateway bypass different from shadow AI?

Shadow AI is unapproved tool use. Gateway bypass is approved systems taking an unapproved path, which is harder to spot because the tools themselves are sanctioned.

Do AI gateways support streaming and gRPC?

Support varies by product and version, and gateways optimised for HTTP REST can degrade streaming performance. That is one reason competent teams route around them deliberately. Test your specific stack rather than assuming.

How do I find out whether traffic is bypassing my gateway?

Start with credential reconciliation and billing comparison, both of which take minutes. Our detection guide covers the four layers in order.

Does an AI gateway govern AI agents?

It governs their model calls. Agent behaviour, tool use, permissions, and approvals sit above it and need separate governance.

Book A Demo: Click Here
Join our Slack: Click Here
Link to our GitHub: Click Here