What Is an Agentic Mesh? The Infrastructure Layer Your Multi-Agent System Is Missing

You built one agent. It works. You built a second. It also works. Now you need them to talk to each other, and everything breaks.

This is the story playing out across enterprise AI right now. Organizations that solved the single-agent problem are discovering that multi-agent coordination is a fundamentally different challenge. The agent harness, as we explored in What Is an Agent Harness?, is the operating system for one agent. It handles context, tools, memory, verification. But an operating system alone does not create a network. For that, you need a protocol layer.

Enter the agentic mesh.

From Harness to Mesh

An agentic mesh is a distributed infrastructure layer that enables multiple autonomous AI agents to discover each other, communicate, delegate tasks, and collaborate according to governance policies. Eric Broda, then at IBM, coined the term in November 2024. The concept has since been formalized in an O’Reilly book and adopted by the IEEE Computer Society as an architectural reference.

The analogy to service mesh (Istio, Linkerd) is deliberate. A service mesh manages microservice-to-microservice traffic: TCP connections, HTTP headers, latency measured in milliseconds. An agentic mesh manages agent-to-agent intent and delegation: semantic meaning, token budgets, task completion measured in seconds or minutes. The traffic is different. The governance need is identical.

But the parallel has limits. Service meshes assume deterministic services with fixed APIs. You call endpoint A with parameters B and get response C. Agents are non-deterministic reasoners with dynamic tool use. You ask agent A to solve problem B and it might use tools C, D, and E, or it might decide it cannot solve the problem at all and delegate to agent F. The mesh has to handle both outcomes.

The Architecture

The IEEE Computer Society published a six-layer reference architecture for agentic mesh in November 2025. It is worth understanding, even if your organization will not implement all six layers soon.

Layer 1: Agent Runtime. The secure execution environment where agents run. Sandboxing, resource limits, crash isolation. This is the harness layer, now standardized across multiple agents.

Layer 2: Identity and Governance. IAM for agents. Which agent is allowed to call which other agent? What data can it access? What actions can it take? Audit logging for every interaction. This is where most organizations have nothing today.

Layer 3: Communication Fabric. The protocols agents use to find and talk to each other. Google’s A2A (Agent-to-Agent) protocol uses HTTP and JSON-RPC with Agent Cards that advertise capabilities. Anthropic’s MCP (Model Context Protocol) handles tool discovery and invocation. Both are gaining adoption. Neither has won.

Layer 4: Model Abstraction. Vendor-agnostic model selection so agents are not locked to a single provider. An orchestrating agent can route subtasks to the model best suited for each one.

Layer 5: Observability and Safety. Metrics, anomaly detection, compliance monitoring across the entire agent network. Not just “is agent A working?” but “is the emergent behavior of agents A through F producing safe outcomes?”

Layer 6: Enterprise Integration. Connectors to existing systems: ERP, CRM, data lakes, IoT platforms. The mesh must integrate with the enterprise, not replace it.

Why This Matters Now

The timing argument rests on numbers, not speculation.

McKinsey and QuantumBlack found that over 80% of companies report no material contribution to earnings from generative AI. The three blockers they identified were fragmentation (agents operating in silos), inconsistent coverage (some departments automated, others untouched), and production limitations (agents that work in demos but fail under real conditions). These are coordination problems. They are the exact problems a mesh architecture addresses.

Gartner projects that 40% of enterprise applications will include AI agents by the end of 2026, up from under 5% recently. They also warn that 40% of agentic AI initiatives risk cancellation by 2027 without an architectural shift. Their framing is useful: traditional integration platforms (iPaaS) were designed for deterministic connectors. Agents break every assumption those platforms rely on. They need dynamic discovery, real-time context sharing, delegated identity, and multi-protocol fluency. None of which a standard API gateway provides.

IDC estimates that 60% of AI failures by 2026 will stem from governance deficits, not model performance. The models are good enough. The infrastructure around them is not.

The Governance Case

Five principles from Gravitee’s analysis of mesh implementations deserve attention.

Centralized control with decentralized execution. One policy engine. Many execution points. You define governance rules centrally. Agents enforce them locally. This solves the “every team builds its own guardrails” problem that we identified in The Rise of Agentic Platforms.

Universal observability. Every agent interaction logged, traced, and auditable. Today, roughly 80% of organizations cannot tell what their autonomous AI is doing in real time. A mesh with built-in observability changes that.

Zero-trust security. No agent is trusted by default. Every request authenticated. Every action authorized. Every delegation verified. The data on this is alarming: 55% of organizations cite data exposure as their top AI concern, and 52% worry about unauthorized agent actions (Strata, 2026).

Policy-driven governance. Rules expressed as code, not as hopes. Rate limits, data access boundaries, escalation triggers. Applied uniformly across every agent in the mesh.

Interoperability through standardization. Open protocols (A2A, MCP, OpenTelemetry) instead of proprietary integrations. This matters because protocol fragmentation is already a real problem: alongside A2A and MCP, there is IBM’s ACP, China’s ANP, Agora, Eclipse LMOS, and several others. The mesh must abstract across them.

The Protocol Fragmentation Problem

The communication layer deserves its own discussion because it is where the market is messiest.

Google’s A2A protocol, launched with 50+ partners, uses HTTP and JSON-RPC. Agents publish Agent Cards describing their capabilities, and other agents discover and invoke them through those cards. It is designed for cross-organization agent communication.

Anthropic’s MCP focuses on tool discovery and invocation. It is stateful and has grown to over 16,000 registered servers in 2026. It solves a different problem than A2A: where A2A handles agent-to-agent delegation, MCP handles agent-to-tool integration.

Then there is IBM’s ACP, China’s ANP, the Agora framework, Eclipse LMOS, and the proposed agent.json standard. Each solves a legitimate problem. None is compatible with the others without translation layers.

A mature agentic mesh will need to speak all of these protocols, or at least provide adapters between them. This is exactly what service meshes did for HTTP, gRPC, and TCP. The precedent exists. The implementation does not, yet.

When You Actually Need a Mesh

Here is the part most vendor presentations skip.

Most organizations today deploy one or two agents. A coding assistant. A customer service bot. Maybe an internal knowledge agent. These do not need a mesh. They need good harnesses, as we covered in What Is an Agent Harness?, and solid orchestration patterns, as we documented in AI Agent Orchestration: From Prototype to Production.

A mesh becomes necessary when three conditions converge:

Multiple agents need to collaborate on shared tasks. Not just multiple agents existing in the same organization, but actively passing work between each other. An analysis agent that hands findings to a writing agent that routes drafts to a review agent. If your agents are independent, you need isolation, not a mesh.

Cross-team or cross-system agent communication. When marketing’s agent needs to query engineering’s agent, you have an identity and governance problem that point-to-point integrations cannot solve at scale. This is the same realization that drove the service mesh adoption in microservices.

Regulatory or compliance requirements demand auditability. If you must prove what every agent did, why, and who authorized it, you need the observability and identity layers that a mesh provides. Ad-hoc logging will not survive an audit.

If none of these conditions apply, building a mesh is over-engineering. And over-engineering in AI carries real costs: more latency, more token consumption, more failure surfaces, more things to debug at 2 AM.

What Honest Assessment Looks Like

Five concerns deserve direct acknowledgment.

Premature standardization. The agentic mesh concept emerged in late 2024. Most enterprise architectures take 5 to 7 years to mature from concept to reliable standard. We are in year two. Organizations adopting full mesh architectures today are betting on a specification that has not been tested at scale.

Vendor-driven narrative. Solo.io, Gravitee, Kong, and others are already selling mesh products. This is not inherently bad, but it means the urgency narrative (“you need this now or face AI chaos”) is coming from companies with products to sell. As we explored in Agent Teams and the Shift from Writing Code to Directing Work, the coordination challenge is real. The timeline pressure may not be.

Complexity tax. Every layer in the mesh adds potential failure points, latency, and operational overhead. A six-layer architecture that crashes at layer three is worse than a simple point-to-point integration that works. Start with the layers you need. Add the rest when the pain justifies the cost.

Protocol fragmentation. A2A versus MCP versus ACP versus ANP versus Agora versus LMOS. Choosing a protocol today means gambling on which one survives. The mesh is supposed to abstract this problem away. In practice, someone still has to build and maintain the adapters.

Identity and trust remain unsolved. Agent-to-agent identity at enterprise scale is an open research problem. OAuth and JWTs handle human-to-service authentication well. Agent-to-agent delegation chains (agent A asks agent B, which spawns agent C, which accesses database D) create trust propagation challenges that existing standards were not designed for.

The Pragmatic Path

Build the harness first. Get one agent working reliably in production. Then add orchestration for multi-step workflows. Only then consider mesh infrastructure, and only for the layers where you feel actual pain.

Layer 2 (identity and governance) is where most organizations should start their mesh investment. Not because it is the most exciting layer, but because it is the one whose absence causes the most damage. An agent without identity controls accessing customer data is not a technical problem. It is a liability.

Layer 5 (observability) is the next priority. You cannot govern what you cannot see. And right now, most organizations cannot see what their agents are doing.

The rest (communication fabric, model abstraction, enterprise integration) can grow incrementally. You do not need the full six-layer architecture to get value. You need the two layers that prevent the scenarios your legal team loses sleep over.

McKinsey’s warning is worth repeating: “Organizations delaying risk AI chaos that will be hard to resolve without rebuilding.” The key word is “delaying,” not “not implementing the full stack today.” The pragmatic move is to start with governance foundations and grow the mesh as your agent count and coordination needs demand it.

---

This analysis synthesizes AI Agentic Mesh (IEEE Computer Society, November 2025), Agentic Mesh: The Future of Enterprise Agent Ecosystems (InfoWorld, May 2025), What Is an Agent Mesh? (Nordic APIs, June 2025), Agent Mesh for Enterprise Agents (Solo.io, 2025), Best Practices and Principles for Agent Mesh Implementations (Gravitee, 2025), Gartner’s Context Mesh (Kong, 2026), Agent-2-Agent: Building the Agentic Mesh (Coforge, 2025), QuantumBlack/McKinsey (2025), Eric Broda’s Agentic Mesh (O’Reilly, 2025), and Gartner (August 2025).

Victorino Group helps organizations design the governance layer that makes AI agents reliable in production. Let’s talk.