- Home
- The Thinking Wire
- The Operations Discipline Gap: What Cloudflare and AWS Reveal About Governing Automated Systems
The Operations Discipline Gap: What Cloudflare and AWS Reveal About Governing Automated Systems
On February 20, 2026, a routine cleanup task at Cloudflare withdrew 1,100 IP address prefixes from the internet, triggering a six-hour outage. Two months earlier, AWS’s internal AI coding tool reportedly deleted and recreated an entire production environment over thirteen hours.
These are different failures. One involved zero AI — a traditional code defect amplified by automation. The other involved a non-deterministic AI agent operating with production-level permissions. But they share a root cause that matters more than the technical details: both organizations deployed automated systems faster than they applied operational safeguards they already knew how to build.
This is the operations discipline gap. Not a knowledge gap. Not a technology gap. A discipline gap.
The Cloudflare Failure: Known Engineering, Unapplied
Cloudflare’s post-mortem is a masterclass in honest disclosure. Here is what happened.
A buggy automated cleanup task queried an internal API with an empty pending_delete parameter. The API server, rather than returning an error or an empty set, returned all 4,306 BYOIP prefixes in the system. The cleanup task dutifully withdrew them. Over fifty minutes, 1,100 prefixes — roughly 25% of the total — disappeared from the internet. Total outage duration: six hours and seven minutes.
This is not a novel failure mode. Every experienced engineer recognizes the pattern: an API that interprets missing parameters as “select all” instead of “select none.” It is the database equivalent of DELETE FROM users WHERE id IN () silently becoming DELETE FROM users. The bug itself is mundane. What makes it instructive is what was missing around it.
No circuit breakers. No blast radius limits. Staging data that did not match production scale. No health-mediated deployment that would pause a rollout when downstream metrics degraded.
Cloudflare’s remediation list reads like a checklist of practices that already exist in their own engineering culture — API schema standardization, circuit breakers, health-mediated deployments. These are not inventions. They are applications of known operational discipline to a system that grew faster than its governance.
The pattern is familiar. CrowdStrike’s July 2024 update pushed to 8.5 million systems simultaneously because their deployment pipeline lacked staged rollouts and kill switches. Same discipline gap, same category of preventable failure, same “we already knew how to prevent this” remediation.
The AWS Kiro Incident: A Different Category of Problem
The AWS incident is more complex, more contested, and more revealing.
In December 2025, AWS’s internal AI coding tool Kiro reportedly deleted and recreated an entire production environment over approximately thirteen hours. Amazon’s official position is clear: this was a permission misconfiguration, not an AI autonomy failure. The impact, Amazon says, was limited to Cost Explorer in a single region, and they received no customer inquiries about it.
A senior AWS employee, speaking to the Financial Times, called the incident “entirely foreseeable.”
Both framings deserve attention because both point to the same governance question. If the cause was misconfiguration — a human gave a tool too much access — then the question is: why did the permission model allow a single tool to delete an entire environment? If the cause was the AI agent making autonomous decisions beyond its intended scope — then the question is: why were there no constraints on what actions it could take?
Either way, the governance was insufficient. And this is where the Cloudflare and AWS incidents diverge in an important way.
The Deterministic-Nondeterministic Divide
Cloudflare’s failure is deterministic. Given the same buggy code and the same API behavior, the outcome would be identical every time. You can write a test for it. You can reproduce it in staging. You can build a circuit breaker that catches it. The engineering practices are well-established.
AI agent failures are non-deterministic. The same prompt, the same permissions, the same context can produce different actions on different runs. An AI coding tool that helpfully refactors a module on Monday might decide to delete and recreate the entire service on Tuesday. The model’s reasoning is probabilistic, and its action space — when given broad permissions — is effectively unbounded.
This distinction matters because organizations are applying the same trust model to both categories. A deployment pipeline gets the same level of access control whether it runs a deterministic script or an AI agent that interprets instructions creatively. A tool that can read code to suggest improvements often has write access to the same codebase. The permission model does not distinguish between “will do exactly what the code says” and “will do what it interprets as helpful.”
This is a category error. Deterministic automation needs safeguards against bugs in the code. Non-deterministic automation needs safeguards against emergent behavior from the agent itself. The second category requires fundamentally different governance: not just “did the code execute correctly?” but “should the agent have taken this action at all?”
The AI SRE Illusion
If you look at the vendor landscape for AI-powered operations, the picture appears promising. PagerDuty, Datadog, incident.io, Rootly, Cleric, Resolve.ai, Anyshift.io, RunWhen — all are shipping AI capabilities for incident diagnosis and mitigation. Microsoft is building Copilot into Azure operations. The tools exist.
But they are solving the wrong half of the problem.
Lorin Hochstein, formerly of Netflix’s CORE team and staff SRE at Airbnb, identifies the fundamental limitation: “Incident response is a team sport.” A single AI agent, no matter how capable, cannot coordinate the multi-perspective response that complex incidents demand.
The reason is fixation. A single agent pursuing a hypothesis will continue down that path, deepening its analysis, potentially ignoring signals that contradict its initial interpretation. Breaking fixation — recognizing that your current hypothesis is wrong and pivoting to a different one — requires diverse perspectives. It requires someone looking at the same system from a different angle and saying “have you considered that the database is fine and the problem is actually in the load balancer?”
This is not a theoretical concern. It is why experienced incident response teams assign specific roles: an incident commander who maintains the big picture, subject matter experts who deep-dive specific systems, a communications lead who tracks what has been tried and what has not. The multi-agent structure is not bureaucracy. It is a cognitive architecture designed to break fixation.
Current AI SRE tools are single-agent systems bolted onto existing incident workflows. They can surface relevant logs, suggest runbooks, and accelerate diagnosis of known failure patterns. They cannot coordinate a response to a novel failure across multiple teams and systems. The gap between “AI-assisted diagnosis” and “AI-coordinated incident response” is not a feature gap. It is an architecture gap.
The Measurement Void
A survey of approximately 500 platform engineering practitioners found that 29.6% have no success metrics at all, and 24.2% cannot determine whether their platforms are actually improving. Over half of the people building internal platforms — the infrastructure that other teams depend on — cannot tell you if their work is making things better or worse.
This is the measurement void that makes the discipline gap invisible. You cannot close a gap you cannot see.
The temptation is to add metrics. More dashboards. More KPIs. More quarterly reviews with color-coded scorecards. But Goodhart’s Law applies with particular force here: when a measure becomes a target, it ceases to be a good measure. Teams that are measured on deployment frequency will deploy more often. That does not mean they are deploying better.
The DORA and SPACE frameworks, used together, offer a more honest measurement approach. DORA captures system performance — deployment frequency, lead time, change failure rate, recovery time. SPACE captures the human dimension — satisfaction, performance, activity, communication, efficiency. Neither alone is sufficient. Both together create a picture that is harder to game because it triangulates from multiple angles.
But measurement is infrastructure. It requires instrumentation, collection, analysis, and — critically — the organizational willingness to act on what the numbers reveal. Most organizations that deploy AI systems do not have this measurement infrastructure. They are operating automated systems without the telemetry to know whether those systems are helping or hurting.
The Governance Maturity Gap
The numbers are stark. A Strata and Drexel University study found that 41% of organizations are using agentic AI, but only 27% have mature governance frameworks for it. Kiteworks surveyed 225 security leaders and found that 63% cannot enforce AI purpose limits — meaning they cannot restrict an AI agent to its intended function — and 60% cannot terminate a misbehaving agent.
Read that again. Sixty percent of organizations cannot stop an AI agent that is doing something it should not be doing. Not “choose not to.” Cannot.
This is the equivalent of deploying a fleet of autonomous vehicles without brakes. The acceleration technology is mature. The steering is adequate. The ability to stop is missing.
Gartner predicts that 40% of firms will face shadow AI security incidents. Forrester predicts an agentic AI breach in 2026. These are not alarmist projections. They are extrapolations from the current governance maturity numbers applied to the current deployment velocity.
Two Gaps, One Root Cause
The Cloudflare outage and the AWS Kiro incident represent two distinct governance failures that share one root cause: governance treated as a follow-on project rather than a prerequisite.
For deterministic automation — the Cloudflare category — the governance practices are well-known. Circuit breakers. Blast radius limits. Staged rollouts. Health-mediated deployments. API schema validation. Kill switches. These are not innovations. They are standard operating procedures that need to be applied consistently to every automated system, including the “simple” internal cleanup tasks that nobody thinks to govern.
For non-deterministic AI agents — the AWS category — the governance practices are still emerging, but the principles are clear:
Scope constraints. An AI agent should not be able to take actions outside its defined purpose. A coding assistant should not be able to delete production infrastructure. The permission model must be narrower than the agent’s capability, not equal to it.
Action budgets. Limit the total impact an agent can have in a given time window. If a coding tool deletes more than N resources in an hour, it stops and asks for human review. This is the circuit breaker pattern applied to non-deterministic systems.
Reversibility requirements. Before an agent takes an action, evaluate whether the action is reversible. Irreversible actions — deleting resources, modifying production data, withdrawing network prefixes — require explicit human approval regardless of the agent’s confidence level.
Behavioral monitoring. Track not just whether the agent succeeded at its task, but how it pursued its task. An agent that achieves its goal by deleting and recreating an environment is operating differently from one that makes targeted modifications. The outcome may be the same. The operational risk is not.
Multi-agent oversight. For high-stakes operations, a second agent — or a structured review process — should evaluate proposed actions before execution. This is the fixation-breaking architecture that current AI SRE tools lack.
The Discipline, Not the Technology
The common thread across every failure discussed here — Cloudflare, AWS, CrowdStrike — is not technological limitation. The safeguards existed or could have been built with known engineering practices. The gap is discipline: the organizational commitment to apply operational governance to every automated system, especially the ones that seem too simple or too internal to warrant it.
The internal cleanup script that nobody reviews. The AI coding tool that runs with admin permissions because it was easier to configure. The deployment pipeline that pushes to all regions simultaneously because staged rollouts were “planned for Q3.” The monitoring dashboard that exists but nobody checks because there is no defined response procedure.
These are not technology problems. They are governance problems. And governance is not a product you buy or a tool you deploy. It is a practice you build, staff, maintain, and enforce. Every day. On every system. Including the boring ones.
Especially the boring ones.
Thiago Victorino is the founder of Victorino Group, a consulting firm that helps organizations build the governance and operational infrastructure for AI systems. For more on AI operations strategy, visit victorinollc.com or reach out at contact@victorinollc.com.
If this resonates, let's talk
We help companies implement AI without losing control.
Schedule a Conversation