What changes when AI agents write production code — and what should remain human.
The Core Question
AI coding agents — Cursor, Claude Code, GitHub Copilot, Kiro, Windsurf, and others — now select dependencies, generate implementations, execute build steps, create pull requests, and push changes to repositories. They produce code faster than any human reviewer can inspect it.
This creates a governance question that traditional software engineering hasn't had to answer: how do you maintain the checks and balances of a mature engineering organization when the developer is an autonomous agent?
What Changes When Agents Write Code?
Human review cannot scale to agentic output
A single engineer with agentic tooling can produce hundreds of files per day. Reviewing every line is no longer feasible. Organizations must decide what requires human judgment and what can be independently verified by machines — and build systems that enforce that separation.
Self-review is structurally meaningless
When an AI builds the code and then an AI reviews the code — using the same model, same training, same blind spots — the review adds no independent signal. Effective governance requires a different model, different prompt, and adversarial posture. The builder cannot be its own checker.
Intent can drift silently
An AI agent can produce code that compiles, passes tests, and appears correct — but no longer accomplishes what the human originally authorized. This is especially dangerous in regulated domains (healthcare, finance, security) where "works correctly for the test cases" is not the same as "satisfies the regulation."
Dependency decisions compound risk
AI agents add dependencies without the deliberation a human developer applies. They optimize for "solve the immediate problem" rather than "is this package maintained, secure, and appropriate for our trust boundary?" Supply chain risk accumulates faster in agent-driven development.
Security at sprint cadence is too slow
If agents deploy daily and attackers operate in minutes, quarterly security reviews are structural obsolescence. Security verification must be continuous, embedded in the development loop, not applied as a phase gate after the code is complete.
What Should Remain Human?
Strategic decisions — what to build, what to kill, what to hold
Go/kill/hold at gates — the decision to ship is never delegated
Compliance interpretation — regulatory applicability requires legal and domain judgment
Architecture tradeoffs — consequential design decisions with long-term implications
Risk acceptance — formally accepting known risks with documented rationale
Accountability — a human is always responsible for what ships
What Should Become Automated?
Test execution — every commit, every merge, continuously