Your IDE Became a Runtime: How Agentic Coding Rewrote Code Review

When most pull requests are drafted by an agent and shepherded by a human, the old review checklist measures the wrong things. Here is what replaces it, and what to gate instead. This techstrendery.com report collects the checklist changes, CI gates and ownership rules teams adopted after the switch to agentic coding.

The pull request looks exactly as it did five years ago: a branch, a diff, a description, a row of checks, an approve button. What has changed is the distribution behind it. On teams that have handed routine work to coding agents, a large and growing share of open PRs were drafted by a model and shepherded by a human who reviewed the result at a far higher level of abstraction than they would have written it. The artefact is unchanged. The evidence it carries is not.

That mismatch is the whole problem. Review checklists are compressed inferences about an author. “Is the naming consistent with the rest of the module?” was never really a question about naming; it was a cheap proxy for whether someone had read the surrounding code. “Does this look like the author understood the failure mode?” was a proxy for whether they had. Those proxies were reliable because producing idiomatic, well-named, internally consistent code used to be expensive. It is now nearly free, and it decouples cleanly from comprehension.

The practical consequence is not that review matters less. It is that the signal has moved: out of the diff and into intent statements, tests, and the gates that run without a human in the loop. Below is where each piece goes, and what a revised checklist looks like when you stop pretending the author is a person who typed every line.

A clean diff used to be evidence that someone thought it through. It is now evidence of nothing in particular.

Key takeaways

• Style consistency, plausible naming and a tidy diff no longer signal that anyone understood the change. Stop treating them as evidence.
• Review the intent statement and the tests first; the diff is the output, not the reasoning. Verify each new test actually fails without the change.
• CI gates become load-bearing rather than advisory: changed-line coverage, mutation score, suppression counters, lockfile diffs and dependency allowlists.
• Accountability sits with the human who merges. Agents belong in commit trailers, never in CODEOWNERS, and never with self-merge rights.
• Treat volume as a throughput problem: one PR per intent, a WIP cap per reviewer, scheduled review windows, and sampled deep audits for mechanical changes.

The checklist items that quietly stopped working

Four common checklist items now misfire. First, stylistic consistency: an agent matches the surrounding file’s conventions by construction, so consistency tells you nothing about whether it understood the file. Second, confidence-as-signal: a crisp, minimal diff with no leftover debugging used to correlate with care; it now correlates with the model’s output distribution.

Third, comment quality: generated comments describe the code’s mechanics fluently and say nothing about why the change was made, which is the only part a reviewer cannot reconstruct. Fourth, “did the author consider X”: there is no author to have considered anything, only a transcript you probably cannot see.

Meanwhile, a set of failure modes that were rare in hand-written PRs became common enough to deserve dedicated checks. Scope creep inside a single change, where a bug fix quietly refactors two adjacent functions. Reimplementation of a helper that already exists three directories away, because the agent’s context window did not reach it. Tests relaxed rather than code fixed when something went red.

Error handling that catches broadly and logs, satisfying a lint rule while erasing the failure. New dependencies added to solve a problem the standard library already solves. None of these look wrong line by line. They look wrong at the level of the change as a whole, which is precisely the level a line-oriented diff view suppresses.

Read the intent and the tests before the diff

Invert the reading order. Start with the stated intent: what behaviour is supposed to change, what invariant is supposed to hold, and what was explicitly out of scope. If the PR description is a restatement of the diff, send it back. A description that says “adds retry logic to the payment client” is a summary; “payment submissions must be idempotent across retries, so the client now sends a caller-supplied key and treats 409 as success” is an intent you can review against.

Then read the tests, and apply one rule mechanically: a new test must fail without the change. Agents are good at writing tests that pass either way, because passing tests are what the training signal rewards. The crude verification is to revert the source hunk locally and confirm the test goes red.

The systematic version is mutation testing scoped to changed files, which flips operators and return values and reports what your suite fails to notice. A changed-file mutation score is expensive to run repository-wide and cheap to run on a diff, which makes it one of the few new gates that pays for itself.

Only then read the diff, and read it for shape rather than syntax: what did this touch that the intent did not mention?

Gates stop being advisory

When a human wrote every line, CI was a backstop for mistakes a careful person might still make. When an agent writes the line, CI is the review. That shifts which gates are worth the wall-clock cost.

The ones that earn their place now are the ones an agent can satisfy only by doing the right thing: type checking with no new suppressions, a counter that fails the build if the count of ignore directives, disable comments or suppressed warnings rises without a linked issue, changed-line coverage rather than repository coverage, a lockfile diff gate that requires explicit approval for new transitive dependencies, and performance or query-count budgets on hot paths.

Build provenance matters more too: if a machine account can open and merge changes, you want to be able to prove which source and which build process produced a released artefact, which is the problem the SLSA build levels exist to describe.

There is an obvious Goodhart risk here, and it is worse than usual because agents optimise directly against the visible objective. A coverage gate produces tests that execute code without asserting on it. A lint gate produces suppressions. Every gate you add needs a paired counter-gate that measures the cheap way out: coverage plus mutation score, lint plus a suppression count, dependency scanning plus a lockfile diff.

The gate pairs described above, side by side:

Gate latency becomes a real design constraint. If the full pipeline takes forty minutes, agent-opened PRs queue behind each other and humans start merging on partial results; a two-tier pipeline, with a fast gate on every push and the expensive suite in a merge queue, is usually the workable compromise. Agents that hit failing gates repeatedly also cost money, as our 30-day log of AI agent token costs showed with retry loops.

Every gate you add needs a paired counter-gate that measures the cheap way out.

Ownership does not transfer to the tool

Write the rule down before you need it: the human who merges is the author of record. Not the human who wrote the prompt, not the agent, not the platform team that provisioned the agent. Merging is the act that puts code in production, and it is the only point in the chain where a person exercises judgement that can be audited afterwards.

Three mechanics make that rule stick. Machine accounts never appear in CODEOWNERS, because code owner approval is supposed to represent a human who will be paged; CODEOWNERS itself should be owned by a team that the agent’s account cannot modify. Agent-opened PRs cannot be self-merged by the account that opened them, regardless of who triggered the run.

And commits carry a trailer linking to the task or session that produced them, so that six months later, when bisect lands on the commit, you can recover what was actually asked for rather than guessing from the diff. Limiting what an agent account can do is the same containment principle our prompt injection field guide applies to agent credentials, and our cybersecurity coverage follows the supply-chain side of the problem.

In incident review, “the agent wrote it” is not a root cause. The useful questions are which gate should have caught it, why the reviewer’s attention was elsewhere, and whether the intent statement was specific enough to review against. All three are process defects with owners. For the wider shift in developer tooling, see the software reporting on techstrendery.com.

Fatigue is a throughput problem, not a discipline problem

Review capacity is roughly fixed. If a team’s weekly PR count triples while headcount stays flat — a pattern worth measuring on your own repositories rather than assuming from anyone’s published figure — average attention per PR falls by two thirds, and it does not fall evenly. It falls hardest on the PRs that look routine, which is exactly where agent output hides its problems.

Four controls help, in rough order of effect. Cap work in progress: a hard limit on open agent PRs per reviewer, enforced by the bot refusing to open more, converts an unbounded queue into a bounded one. Enforce one PR per intent: an agent asked to fix three bugs should open three PRs, each independently revertible, because a revertible change is a change you can approve faster.

Batch review into scheduled windows rather than servicing interrupts, since context-switching cost dominates at small PR sizes. And split mechanical changes — dependency bumps, codemods, generated clients — into a separate lane that is gated automatically and sampled rather than read line by line.

Sampling deserves a number you choose deliberately. Read one in five mechanical PRs in full, selected randomly, and record the defect rate you find. If it stays at zero for a month, widen the interval. If it does not, the gate is wrong, not the sample.

The revised checklist

This is short on purpose. A checklist that takes longer than the review is theatre.

• Intent: does the description state the behaviour change and the invariant, separately from the diff? Does it name what was out of scope?
• Blast radius: does the diff touch anything the intent did not mention? Every unmentioned file is a question, not a nit.
• Test falsifiability: does at least one new test fail with the source change reverted? Is there a negative case, not only the happy path?
• Suppressions and escapes: any new ignore directives, disabled rules, widened exception handlers, or relaxed assertions? Each needs a linked issue.
• Dependencies: any new direct or transitive dependency, and does the standard library or an existing internal helper already do this?
• Duplication: does a helper with this behaviour already exist elsewhere in the repository? Search before approving, because the agent’s context probably did not.
• Reversibility: can this be reverted as a single commit without a data migration? If not, what is the rollback plan?
• Accountability: is a human code owner approving, is the merging account a person, and does the commit link back to the originating task?

What to do on Monday

Pick one repository and instrument it before changing anything. Count PRs per week by author type, median time-to-first-review, and the proportion of changes reverted or hotfixed within seven days. Without that baseline you cannot tell whether a new gate helped or simply slowed you down, and the reverted-within-a-week figure is the one that correlates with the outcomes long-running delivery research keeps pointing at.

Then make three changes in order. Add the falsifiability rule to your review template, because it costs nothing and catches the most common class of generated-test problem. Add a suppression counter to CI, because it is a few lines of script and it closes the cheapest escape hatch. Remove machine accounts from CODEOWNERS and disable self-merge on bot-opened PRs, because that is the change you will wish you had made when something goes wrong at 3am.

Everything else — mutation scoring, WIP caps, a separate mechanical lane, merge queues — can wait until the baseline tells you which constraint is actually binding. The review practice that survives this transition is not the one with the longest checklist. It is the one that stopped spending human attention on questions a machine now answers for free, and started spending it on the two questions no gate can answer: is this the change we wanted, and can we undo it.

Sources and further reading

• NIST SP 800-218: Secure Software Development Framework (SSDF) v1.1
• GitHub Docs: About code owners (CODEOWNERS)
• SLSA v1.0 Specification: Build levels
• Git documentation: git-bisect
• DORA Research (software delivery performance and AI)

How this article was produced

This article was researched, written and fact-checked by the Editorial Team and edited to the techstrendery.com editorial standards described on our About us page. Figures described as illustrative are exactly that and are labelled in the text. More reporting on this beat is collected under Software. If you find an error, tell us through the contact page; corrections are published in place with a dated note.