Agent PRs Do Not Converge Under Design Review

Three 2026 MSR studies (arxiv 2605.22534, 2606.13468, 2602.19441) measured the shape empirically: reviewer comments correlate positively with human PR merge (+2.7% per comment) and negatively with agent PR merge (-2.8% per comment); 38% of rejected agent PRs die of reviewer abandonment rather than an explicit fix-and-resubmit. Iteration is where human PRs converge and where agent PRs collapse. Local iteration on an agent PR still works; design iteration under review does not. The discipline move is to force the design decision upstream of the PR, and to treat 'this needs a rewrite' as a new PR against a sharpened spec.

By Travis Frisinger · October 15, 2026 · 9 min read
AI AgentsCode ReviewWorkflowDesign

Related reading: Where the Review Point Moved argued the review surface migrated upstream; this post names the specific empirical failure mode of running the old review turn on an agent PR anyway. AI Reviewing AI Is Not Review covered noise amplification when a second agent sits in the loop; this post covers iteration asymmetry when a human reviewer is still the one commenting.

The reviewer left fourteen comments on the agent’s pull request, got three rounds back with cosmetic edits, and closed the tab.

That tab closing has a number next to it now. Three 2026 MSR studies landed on the same shape from different angles. “Why Are Agentic Pull Requests Merged or Rejected?” (arxiv 2605.22534) looked at 11,048 closed agentic PRs against 9,799 human-reviewed ones and found that 38% of rejected agentic PRs died of reviewer abandonment rather than an explicit decision. “When AI Teammates Meet Code Review” (arxiv 2602.19441) measured the per-comment effect: roughly +2.7% on a human PR’s merge probability, roughly -2.8% on an agent PR’s. The AIDev rejection paper (arxiv 2606.13468) traced the same shape across a larger corpus. Three studies, same sign flip, same mechanism, same consequence.

The industry read the first wave of papers as a model-quality problem: the agent writes code the human review turn can mostly accept, with iteration patching the small gaps. The second wave reads differently. The sign on the iteration mechanic flipped. Every comment on a human PR makes the merge more likely. Every comment on an agent PR makes it less. That is not a quality curve; it is a workflow failure. More specifically, it is a design-review workflow failure, because local iteration on an agent PR still converges. The negative correlation lives in design iteration: the comment that names an intent the diff does not honor.

The Numbers Land on a Workflow Question, Not a Model Question

Same iteration mechanic, opposite outcome. The variable that moved is the author.

A reviewer comment is a signal. On a human PR, the signal travels into the author’s head, meets a working mental model of the change, and comes out as a diff that integrates the feedback. The signal is additive: comment one sharpens the diff, comment two sharpens it further, the merge probability climbs because each round brings the artifact closer to what the reviewer and author now jointly understand. The +2.7% per comment is the shape of that climb.

On an agent PR, the signal travels into a session whose context either dropped the original design frame or never held it in the first place. The agent reads the comment, re-plans against the local diff and whatever tests it can see, and produces a new diff. The new diff touches different lines. The intent the reviewer flagged is the same intent the new diff fails to honor, because the thing the reviewer is pointing at was never in the model being used. The second round looks like iteration from the outside. It is a fresh run of the same prompt with the review comment appended. The -2.8% per comment is the shape of that re-run.

Upgrading the model does not change the sign. A better optimizer runs the retrieve-and-replay faster. The retrieval still has the same gap. The replay still produces a diff that does not integrate the reviewer’s intent, because the integration mechanism assumes a mental model that persists across rounds, and the agent does not hold the kind of mental model a human author updates between rounds.

The sign flipped on authorship, not on quality. The author cannot be swapped out by upgrading the model.

Iteration Is Where Human PRs Converge and Agent PRs Retry

A human reviewer’s design comment triggers a reconsideration. The author re-reads the spec in their head, re-locates the boundary the reviewer named, and the diff moves. The movement is not uniform: sometimes the second round is a small edit, sometimes a near-rewrite, sometimes the author pushes back and the reviewer updates their own position. All three outcomes converge, because both sides hold a working model of the change and can update it between rounds.

An agent does not hold that kind of working model. It holds a session, a context window, and a diff. The context may carry the original prompt, the test suite that constrained the first run, and the review comment the second run has to address. It does not carry the chain of design decisions a human author assembled in their head between round one and round two. There is no persistent author in that sense. There is a replay.

Retry is not iteration. Iteration updates a model; retry re-executes against a slightly different prompt. The visible artifact can look similar. The method name changes. A null guard appears. A test case gets added. The thing the reviewer actually asked for (a change in what the diff is trying to be) does not appear, because the frame that would need to shift was never carried into the retry.

// Round one comment: "This flow treats the gift-card refund as a payment refund.
// They are different domain events; the ledger reconciliation downstream depends on it."
//
// Round two: the method got renamed; the enum got a new value; the ledger call still
// routes gift-card refunds through RefundPayment(). The reviewer's intent did not land.
//
// Round three: a comment is added above the call site explaining the distinction.
// The call still routes through RefundPayment(). The reviewer closes the tab.
public interface IRefundService
{
    RefundResult RefundPayment(PaymentRefundRequest request);
    RefundResult RestoreGiftCardBalance(GiftCardRestoreRequest request);
}

A human author in round two reads the comment, feels the ledger reconciliation point as a load-bearing distinction, and splits the call site. The agent in round two reads the comment, finds the closest surface edit that acknowledges the words in it, and ships the edit. One is a mental-model update. The other is a text transformation. The reviewer can feel the difference by round three.

Humans iterate. Agents retry. The empirical shape is what tells the two apart.

Reviewer Abandonment Is a Structural Failure, Not Reviewer Fatigue

38% of rejected agent PRs die because the reviewer stopped engaging. That is not an attention-budget problem. That is a convergence problem the reviewer diagnosed and refused to spend more rounds confirming.

The reviewer who abandons a human PR tends to do it after an argument: a disagreement about scope, a shift in priorities, a design the author will not change and the reviewer will not approve. There is a conversation on the way out. The reviewer who abandons an agent PR tends to do it after three rounds of non-convergence. There is no argument. There is a pattern the reviewer has started to recognize across every agent PR this quarter: the comment goes in, the diff comes back cosmetically different, the intent the comment named is still missing. By round three the reviewer stops writing comments, because the comments are not causing the thing comments are supposed to cause.

Calling the reviewer lazy misreads the measurement. The reviewer is responding to a signal, and the signal is honest. The loop is not converging. More rounds will not help. The reviewer closes the tab because the reviewer has learned, across the previous twenty agent PRs, that a fourth round is noise. The 38% is the aggregate of that learning across many reviewers and many PRs. It is a diagnosis delivered in the only language a review tool records: the absence of further activity.

A rational reviewer with a working heuristic produces this shape. The failure is upstream of the reviewer, in a workflow that routes design feedback through a review turn the agent cannot honor.

Local Iteration on an Agent PR Still Works

Not every review comment names an intent. Some name a surface defect: a rename, a missing test case, a one-function refactor, a boundary adjusted by a few lines. The agent integrates these in one round. The diff updates. The PR merges. The loop that still works is the loop whose feedback does not require a model update.

The rough heuristic sorts the two cases at the comment layer. If the comment can be answered by editing the diff without changing the spec that governs it, the comment is local. The agent re-reads the diff, applies the edit, and produces a new diff that converges. If the comment can only be answered by changing the spec (because the reviewer is pointing at an intent the current spec does not express), the comment is design. The agent, lacking a durable model of the spec, retries against the old spec with cosmetic awareness of the comment. The diff does not converge.

The aggregate -2.8% per-comment correlation averages the two cases. Local iteration pulls the number up; design iteration pulls it down harder. The sharp move is not to refuse all iteration on agent PRs. The sharp move is to route each kind of feedback to the workflow that handles it. Local comments stay on the current PR. Design comments move upstream.

Treating every agent PR comment as the same kind of comment is where the sign flip bites. Separate the kinds; separate the workflows.

Move the Design Decision Upstream of the PR

The upstream move is concrete. When the reviewer’s comment amounts to “this needs a different design,” the next action is not another round on the current PR. The current PR closes. The design decision commits to main as a new scenario test, a new contract on the seam the reviewer flagged, a new named domain constraint the next agent run will see. A fresh PR opens against the sharpened surface. The spec the agent must satisfy is now the thing the previous round of review was trying to produce.

// Sharpened spec on main (committed before the agent's next PR):
[Fact]
public void A_gift_card_refund_restores_balance_without_touching_the_payment_ledger()
{
    var refund = aRefundFor(aGiftCardPurchase().costing(40.dollars()));

    var result = refundService.process(refund);

    result.Should().Satisfy(giftCardBalanceRestoredBy(40.dollars()));
    paymentLedger.Should().NotHaveEntriesFor(refund.transactionId);
}

The scenario names the distinction the reviewer spent three rounds trying to introduce. The agent’s next PR runs against a suite that fails until the distinction is honored. The diff converges in one shot, because the thing the reviewer wanted the agent to understand is now a test the agent cannot ship past. The review conversation on the new PR is two nits and a rename.

The move puts the design decision where it can land cleanly: committed to main as a durable artifact, visible to the next session as a constraint rather than as a comment on a diff that no longer exists. The reviewer has produced a spec, not a backlog of comments. The agent has a surface to converge against, not an intent to guess from English prose.

The workflow rule follows from the empirical shape. If the first round of design feedback does not converge, do not run a second. Close the PR. Sharpen the spec. Open a new PR against the new surface. Iteration on the current PR was never the artifact; the sharpened spec was.

The reviewer who left fourteen comments on the agent’s PR and closed the tab was reading the measurement correctly. The next move is not a fifteenth comment. The next move is a scenario test on main that the agent cannot replay past.

Iteration is a model update. Retry is not. A workflow that pretends they are the same ships rejected PRs; a workflow that routes them separately ships merged ones.