The Reviewer Fails on Schedule

Start with the cleanest measurement of what a reviewer actually catches. In September 2025, researchers at LMU Munich and the University of Maryland posted a preprint of a randomised experiment in which 2,784 people checked readings of corporate emissions tables presented as AI-generated and seeded with planted mistakes. The reviewers corrected 82 percent of the swapped digits. They corrected roughly 77 percent of cases where the right number sat in the wrong cell. When spotting the error required understanding the reporting rules, the catch rate fell to 31 percent. On one further table the AI was right, but confirming it needed the distinction between market-based and location-based Scope 2 emissions, and only 21 percent of reviewers handled it correctly; most rejected a correct answer. The reviewer is reliable exactly where the machine is already good, on the surface, and unreliable where the machine is weakest, on meaning, in both directions. That is the inverse of what the sign-off is supposed to provide. The task was annotation, not a boardroom decision, but the shape of the curve recurs everywhere anyone has measured it.

The shape was known long before language models. Parasuraman and Manzey's 2010 review in Human Factors compressed two decades of automation research into one finding: automation bias occurs in both naive and expert participants, cannot be prevented by training or instructions, and affects teams as well as individuals. A 2012 systematic review in the Journal of the American Medical Informatics Association put a base rate on it. Across four clinical studies, between 6 and 11 percent of decisions that were correct before the decision-support system spoke became wrong after it did, and pooled, erroneous advice was 26 percent more likely to be followed when a system delivered it. A later review in the same journal found the bias even in single tasks, whenever verifying the aid is cognitively expensive. The harder an output is to check, the more the checker defers. Agent output, long, fluent and finished-looking, is the hardest class of output a reviewer has ever been handed.

The checker's second property is that it degrades with exposure, and this has now been measured in production. A June 2026 preprint tracked 400 repeat reviewers across 11,429 reviews of AI-agent code in open-source repositories over seven months. Between their early and late reviews, approval rose from 30.1 percent to 36.8 percent. The gap from the first to the tenth decile of experience was 14.5 points. Inline comments fell by 22 percent, and the time a request waited for review rose 3.5 times. The effect held after controlling for calendar time and did not appear for human-written code, where approval fell. The authors read this as most consistent with reflexive habituation under a growing workload rather than trust calibration on its own. Nobody decided to stop reading. The queue grew, the output looked fine, and the stamp came down a little faster each month.

The instinctive response is to put a better reviewer on it, and here the evidence is least comfortable. In a 2023 study in Radiology, 27 radiologists read 40 test mammograms with a purported AI suggestion attached, twelve of which were deliberately wrong. With correct suggestions, accuracy was around 80 percent at every experience level. With wrong ones, inexperienced readers fell to 19.8 percent, moderately experienced readers to 24.8 percent, and the very experienced to 45.5 percent. Seniority bought back some accuracy and still lost more than a third. Training does no better. A 2025 randomised trial in Pakistan, a preprint, recruited 44 physicians who had completed a 20-hour AI-literacy course and let all of them consult a model; for half, its advice carried planted errors in three of six cases. The physicians whose model was seeded with errors scored 14 points lower after adjustment, 73.3 percent against 84.9 for those given error-free advice. The standard corporate remedy, an experienced person who has been on the course, is the configuration these studies tested.

Put the findings together and the checker has a specification. It catches surface errors and passes errors of meaning. Its approval rate drifts upward with exposure while its inspection effort drifts down. Its accuracy collapses in proportion to how convincing the wrong answer is; neither seniority nor training restores it. It also responds to friction in the wrong direction: in the emissions experiment, requiring reviewers to type a correction for every flagged error produced fewer corrections and more acceptance of wrong suggestions, and attitude towards AI predicted accuracy better than any demographic; the sceptics caught more. A policy that says a human reviews every output has therefore made a claim about a component whose failure curve is published. The question that decides whether an agent is safe to run is not whether a person is in the loop but which errors that person can physically detect, at what volume, for how long, and what catches the rest.