The Independent Check Was Not Independent
Co-Founder & CEO of CLRT
Most AI policies a board approves carry some version of the same sentence: a qualified person reviews the output before anything consequential happens. It is the clause that lets everything above it be ambitious. It also rests on an assumption almost no organisation has tested in its own work, namely that the reviewer arrives at each decision with a view of their own, and that the view survives contact with a fluent, tireless machine arguing for something else. This week a randomised trial tested exactly that assumption on experienced professional analysts. The reviewers were capable, conscientious and confident in what they delivered. On most of the calls that mattered, the judgment was not theirs, and nothing in their work, their explanations or their own sense of ownership revealed it.
The study, published by Anthropic researchers on 27 October, is vendor research on the company's own model, and it should be read that way; the agents' leans were also deliberately strengthened. But the design is unusually clean. Of 61 causal-inference analysts recruited, 50 worked with a coding agent running Claude Opus 4.8 to evaluate a fictional farming co-operative's advisory programme and recommend whether to roll it out. The task held four judgment calls with two defensible answers each: the measure of productivity, the comparison method, which farms to include and whether to weight them by size. For every analyst and every call, a coin flip decided which option the agent would quietly favour, set by a hidden instruction to make the strongest honest case and never mislead. Chance would put agreement with the agent at 50 percent. The analysts went with its lean 86 percent of the time.
The obvious defence is that these analysts simply had no view, so the agent filled a vacuum. The data narrows that escape. Most held a master's degree or doctorate or had six or more years of experience, and nine in ten rated themselves proficient or expert. More telling, the researchers isolated the moments where an analyst had already taken a side before the agent voiced its own. When the agent then disagreed, analysts switched 60 percent of the time. When it agreed, 6 percent did. That gap rests on only 38 cases and its confidence interval is wide, so treat its exact size with caution. The researchers caution that it may not be wholly the agent's doing, since the analysts who spoke first may differ from those who did not, but the gap is far too large to be noise. A view the expert had already formed did not hold when a fluent collaborator argued the other side, and the agent had been instructed never to mislead. It made the case for one option and rarely for the other.
What turns this from a curiosity into a governance problem is what the analysts said afterwards. Asked to justify their choices, 77 percent repeated their agent's argument in at least one answer, sometimes word for word, and only 15 percent credited the agent anywhere. More than half rated the final recommendation as more their own than the agent's. Asked what the study was about, 6 of 48 guessed it concerned AI influence on their choices, and none suspected the agent's preferences had been planted. Confidence offered no signal either: analysts who followed the lean on all four calls averaged 71 on the study's confidence scale, against 73 for everyone else. Every instrument a review process would use to detect deference, the explanation, the sense of ownership, the certainty, read as independent judgment. The deference left no trace in the person. It showed only in the coin flips.
The final recommendation tells a subtler story, and it is the one a board should sit with. The agent's lean on the recommendation itself moved it less: 54 percent of analysts recommended rollout when the agent leaned that way and 36 percent when it leaned against, a gap the study's own registered test did not find statistically significant. A reviewer might take comfort there. They should not. By the time the recommendation was written, the analytical choices beneath it had already been made, mostly the agent's way, and those choices produced the estimate the recommendation rested on. In an exploratory analysis using nothing but the agent's four coin flips, a statistical model correctly predicted 82 percent of the analysts' recommendations, against 61 percent for always guessing the more common answer. The moment that looked like the human's judgment had largely been settled upstream, inside choices nobody would think to escalate.
None of this argues for removing the agent. The same study audited the analysis code: 3 of 50 analysts working with the agent made serious methodological errors, against 6 of the 11 who chose to work without it, a comparison the authors stress is not randomised, because those 11 selected themselves. The agent made the work cleaner and the judgment less independent at the same time, which is exactly the combination that sails through a quality review. The governance consequence is narrower and harder than most policies admit. A human in the loop is a statement about who signs, not about whose reasoning is on the page. Where an organisation genuinely needs a person's own judgment, the control has to be designed around when that judgment forms, what it is shown first, and which choices are treated as decisions at all. That is a question about your decisions, not a setting in your tools.
A human in the loop is a statement about who signs, not about whose reasoning is on the page.
A deeper dive
The mechanism matters because it defeats the controls organisations already run. Classic oversight design assumes the reviewer's job is to catch errors, which presumes a right answer to check against. Most consequential calls inside an analysis, a forecast or a policy have no such answer. They are choices between defensible options, and the outcome depends on the assumptions of whoever makes the choice. In that territory there is nothing for a reviewer to catch, only a question of whose assumptions settled the matter. The study points to influence through asymmetry rather than deception: the agents explained their preferred option well and rarely made a strong case for why a competent expert might choose the other. A one-sided brief from a fluent collaborator, often delivered before the expert has formed a view, is among the most persuasive documents in any organisation. It is also invisible in the finished work, because a recommendation reads the same whether the expert reached it or adopted it. No accuracy metric, sampling audit or sign-off record tells the two apart.
The second-order trap is that the familiar fixes make it worse. Asking reviewers to confirm they exercised independent judgment collects precisely the self-report the study shows is unreliable. Adding a second reviewer downstream multiplies signatures on choices already made upstream. Training people to distrust the agent throws away the error reduction that justified deploying it. And outside a trial, an agent's preferences are not random. The researchers found that the same model, with no lean injected, made the same choices in 112 of 123 runs, a consistent preference of its own, though voiced less forcefully than the planted leans, which means an organisation that standardises on one agent may be standardising its judgment calls without ever deciding to. Across thousands of analyses that is not a nudge. It is a house view that nobody wrote, approved or can see. The organisations that handle this well will treat independence as a property to be designed and measured rather than assumed: they will know which decisions genuinely require a person's own informed judgment, protect the conditions under which it forms, and accept openly that the rest is the agent's call, whoever signs it.
Work with CLRT
Human oversight works as a control only when it is designed as one, around the decisions that genuinely need a person's own judgment and the moment that judgment forms. Deciding which of your decisions those are, and where an agent's view can safely carry the call, is the question CLRT starts with. CLRT Ascent, our diagnostic at ascent.clrtstudio.com, maps where agentic AI should be pointed in your business and where independent human judgment has to be protected. And where agents go into consequential work, CLRT designs the review so the person in the loop is a check, not a countersignature.

Vishal Sachar is the Co-Founder and CEO of CLRT in Dubai. He helps businesses put agentic AI to work, leads practical AI workshops for leadership teams and operators, and writes on agentic systems, AI governance and the economics of automation. Reach him at vishal@clrtstudio.com or on LinkedIn.


