A human approval button proves who had authority. It does not prove the person formed an independent view before the AI agent advocated one option.
Anthropic recently randomized which of two defensible analytical choices a Claude Code agent would favour. Across four choices, 50 causal-inference practitioners matched the hidden lean 86% of the time. The practical response is not to distrust every agent. It is to capture a small “before photo” for consequential judgment calls.
Use this record when more than one reasonable interpretation exists and the choice affects money, people, customers, policy, safety or reputation.
Step 1: Present the decision without the recommendation
Give the accountable person:
- the decision question;
- the raw evidence or a neutral evidence summary;
- the known constraints;
- the deadline;
- the plausible options, without marking one as preferred.
Do not include the agent's conclusion, persuasive ordering or generated rationale yet.
Pass condition: the reviewer can state a provisional choice using the evidence rather than guessing what the agent wants.
Step 2: Capture the human baseline
Record these fields before revealing the agent recommendation:
- Decision ID:
- Accountable person:
- Question:
- Provisional choice:
- Confidence (0–100):
- Decisive reason:
- Most important unknown:
- Evidence reviewed:
- Timestamp:
Keep the reason short. One or two sentences are enough to create a meaningful comparison.
Pass condition: the record is timestamped and cannot be silently overwritten by the agent's later wording.
Step 3: Reveal the agent's case
The agent should now provide:
- its recommended choice;
- the evidence supporting it;
- the strongest counterargument;
- material uncertainty;
- what evidence would reverse its recommendation;
- any policy or permission boundary that applies.
Do not ask the agent to produce fake consensus. A useful recommendation makes its assumptions inspectable.
Pass condition: fact, interpretation and forecast are visibly separated.
Step 4: Capture the final decision
Record:
- Final choice:
- Final confidence (0–100):
- Changed from provisional view? yes / no
- Evidence or argument that caused the change:
- Decision owner:
- Required approval or escalation:
- Outcome review date:
Changing the answer is not failure. Changing it without being able to name the evidence is the signal to inspect.
Pass condition: the final rationale does not merely copy the agent's prose without attribution or evidence.
Step 5: Review the outcome
When the result becomes observable, add:
- outcome achieved;
- serious errors or near misses;
- which original assumption was right or wrong;
- whether confidence was calibrated;
- what the human or agent should do differently next time.
This turns disagreement into organisational learning instead of a one-off argument.
Copyable decision card
PRE-AGENT VIEW
Decision ID:
Owner:
Question:
Provisional choice:
Confidence (0–100):
Decisive reason:
Most important unknown:
Evidence reviewed:
Timestamp:
AGENT VIEW
Recommendation:
Evidence:
Strongest counterargument:
Uncertainty:
Reversal condition:
FINAL VIEW
Final choice:
Final confidence (0–100):
Changed? yes / no
Why it changed or held:
Approval / escalation:
Outcome review date:
OUTCOME
Observed result:
Serious errors / near misses:
Calibration lesson:
Next change:When to use it
Use the record for:
- supplier or investment selection;
- hiring and performance decisions;
- incident-response hypotheses;
- policy exceptions;
- fraud, credit or insurance review;
- allocation of scarce staff or capital;
- analytical assumptions that materially change the conclusion.
Skip it for reversible, low-consequence tasks such as formatting, routine extraction or spelling corrections. A form attached to every trivial action trains people to click through forms.
Metrics worth tracking
Review the records monthly:
- initial human–agent disagreement rate;
- final change rate;
- changes supported by new evidence;
- copied or unattributed agent reasoning;
- decision time;
- serious-error rate;
- confidence calibration;
- categories that repeatedly require escalation.
Do not optimise for agreement or disagreement alone. High agreement may mean the agent is excellent. It may also mean the agent owns the frame. Outcomes and reasons distinguish the two.
Evidence and limitations
Anthropic's first-party report and technical paper describe a randomized trial in which 50 causal-inference practitioners used Claude Code on a fictional task. Hidden coin flips assigned the agent's lean on four analytical decisions. Participants matched that lean 86% of the time, 77% later used at least one agent talking point and 15% credited the agent as a source.
The trial used one model, one field and one approximately hour-long fictional case. The agent's preferences were deliberately strengthened. The results do not establish an 86% deference rate for normal production work, and the direct effect on the final rollout recommendation was smaller and not statistically significant in the primary analysis.
Rapkyn's inference: Over the next 12–36 months, high-consequence agent audit trails will increasingly preserve the accountable human's pre-agent view alongside the final decision. Judgment provenance will become as important as recording who clicked approve.