THE FIELD GUIDE · DECISION AID
The AI Answer Sounds Plausible. What Do You Verify?
You ask your approved AI tool to turn three source documents into a one-page background note for an upcoming meeting.
The result is lovely.
Clean bullets. Sensible headings. Very authoritative.
One line says:
“The customer is prioritizing European expansion next year.”
You stare at it.
You remember reading that the customer opened two European offices.
You do not remember reading that European expansion was a stated priority.
Is the AI wrong?
Maybe.
But first, we need to be more precise about what kind of claim it just made.
Do not start by fact-checking every comma
Start with the claims that would actually matter if they were wrong: the number somebody will repeat, the deadline somebody will plan around, the attribution that will go into a document, the conclusion that changes your recommendation.
Those deserve your attention before a harmless turn of phrase.
Then trace those claims back.
Source → inference → unsupported
What would you check first?
In our example, imagine the sources actually say:
Source says
Source 1: The company opened offices in Paris and Berlin.
Source 2: An executive says international growth remains important.
AI infers
“European expansion is a top priority next year.”
That may be a perfectly plausible interpretation.
It is still an interpretation.
That distinction matters if your meeting preparation, recommendation, or messaging depends on it.
Model language
“The sources confirm two European office openings and describe international growth as important. I can’t find support for the stronger statement that European expansion is a stated priority for next year, so I’d treat that as an inference rather than a fact.”
Sometimes the correct answer really is:
We do not have evidence for that.
I know. Deeply unsatisfying. Still an answer.
Then check the thing the source cannot tell you
Even a perfectly sourced fact may be missing the context that matters now.
The AI might know what the annual report said.
It does not automatically know what happened in Tuesday’s meeting, which direction your manager rejected yesterday, which customer concern changed the brief, or whether the old strategy is still the current one.
That part remains yours.
When something feels off
Start with the consequential claim and trace it to the actual source—not another AI summary of the source. Ask what the source really establishes, what the model inferred, and what current context neither the source nor the model could know.
If you still cannot support the claim, leave the uncertainty visible. A beautifully punctuated sentence does not get to promote itself to fact.
The Office Auntie version
Use AI. It is useful. But polish is not provenance, and confidence is not evidence.
The part that stays human is knowing which claims need proof, which sentences are interpretation, and where the available evidence simply stops.
Boundary worth knowing
Low-stakes brainstorming does not need the same verification burden as a claim you will publish, repeat, recommend from, or act on. The more consequential the use, the stronger the verification should be.
A NOTE ON THE EVIDENCE
This is a practical verification aid informed by source-audit work and Office Auntie’s AI/evidence material. It is not a validated assessment, and it does not imply that AI output is generally unreliable.
WHERE TO GO NEXT
Same hidden problem: Make a Recommendation Before You Have Perfect Information
Same judgment move: You Found a Problem. Now What?