Checking: the fact-checker agent
Lesson 7 of 22 in the course “AI Agents for Life”. Free, with animations and a quiz.
Confident mistakes cost money
AI makes mistakes in a calm, confident tone, and it costs even giants dearly. In February 2023 Google’s Bard got a space telescope fact wrong right in a promo video, and Alphabet lost about $100 billion in value in a day. In 2023, American lawyers cited court cases that ChatGPT had made up and were fined $5,000. In October 2025 Deloitte agreed to refund part of A$440,000 for a report for the Australian government: it contained a made-up quote from a court ruling and references to studies that don’t exist.
Why AI lies with confidence
A model doesn’t look up the truth in its head. It continues the text so it sounds plausible. Ask for three studies with links, and a plausible answer is three surnames, years and neat links. Whether they exist, the model won’t check by itself unless it has search or you asked. That’s called a hallucination. By default it has no inner “I’m not sure” signal: a made-up fact and a real one sound the same. So check the source, not the tone.
Red flags
Checking everything takes ages, so look for red flags first. No link, or it won’t open, or it goes to a homepage instead of an article. A very precise number like “73.4% of students” with no source. A quote that sounds too smooth and convenient. A “fresh” fact citing something three years old. No single flag proves a lie, but each one says: check here first. The 2023 lawyers had a flag right in front of them: cases that no legal search could find.
Check it in 3 steps
Found a flag? Checking takes a couple of minutes. Step one: open the link and make sure the page exists and it’s the right article. Two: find the number or quote using find-on-page, Ctrl+F on a computer or “Find on page” on a phone. If it isn’t there, that’s your answer. Three: see who wrote it and when. An official site and a recent date, good. Anonymous and five years old, look for another source. That’s how journalists check, and it’s how Deloitte’s team should have checked its report.
A second agent as the critic
You can hand part of the checking to an agent, just a different one. Anthropic’s guide to agents describes this pattern: one agent writes, a second evaluates and sends back notes, round and round until the result is good. Why a different one? The author is sure of its text and can’t see its own mistakes. A critic in a new chat looks with fresh eyes and hunts only for mistakes. Important: the critic can be wrong too. It narrows things down, but the final check of the most important facts is yours.
Try it: the fact-checker agent
Take any text where accuracy matters: your essay, the report from the last lesson, a post you want to repost. Open a new chat with web search and paste the brief below, then the text under it. The agent will list every checkable claim and label it. The key part of the brief is “never”: it fixes nothing by itself and doesn’t fill gaps. If a source isn’t found, it says so. Liza ran her essay through this and found two quotes that weren’t in the sources. She double-checked by hand, rewrote them and handed it in with peace of mind.
Check yourself
- What should he do first?
Correct answer: Open the link and find the number on the page. A precise number without confirmation is a red flag. Checking the link takes a minute and gives a real answer. - Why is it better to check in a new chat?
Correct answer: The author is sure of its text, the critic looks with fresh eyes. That’s the “author and critic” pattern from Anthropic’s guide: the one who evaluates isn’t the one who wrote. - What went wrong?
Correct answer: The agent was allowed to fix things itself and covered up a gap. A fact-checker that edits the text turns “not found” into a new invention. That’s why the brief says “don’t fix anything and don’t fill gaps.” - What’s the move?
Correct answer: Find the quote in the original source before reposting. A too-perfect quote with no link is a red flag. If it’s not in any interview or book, it most likely was never said.
← Deep research: the researcher agent Connectors: the agent sees your day →