Blog

Your AI Just Made Something Up. Here's How to Catch It.

by leonard j mckinzy

Sooner or later, an AI is going to tell you something that is simply not true. It will do it calmly, in a clean sentence, with no hesitation at all.

People in the field call this a hallucination. That word makes it sound like a rare malfunction. It is not. It is a normal consequence of how these tools work, and once you understand why, catching it becomes easy.

Why it happens

An AI is not looking things up. It is predicting what should come next, one word at a time, based on patterns from an enormous amount of writing it learned from.

Most of the time that prediction is right, because the pattern matched reality. Sometimes it is wrong.

Here is the uncomfortable part: the AI has no way to tell the difference. A confident wrong guess is produced by exactly the same process as a confident right one. So they come out sounding identical.

It is not lying to you. Lying requires knowing the truth and choosing to say something else. This is closer to a very well-read person answering from memory, at speed, without ever saying "actually, I'm not sure about that one."

It is not being dishonest. It is guessing, and it does not know which of its guesses are wrong.

What it tends to make up

Specific numbers. Statistics, percentages, prices, market sizes. Anything that sounds precise deserves suspicion precisely because it sounds precise.

Names and dates. Who said something, when something happened, which company did what.

Quotes. It will happily attribute a plausible-sounding sentence to a real person who never said it.

Sources. It can invent an article, a study, a book, or a web address that does not exist. The fake ones look completely normal.

Rules and law. What a regulation requires, what a contract clause means, what you are allowed to do in your state. This one can cost you real money.

What it rarely gets wrong

This is the half that people miss, and it is why the tool is still worth using.

Rewriting something you gave it. If you pasted the text in, it is working from your words, not its memory.

Summarizing a document you provided. Same reason.

Drafting. There is no fact to get wrong in "write me a friendly reminder about an overdue invoice."

Explaining a common, settled idea. How compound interest works. What a security deposit is for. The basics are heavily represented in what it learned and it handles them well.

So the pattern is simple. The further it gets from text you handed it, and the more specific the claim, the harder you should check.

The thirty-second check

You do not need to fact-check everything. You need a habit for the things that matter.

Before you act on something, send it, or repeat it to another person, ask yourself one question: is this a fact, or is this a draft?

If it is a draft, a summary, or a rewrite, you are on safe ground. Edit it and go.

If it is a fact - a number, a name, a date, a rule, a price, a source - open a search engine and confirm it. Thirty seconds. Every time.

A useful trick

Ask it directly:

It will often flag its own weak spots. This is not a guarantee and it is not a substitute for checking. But it is a fast way to find where to look first.

What to do when you catch one

Tell it. "That's wrong - the deadline is in April, not February." It will correct itself and carry on with the right information for the rest of that conversation.

Then verify the correction too. The second answer is not automatically right just because you pushed back.

The thing to take away

This is not a reason to avoid AI. It is a reason to use it the way you would use a fast, eager, brand-new assistant who has read a great deal and has been working for you for three days.

You would not send that person's work out under your name without reading it. You also would not refuse their help.

Trust it with the first draft. Never trust it with the final fact. That one sentence covers almost everything.