Prompts

How to fact-check a chatbot's answer in five minutes

Three simple habits that catch most invented facts and links in a chatbot's answer, in about five minutes.

For you: A chatbot's mistake stays out of your essay, post or client email, so you never have to correct it after publishing.

How to fact-check a chatbot's answer in five minutes
How to fact-check a chatbot's answer in five minutes

promptBelow is a chatbot's answer to the question: [question]. List every fact in it: numbers, dates, names, titles, links. For each one, say where it comes from, how confident you are (high, medium, low) and how I can check …

A chatbot sounds confident even when it is wrong. It does not know that it has invented a date or a link: a language model picks the most likely next words, and a made-up fact comes out just as smoothly as a real one. So check what the answer rests on — numbers, names and links — not how it sounds.

The good news is that five minutes and three simple habits catch most mistakes. Below is how to use them, a ready prompt for checking, and what to do when an answer fails.

Why a wrong answer can sound right

A model learned from huge amounts of text and writes what usually comes next. It has no built-in sense of "I don't know". When a fact is missing, it fills the gap with something that fits the pattern: a plausible year, a real-looking web address, a quote in the right style. These errors are often called hallucinations.

Mistakes cluster in predictable places:

  • exact numbers: percentages, prices, populations, dates;
  • names and job titles, especially of less famous people;
  • links and the titles of articles, books and laws;
  • quotes attributed to real people;
  • local details — Kazakh cities, agencies, rules. There is less of this in the training data than of global topics.

General explanations ("what is inflation", "how does a vaccine work") are usually more reliable than specific claims. That tells you where to spend your five minutes.

Habit one: ask for the source and open it yourself

Ask the chatbot: Which sources is this answer based on? Give links. Then open every link yourself. Do not trust the link text or the title — look at the page.

There are usually three outcomes:

  1. The link opens and the page says the same thing. Good — the fact has support.
  2. The link opens, but the page is about something else or shows a different number. The answer is in doubt.
  3. The link does not open at all, or the site has no such page. Treat the fact as unconfirmed.

If the link is dead or says something else, the answer is in doubt. Check the date of the page too: a correct figure from five years ago may be out of date today. Some chatbots search the web and show where the text came from; others answer from memory only. In the second case, always check the link — it may have been made up rather than found.

Habit two: ask the same question in different words

Open a new chat so the model does not repeat its last answer, and ask the same thing another way. For example, instead of "When was the law passed?" ask "What changed after this law came into force, and from what date?".

If the facts in the two answers disagree, at least one of them is made up. If they match, that is not proof, but it is a good sign. This takes a minute and catches a lot of invention, because a made-up fact rarely comes out the same way twice.

Habit three: compare the key number with an official website

Pick the one number that matters most in the answer — the one your text, decision or argument depends on. Find it on an official website: the agency in charge of that area, the company itself, the university, the statistics office. If the official figure is different, use it, and treat the rest of the answer with more caution.

You do not need to check every word. One key number checked properly beats ten checked in a hurry.

A prompt for checking

You can make the chatbot help check its own answer. Paste the answer and ask it to list every fact in it:

Below is a chatbot's answer to the question: [question]. List every fact in it: numbers, dates, names, titles, links. For each one, say where it comes from, how confident you are (high, medium, low) and how I can check it myself. If there is no source, say so. Do not add anything of your own.
[answer text]

Replace the brackets with your question and the answer you are checking. The model will not find all of its own mistakes this way, but you get a clear list of what to verify. Start with the low-confidence items and anything without a source. Whatever you cannot confirm, take out of your text.

Variations:

  • For students: add "mark which facts I need to cite in my references".
  • For journalists: add "mark every quote separately and who said it".
  • Short version: List the facts in this answer that I should check before using it.

What this means for you, and what to do today

In Kazakhstan, people use chatbots to write coursework, posts, business plans and letters in Kazakh and Russian. Local facts — agency names, rules, city figures — are exactly where models are weakest. So it is worth building the habit now:

  1. Before you copy an answer into anything public, mark every number, name and link.
  2. Open each link. Delete the ones that do not open.
  3. Ask the main question again in a new chat, in different words.
  4. Check the one key number on an official website.
  5. Save the prompt above in your notes.

Five minutes of checking costs less than a correction after publication — and less than a lower grade or a lost client.

Ad Your ad hereA banner on the homepage, in stories and sections. Formats and prices on our Advertise page.

Comments

No comments yet. Be the first.