Keeping an AI from inventing facts in a summary

A language model is fluent by construction, not truthful. Faced with a gap, it fills it with what sounds plausible — a round number, a familiar name, a reasonable conclusion. In a summary of someone else’s content, an invented fact is not a bug: it is misinformation published with your name on it.

The principle that fixes this is easy to state and hard to hold under pressure: the model is not a source of fact. It receives the article’s text and may only reformulate what is there. Any number, name, date or claim not present in the input is treated as an error — and the item is discarded, not corrected.

Strict anchoring to the source

The first guardrail is in the prompt: provide the source text, forbid outside knowledge, and tell the model to report only what the source says. If the text is vague, the summary should be vague too. And run at temperature 0 — creativity is exactly what you do not want here.

But a prompt is a request, not a guarantee. The model can comply 99 times and invent on the hundredth. So the second guardrail does not trust it.

Constrain what the AI must not be able to do

The cheapest and most effective check is deterministic, outside the model: every number with two or more digits that appears in the summary must exist in the source text. No judgment, no second AI — just a string comparison that catches the most common hallucination.

// Every 2+ digit number in the summary must exist in the source.
function numbersAreGrounded(summary: string, source: string): boolean {
  const nums = summary.match(/\d[\d.,]{1,}/g) ?? []
  const haystack = source.replace(/[.,\s]/g, '')
  return nums.every((raw) => {
    const n = raw.replace(/[.,]/g, '')
    return n.length < 2 || haystack.includes(n)
  })
}

It is not elegant, and that is the point: the safety net that matters is the one that does not depend on the model being in a good mood. Alongside it, the process guardrails hold:

• Discard when the source text is insufficient — a teaser does not make an honest summary.
• A source URL is mandatory: with no verifiable origin, the item does not exist.
• The item is born as a draft and only ships after human review.

That is the design: the model drafts fast, the deterministic checks fence off what it must not do, and a person approves. AI as a fast intern, not an oracle — and that is the difference between speeding up the work and outsourcing your credibility.