Hallucination reduction: how to cut LLM errors in production

TL;DR: Hallucination reduction is not one fix. It's layers. Ground the model in your own data with RAG, tag every source, add a detector, then keep watch after launch. Below: why models make things up, how to catch it, what to fix at each stage, and when to block an answer or just flag it.

A model that makes things up will cost you trust fast. One wrong refund rule. One fake citation. After that, users doubt the rest.

Here's the short version. You can't patch hallucinations out. You stack layers. Clean data. Good retrieval. A detector. Checks at answer time. Then you keep watch after launch. No one layer is enough on its own. A 2025 tutorial in the journal Computers makes the same case with a multi-layered framework for high-stakes apps.

Three moves pay off first:

1. Ground the model with RAG. Feed it your own documents and tag where each one came from.
2. Add a detector. Score each answer for how much it leaned on your documents and how much on its own memory.
3. Check at answer time. Run a second pass before the user sees a risky claim.

New to RAG? Start with what RAG is, then come back.

Why models make things up

A large language model is trained to guess the next word. It is not trained to check facts. So it writes smooth, sure-sounding text even when it has nothing to back it up.

That's not a bug you can patch. NIST calls it confabulation. Its Generative AI Profile says these errors are a natural result of the way generative models are designed. So plan for it. Don't hope it away.

Three things push a model to make things up:

• Memory beats context. The model answers from what it learnt in training, not from what you gave it.
• Bad patterns from training. It picked up links between ideas that look right and are wrong.
• Retrieval gets ignored. You hand it the right passage and it still doesn't use it.

That last one hurts most in RAG. The ReDeEP paper looked inside the model to see why. Some parts lean too hard on stored memory. Other parts are meant to copy from your documents, and they drop the ball. So you can get a wrong answer even when the right passage came back.

Three kinds of wrong

Sort the errors before you try to fix them. Each kind needs its own check.

Size matters too. A made-up name is cheap to catch. You look it up. A wrong claim about how two facts relate is harder. You often need a second model to judge it.

How to catch it

Start here. Detection gives you the most for the least work. You don't retrain a thing. You add a check to the pipeline you have.

Three checks worth building:

1. Context versus memory score. Measure how much the answer used your documents and how much it used the model's own memory.
2. Layer tracking. Watch how the model's guess shifts as it moves through its layers. A late swerve away from the source is a warning sign.
3. Name and number check. Look up people, products, citations and figures in a trusted index. It's cheap and it works on streamed output.

The best recent work on the first check is LUMINA. It scores two signals. How much did the model use the retrieved context? How much did it use what it knew? The paper tested four open-source models. It reports up to +13% AUROC over earlier methods of the same kind on the HalluRAG benchmark.

Track that score over time. It tells you if your retriever is being trusted or ignored.

Tip: give the detector its own owner, apart from the model team. Then it won't quietly break when someone swaps the model.

How to measure it

You can't cut what you don't measure. Two kinds of numbers do the work.

AUROC and AUPRC tell you how well a detector splits good answers from bad ones. They're cheap to run once you have labelled examples. Getting those labels is the hard part.

Rubric grading scales further. A second model marks each answer against the source, using a written rubric. You need no labels. One catch: the grader has blind spots of its own.

Don't trust a generic score. The HALoGEN benchmark ran 10,923 prompts across nine domains. It checked about 150,000 answers from 14 models. Even the best models got a lot wrong. In some domains, up to 86% of the facts they wrote were hallucinated. The rate depends on the domain. So build a test set from your own users' questions.

We go deeper on scoring in our guide to LLM evaluation.

What to fix at each stage

Treat this as a pipeline job, not one feature. Skip a stage and the others carry the load.

The retrieval row is where most teams win or lose. RAG only helps if the right passage comes back and the model can trace a claim to it. How you split your documents matters more than people think. See our post on RAG chunking strategies. Starting from zero? AI-Led has a step-by-step on how to build a RAG application.

On the model row, think hard before you fine-tune. It costs more, and it can bake in new errors. Our fine-tuning vs RAG post covers when each one fits.

Tip: tag every passage with a source ID when you ingest it. Adding it later, across a live index, is far more work.

Fixes that need no retraining

Can't touch the model weights? You still have options at answer time.

Contrastive decoding. Compare two views of the same answer and keep the words the stronger view backs. DoLa does it by contrasting early and late layers inside one model. It needs no retrieval and no fine-tuning. The paper reports a lift of 12 to 17 points on the TruthfulQA test for LLaMA models.

A second-pass check. Write the answer. Then have a checker test each claim against the source. The checker can be the same model with a new prompt, or a separate one. It catches more. It also costs more. Every pass adds delay and spend.

Guardrails on the way out. Block or rewrite an answer that fails a check before it reaches the user. Our LLM guardrails guide maps where each check sits.

How much is enough? Price the mistake. A wrong answer in a legal tool costs more than one in a recipe app. Spend to match.

Set targets and keep watch

Shipping the pipeline once isn't the job. Models change. Indexes go stale. Users ask things your tests never covered.

1. Set a baseline. Run a public benchmark and your own test set. Write the numbers down.
2. Run detectors live. Flags should show up in near real time, not in a batch review days later.
3. Set service level targets. What error rate sets off an alert? What share of flagged answers goes to a person? How fast must they act?
4. Trace every rise. Tie it to a cause. A stale index. A changed source. A model update.

For the human review queue, borrow from people who check facts for a living. This five-step guide to fact-checking AI-assisted reports is a handy model for your sampling and sign-off rules. It helps most with fake citations, which you want caught before a user sees them.

We cover where a person should sit in the flow in human in the loop. AI Orchestrators has a 90-day plan for building human-in-the-loop workflows if you want the team side.

Block it or flag it?

Here's the thing. Zero hallucinations is the wrong goal for most products.

Block hard when the stakes are high. Clinical advice. Legal advice. A sure wrong answer there is worse than no answer.

For most other products, flag it. Show the source. Show how sure the system is. Let it say "I'm not certain". Teams that chase zero often end up with a bot that dodges every question. That's safe, and no use to anyone.

The prompt carries part of this. Too strict and the model refuses everything. Too loose and it runs past what it knows. Digiocial's write-up on system prompt patterns shows how to manage that tension in the prompt itself.

A system that tells you when it's unsure beats one that's quietly wrong.

The build order

Here's the order we'd build it in:

1. Ingest your documents and tag each one with source and date.
2. Build the retrieval index, with checks on recall and freshness.
3. Add detectors. The name check first, then the context versus memory score.
4. Add answer-time checks where your speed budget allows.
5. Wire it all into monitoring, with a written runbook.

Keep the runbook short. What does a reviewer check first? Who owns the index? Who signs off a model change?

Already have a pipeline? Our RAG pipeline build guide shows where these checks slot in.

Where Devwiz fits

There's a lot of engineering between a RAG demo and a system you can trust in production. Most teams underrate it.

We build AI apps and turn programs into AI platforms. The AI goes in from the start, not as a chatbot bolted on at the end.

Devwiz has shipped 200+ apps, including work for NSW Government (Justice and Corrective Services), Briometrix, Vivid and Huskee.

Shipping an LLM feature and not sure it's telling the truth? A discovery phase is the fastest way to map your data and your risk. Worth a chat.

Frequently asked questions

How do you stop an LLM hallucinating?

There's no single fix. Ground the model in your own documents with RAG and tag every source. Add a detector that scores how much each answer leaned on those documents. Then monitor after launch, so you can trace a flagged answer back to a retrieval or data problem.

Can you reduce hallucinations without retraining the model?

Yes. Answer-time fixes don't touch the model weights. Contrastive decoding, a second-pass check against the source, and output guardrails all work this way. The trade is speed and cost. Every extra pass adds delay and spend.

What metrics should you track for hallucination reduction?

AUROC and AUPRC show how well a detector splits good answers from bad ones, once you have labelled examples. Rubric grading by a second model scales further and needs no labels. Hallucination rates swing a lot by domain, so a baseline built from your own users' questions matters more than a public benchmark score.

Should you block hallucinations or just flag them?

It depends on what the answer feeds into. Block hard in high-stakes areas like clinical or legal advice. For most other products, flag it. Show the source and how sure the system is. A system that says when it's unsure is more use than one that refuses everything.

What does RAG do for hallucinations?

RAG grounds the answer in documents you supply, so the model isn't relying on memory alone. It cuts made-up detail when retrieval is good. Its main failure is the model ignoring the passage it was given. Research like ReDeEP and LUMINA is built to detect exactly that.