Building a customer-support AI that does not hallucinate

Your organisation owns every word the chatbot says

In 2024, Air Canada's support chatbot told a grieving passenger he could book a full-fare flight and claim a bereavement discount retroactively. That policy did not exist. When he tried to claim it, Air Canada refused. The tribunal ruled against the airline. A fabrication by the bot became a liability for the business.

That case is the frame for every technical decision that follows. When your AI agent tells a customer something, your organisation owns that statement, contractually and reputationally. Air Canada was not a model accuracy problem. It was a governance failure: no mechanism verified the policy existed, no boundary prevented invention, no detection layer caught the response before it reached the customer. Deploy a support AI without those three layers and you are running the same configuration.

The scale of the underlying problem makes this urgent. A 2024 Stanford HAI study put ungrounded LLM hallucination rates at 15 to 30% in customer service responses, depending on query complexity. SQ Magazine's 2026 figure for live interactions sits in the same band, 15 to 27%. In 2024, 39% of AI-powered customer service bots were pulled back or significantly reworked because of hallucination-related errors (Testlio, 2025). This is the default behaviour of an ungrounded system, not an edge case you patch later.

Why hallucinations cannot be engineered away

Before you buy any tool, understand what you are actually solving. Xu et al. (2024) formalised the hallucination problem and proved that eliminating it in large language models is mathematically impossible. Not difficult. Not compute-limited. Impossible, given that these systems generate text by predicting probable token sequences from statistical patterns. A model that guesses the next token will, by necessity, sometimes produce fluent, plausible, entirely unfounded text.

The practical consequence is straightforward. You cannot engineer hallucinations to zero. You can only contain them: reduce their frequency, catch them before they reach the customer, and make sure that when one slips through, you know about it. Every architectural decision in this article follows from that assumption.

RAG is necessary but not sufficient

The industry's default answer is retrieval-augmented generation: give the model access to your actual documents so it generates answers from real sources rather than training data. RAG helps. It is also the most misunderstood part of the stack, because teams treat it as a finished solution when it is a foundation. For a deeper walkthrough of the pattern itself, see our companion piece RAG explained: grounding AI on your own data.

What RAG actually fixes

RAG addresses one failure mode: the model lacking access to correct information. It runs two pipelines. Indexing takes your raw documents, breaks them into meaningful chunks, converts those chunks to vectors, and stores them. Retrieval takes an incoming query, finds the most relevant chunks, and passes them to the model as context. Separating those pipelines is what makes the system updatable without retraining.

Tooling here is mature. LlamaIndex ships over a hundred native connectors for Confluence, Notion, SharePoint, Google Drive and S3, and tracks document hashes so only changed files are re-indexed. LangChain covers similar ground with more composability. Either is a reasonable starting point.

The four gaps RAG leaves open

RAG will still let you down in four specific ways.

1. Confidence without coverage. When retrieval finds no relevant documents, many implementations still generate a response. The model does not know retrieval failed; it sees a thin context and improvises.
2. Stale content. A grounded model will still hallucinate if the source article is out of date. Zendesk's 2025 KB health report puts 30% of enterprise help centres over 12 months old.
3. Contradictory documents. If a 2023 PDF says "no refunds" and a 2025 policy update says "refunds allowed within 14 days", the model may synthesise both into a wrong middle-ground answer rather than flag the conflict.
4. Dynamic data. Prices, stock levels, order status, seat availability. By the time you have embedded and indexed that data, it is stale.

When to use an API call instead of a vector search

For anything that changes hourly or faster, vector search is the wrong tool. A support agent quoting yesterday's price is worse than one that quotes no price at all. The retrieval layer should call a live pricing or inventory API and pass the current value into the prompt as a structured fact. The rule of thumb: use vector search for policies, procedures and knowledge; use API calls for anything with a canonical live source.

The knowledge base is the product

Most teams underestimate this. The quality of your AI's output is a direct function of the quality of your knowledge base. The model is almost irrelevant if the source is stale or contradictory. If you are choosing between a week on prompt tuning and a week on KB hygiene, spend it on hygiene.

Versioned document IDs and source anchoring

Every document in the retrieval layer should carry a stable, versioned ID: something like KB-HOL-2024-RETURNS or KB-2024-Q3-v2. The prompt should require the model to cite that ID for every factual claim, and refuse to answer if it cannot cite one.

A concrete case: a retail chatbot in December 2024 told customers the holiday return window extended to 15 January when the real deadline was 24 December. The team replaced vague references like "holiday policy" with specific document IDs, added a "no speculation" rule, and required source IDs in every response. The hallucination rate fell from 23% to 2.6% over 11 days. QA cycle times dropped from 22 hours to 90 minutes, because every error was now traceable to a specific source.

Your prompt scaffolding should look something like this:

You answer only from the DOCUMENTS below.For every factual claim, include the source ID in square brackets, e.g. [KB-HOL-2024-RETURNS].If the answer is not in the documents, respond exactly:"I don't have that information. Let me hand you to a colleague."Do not speculate. Do not synthesise across documents that disagree.

Contradiction detection before ingestion

Build contradiction detection into the ingestion pipeline, not the retrieval layer. When a new document arrives, embed it, retrieve the nearest existing chunks, and flag any semantic conflict to a human editor before it reaches production. This is not perfect, but it is the difference between "the model synthesised a wrong answer from two conflicting sources" and "the editor resolved the conflict last Tuesday". The former is a customer-facing error. The latter is an internal task.

Confidence thresholds and escalation guardrails

Missing escalation guardrails are the single most common reason hallucinations reach customers. When the model is unsure, it should hand off. Without a confidence threshold it will guess, and guesses delivered in the calm cadence of a support agent are how the Air Canada configuration keeps repeating itself across industries.

A workable threshold has three inputs: the retrieval score (how well did the top chunks match the query), the model's self-reported confidence (treat this as advisory, not authoritative), and a query classifier that flags high-stakes topics such as refunds, cancellations, medical, legal or account changes. If any of those trip below threshold, the system escalates.

Refusal is a feature, not a failure. A bot that says "I don't have that information, let me connect you with someone" is worth ten bots that answer everything with 20% wrong. Customers will forgive "I don't know". They do not forgive a fabricated policy that costs them a flight.

Observability: what to measure and how often

Guardrails are not one-time configurations. You need continuous observability, and the cadence matters: weekly, not monthly. Customer queries drift, knowledge bases update, and hallucination rates can move faster than a quarterly review will catch.

Track at least these five metrics.

• Hallucination rate. Sampled human review of a fixed weekly percentage of conversations. Do not skip the human sample; automated hallucination detection is itself a hallucinating system.
• Recall@5. For every query, did the correct source document appear in the top five retrieved chunks? A low Recall@5 is almost always the upstream cause when accuracy drops.
• Refusal rate. What percentage of queries hit the escalation path? Too low and your threshold is too permissive. Too high and your knowledge base has gaps.
• Resolution rate on refused queries. When you escalate, does the human actually solve it? If not, the escalation is theatre.
• Time-to-source. When a wrong answer is flagged, how long does it take to identify which document caused it? Under 15 minutes is the target; without versioned IDs you will not get there.

Guardrail infrastructure typically adds 10 to 20% of overall AI project spend. That is a floor, not a ceiling, and it is not optional if you are shipping to customers. Tools worth evaluating: NVIDIA NeMo Guardrails for configurable safety workflows, Azure AI Content Safety for enterprise-grade moderation, and Anthropic's constitutional approach where principles replace ad-hoc rules. This is the layer we spend the most time on in our AI development work, because it is what separates a demo from a production system.

Scope narrow, then expand

The last mistake is scope. Teams launch a bot that handles everything, discover accuracy collapses on the long tail, and pull the whole thing back. The pattern that works is the opposite: pick one high-volume, low-risk query type, get accuracy above a defined threshold, then expand.

Shipping status is the classic starting point. High volume, low ambiguity, a canonical data source in the carrier API, low stakes if the answer is delayed by ten seconds. Glossier's early 2025 deployment reportedly hit 91% accuracy on shipping enquiries by keeping the AI focused on that single task rather than the whole knowledge base. Whether you take that figure as gospel or as illustrative, the principle holds: a narrow bot with high accuracy is worth more than a broad bot that hallucinates on edge cases.

Start with 5 to 10% of live traffic. Monitor weekly. Widen the scope only when Recall@5, hallucination rate and refusal rate all sit inside your targets for two consecutive weeks. This is the same build sequence we recommend for adding AI features to your product without the hype, and it is the pattern behind most of the AI agents for business that actually work in 2026.

If you are planning a customer-support AI and want a second pair of eyes on the architecture before you commit, see how we build AI-powered products and get in touch. The cheapest time to fix a governance gap is before the tribunal ruling.