Quick answer
Business chatbots give wrong answers mainly because the right information is missing or outdated, retrieval returns the wrong passages, instructions allow guessing, or the question is outside scope. Fix them by building a test set of real questions, diagnosing each failure by cause, correcting content and retrieval first, requiring citations and making the bot decline or hand off when unsure.
Key takeaways
- Most wrong answers trace to content or retrieval, not the model.
- A labelled test set turns complaints into measurable failures.
- Bots should decline and hand off rather than guess.
- Re-test after every content, prompt or model change.
Who this guide is for
Teams running a website or internal chatbot that gives inaccurate, outdated or invented answers.
Collect real failures
Gather conversations where the answer was wrong, outdated or invented, along with what the correct answer should have been. Ask support staff and sales for examples customers raised. Add common questions that worked, so you can detect regressions. This becomes your evaluation set. Without concrete failing examples, teams tend to change models or prompts based on impressions, which rarely fixes the root cause.
Check whether the answer exists in the content
For each failure, find whether the correct answer is in the bot’s sources. Often it is not: pricing changed, a policy was updated in a PDF the bot never indexed, or the answer only exists in staff knowledge. In those cases no model change will help. Update the content, assign an owner and set a review schedule for facts that change, such as prices, timings and availability.
Check what retrieval returned
If the answer exists, inspect which passages the system retrieved for that question. Common problems include chunks split mid-table, outdated duplicates outranking current pages, keyword mismatch between customer language and document terms, and too few or too many passages passed to the model. Improvements may include better chunking, metadata filters, hybrid keyword and vector search, or removing stale documents.
Check instructions and refusal behaviour
Some bots are instructed to always be helpful, so they fill gaps with plausible guesses. Instructions should require answers to be based on retrieved content, ask for clarification when a question is ambiguous and decline when sources are silent. Test edge cases such as questions about competitors, prices not listed and requests for commitments. A good bot says what it cannot answer and offers a route to a person.
Watch for out-of-scope and adversarial questions
Users ask about topics the bot was never meant to handle, and some try to make it say embarrassing or harmful things. Define scope explicitly and test with out-of-scope and manipulative prompts. Never let the bot make commitments such as discounts or refunds on its own. Log and review a sample of conversations regularly to find new failure types.
Show sources and handoff
Displaying the source page or document beside an answer lets users verify and lets your team spot outdated content quickly. A visible option to contact a person reduces frustration when the bot cannot help. Record which questions lead to handoff; they reveal content gaps and possible new FAQ pages for your website.
Re-evaluate after every change
Run the evaluation set before and after changes to content, retrieval settings, prompts or model versions. Track accuracy, refusal correctness and citation correctness separately. Accept that some error rate remains and design the experience around it. A structured review usually covers failure analysis, content fixes, retrieval tuning, instruction updates and a repeatable evaluation process.
Scope checklist
- Do you have examples of wrong answers with correct answers?
- Is the correct information present and current in the sources?
- What passages does retrieval return for failing questions?
- Is the bot instructed to decline when sources are silent?
- Is there an evaluation set run after every change?
Frequently asked questions
Why does my chatbot make things up?
This often happens when the retrieved content does not contain the answer and the bot is allowed to guess. Requiring grounded answers, showing citations and allowing refusals reduces invented responses.
Will switching to a better AI model fix wrong answers?
Sometimes, but most errors come from missing content, poor retrieval or weak instructions. Diagnose failures with a test set before changing models.
How do I test a chatbot’s accuracy?
Build a set of real questions with approved correct answers and source references, run the bot against it and score correctness, citation accuracy and appropriate refusals. Repeat after every change.
Prepared with AI assistance as an engineering decision guide. Examples are illustrative unless explicitly labelled as an internal implementation. No independent expert review or customer outcome is claimed.
Planning an AI assistant or document workflow?
Share the use case, information sources, and review requirements. Scope, access, dependencies and commercial terms are agreed before delivery. Do not submit passwords, API keys or confidential records.