How Do You Keep an AI Assistant From Promising Something You Do Not Sell?
An assistant that invents a service is not hallucinating so much as filling a gap you left. Scope, refusals, published sources and a real escalation path are editorial work before they are technical work.
On This Page
A prospect asks your website assistant whether you offer a maintenance retainer for mobile apps. You do not. The assistant, being helpful, says yes, describes a plausible engagement and offers to arrange a call. Nobody notices until the call happens and somebody has to explain that the service discussed at length does not exist.
This usually gets filed under hallucination, which is accurate and useless. The model did what it was built to do: produce the most plausible continuation. Nothing you gave it said the answer was no. An assistant that invents a service is filling a gap you left, and the gap is almost always in the material rather than in the model.
So the useful question is not how to make it more accurate. It is what you have actually told it, and what you left it to guess. AI assistant guardrails are an editorial system with a technical implementation, in that order, and four parts do most of the work: a written scope, a published-source constraint, a designed refusal, and an escalation that a human genuinely answers.
The order matters. Teams that start with the technical parts spend months tuning prompts against a moving target, because nobody has agreed what the assistant is allowed to say.
What overpromising actually looks like
It is rarely a wild fabrication. Wild fabrications get caught in testing and everybody laughs. The dangerous version is small, plausible and adjacent to something true.
- Inventing a service. You do three of the four obvious things in a category, so the assistant completes the set. This is the most common failure and the most expensive one.
- Inventing terms. A discount, a trial length, a notice period or a turnaround that sounds like the sort of thing a company would offer. Nobody published it, so nobody can contradict it either.
- Inventing coverage. Saying yes to a region, a language, an industry or an integration, because none of your material anywhere says no.
- Inventing certainty. Answering a question that has a real answer, but one that depends on facts the assistant does not hold. The tone is confident and the content is a guess.
- Inventing agreement. Accepting the framing inside the visitor’s question. "So you can have this live by the end of the month?" is a trap, and a helpful assistant walks straight into it.
Only the fourth of those is a retrieval failure. The rest are scope failures, which is why a chatbot can give wrong answers while every page on the site is correct. Correct pages that never say no leave the no to be improvised.
Scope is a document, not a prompt
Before anybody writes a system prompt, somebody has to write down what the assistant is for. Not in the abstract. In lists, with names, reviewed by the people who will be embarrassed if it turns out to be wrong.
The three lists
Three lists, and the second one is the one nobody writes.
- What it answers. The topics, in your own vocabulary, where the assistant is expected to be useful without a person involved. Be specific enough that a stranger could sort a question into or out of this list.
- What it refuses. The topics where a wrong answer causes real damage: pricing, timelines, capacity, legal terms, anything contractual, anything about a named client, anything regulated in your sector.
- What it hands over. The topics where the correct behaviour is neither answering nor refusing, but routing to a person with the conversation attached.
The refusal list is the one that gets skipped, because writing down what you will not do feels like an admission of weakness. It is the opposite. We have argued that publishing your non-goals is among the cheapest pieces of product hygiene available, and an assistant makes it urgent: an undeclared non-goal becomes an improvised yes at three in the morning.
Who owns the list
Whoever owns the words on the website. Not the engineer who wired the integration, and not the vendor. The list is a statement about what the organisation sells and promises, so it belongs with the people who already approve that language, and it needs a review date the way any published page does.
Where a formal position exists, state it publicly. Our own responsible AI page exists for that reason: an assistant’s limits are easier to hold internally when they are also written down where anyone can read them.
The published-source rule
The single most effective constraint we know is this: the assistant may only assert what is published somewhere a visitor could read. If a claim is not on a page, in a document, or in an approved answer, the assistant does not make it. Not a softened version, not a hedged version. It says it does not know, and offers the route to somebody who does.
This has a pleasant side effect. It converts an AI governance problem into a content problem, and content problems already have owners, review processes and deadlines. Every gap the assistant hits becomes a page somebody should have written, which is a far better backlog than a list of prompt tweaks nobody can prioritise.
Getting there is mostly retrieval work. The assistant needs your material in a form it can genuinely use, chunked and labelled so the right passage surfaces for the right question. That is the subject of grounding an assistant in your own content. Our parent company has covered the same failure modes in why a chatbot gives wrong answers.
Some questions deserve a written canonical answer rather than a retrieved one. Pricing posture, working arrangements, the things you will not take on: write the paragraph, publish it, and let the assistant quote it. A published answer is both a better source and a better reply than anything assembled from three half-relevant pages.
Log every question the assistant could not answer from published material, with the question exactly as the visitor typed it. Review it weekly. Within a month that log will tell you more about what your site is missing than any content audit will, because it is written by the people you are trying to reach rather than by the people who made the site.
Designing a refusal somebody can live with
A refusal is a piece of copy, and most refusals are written by an engineer at the end of a sprint. It shows. "I cannot help with that" is technically a refusal and functionally an exit sign.
What a good refusal contains
Three parts, in this order: the boundary, the reason if the reason is short, and the route onward. The route is what turns a dead end into a handoff, and it is the part most often missing.
So instead of "I do not have that information", something closer to: we do not publish pricing for that work because it depends on scope, and I can pass your question to the team with what you have described so far. The visitor learns that the boundary is deliberate rather than a fault, which quietly changes how they read everything else the assistant says.
Refusals also have to survive pressure. A visitor who rephrases three times is not being difficult; they are doing what people do when they want an answer. If the third phrasing produces a yes where the first two produced a no, the guardrail is decoration. Test exactly that, deliberately, before launch, and write the rephrasings yourself rather than waiting for a customer to find them.
And say plainly what the visitor is talking to. Somebody who knows they are speaking to software reads a refusal as a policy. Somebody who thinks they are speaking to staff reads the same words as a brush-off, and behaves accordingly.
Escalation is the guardrail that carries the rest
Every other guardrail creates dead ends. Escalation is what stops a dead end from being a lost enquiry, and it is reliably the part specified last and built worst.
A working escalation has three properties. It is available from inside the conversation rather than as a link to a contact page. It carries the transcript, so the visitor does not have to repeat themselves to a person who sounds like they were not listening. And it goes to somebody who has agreed to answer within a stated period, which is a staffing commitment rather than a feature.
That last property is where these projects quietly fail. An escalation into an unwatched inbox is worse than having no assistant at all, because the visitor believes they have been heard. Our parent company has written about the moment when an answer is not enough, and the handoff is exactly that moment arriving.
Practically, the handoff belongs in the same surface as the conversation. A widget such as Flidu puts website-informed assistance and the contact, information and conversion actions in one place, which matters less as a convenience than as a control: when the route onward is one tap away, the assistant can afford to refuse far more often. Wiring that into the systems behind it is ordinary automation and integration work.
Tracing a bad answer to its cause
When an answer goes wrong, the instinct is to change the prompt. Usually the prompt is not the problem. This is the triage we use, and it is worth pinning somewhere the whole team can see it.
| What the assistant did | Most likely cause | Where the fix belongs |
|---|---|---|
| Invented a service you do not offer | No refusal list, no published non-goals | Editorial: write and publish the boundary |
| Quoted a price or a timeline | Scope allowed it, or an old page still says it | Editorial: remove or restate the source |
| Contradicted one of your pages | Two pages disagree and both were retrieved | Editorial: reconcile the pages |
| Answered from the wrong page | Retrieval returning plausible near-misses | Technical: chunking, labelling, ranking |
| Refused something it should answer | Refusal list drawn too widely | Editorial: move the topic to the answer list |
| Went quiet or looped | No escalation path out of the dead end | Design: build the handoff properly |
Count the rows. Most of those fixes are things a content owner does in an afternoon, which is why we treat an assistant deployment as a content project with an engineering component rather than the reverse. When the cause genuinely is retrieval, it becomes AI development and integration work, and that is the smaller half of the job.
Testing it before anybody else does
Write the test set before the assistant is built. It takes an afternoon and it is the only evidence you will ever have that the guardrails hold.
- Collect the real questions. From sales email, support tickets and the site search log. Real phrasing, including the impolite ones and the badly typed ones.
- Add the questions you fear. The ones about price, capacity, guarantees, competitors and services you do not sell. This is the part of the set that actually matters.
- Write the expected behaviour, not the expected words. Answer, refuse or escalate. Judging wording produces an endless argument; judging behaviour produces a pass or a fail.
- Run the set on every change. Prompt edits, new pages, provider upgrades: any of them can move behaviour, and an upgrade can quietly widen what the assistant is willing to claim.
- Read a sample of real transcripts weekly. Not dashboards. Actual conversations. Reviewing AI-generated work before a client sees it is the same discipline applied to a different output.
A hosted model changes underneath you. The same prompt and the same content can produce a more confident answer next month than it did this month. If nothing in your process re-runs the questions you fear after a provider update, your guardrails have an expiry date that nobody has written down.
Why the fix is almost always editorial
The technical instinct is to constrain the model harder: stricter instructions, lower temperature, a classifier in front, a second model checking the first. Some of that helps. None of it decides what the answer should have been.
A model cannot infer a boundary that has never been written down anywhere. If no page states which services you do not offer, no amount of tuning produces a reliable no, because there is nothing for it to be reliable about. This is the same argument as where AI earns its place and where a rule is enough: a rule beats a model wherever the answer is fixed, and "do we sell this" is a fixed answer.
There is a concession here and it is a real one. Editorial guardrails do not survive a determined adversary. Somebody who sets out to make your assistant say something embarrassing will eventually manage it, and no scope document prevents that. The right response is to size the damage rather than to promise it cannot happen: keep the assistant away from anything contractual, log everything, and make sure a person reads the log.
The everyday failure is not the adversary anyway. It is an ordinary visitor asking an ordinary question that nobody thought to answer in public, at a time when nobody is watching. That one is preventable, and it is prevented with words.
An assistant cannot refuse on your behalf until somebody has written down what the refusal is.
Where we would start
Take the questions your sales team is asked most often and mark each one: answer, refuse, escalate. That single sheet is the specification, and producing it will start an argument internally that is much cheaper to have now than in a transcript a customer has already read.
Then publish the refusals. Every boundary the assistant will state should exist on a page first, in the same words. Two benefits follow: the assistant has a source it can quote, and the visitor who never opens the widget gets exactly the same answer as the one who does.
Only after that is it worth building anything. If the assistant is going to sit across several internal systems rather than only the website, that is an AI-enabled operations question, and the scope document gets longer rather than shorter.
The condition that reverses all of this: where a wrong answer costs nothing and a slow answer costs a customer, loosen the scope and let the assistant talk. Guardrails should be proportionate to consequence. What they should never be proportionate to is how impressive the demo was.
Common questions.
How do you stop an AI chatbot from making things up about your business?
Constrain it to published material and give it an explicit refusal list. If a claim is not on a page, in a document or in an approved answer, the assistant should say it does not know and offer a route to a person. Most invented answers are not model faults; they happen because nothing in the source material ever stated the boundary, so the model completed a plausible pattern instead.
What should an AI assistant refuse to answer?
Anything where a wrong answer creates an obligation or real damage. That normally means pricing, delivery timelines, capacity, contractual and legal terms, details about named clients, and anything regulated in your sector. The safest test is to ask whether a person would need approval before saying it out loud. If they would, the assistant should refuse and hand the question to somebody who can answer it.
Should a website assistant give prices?
Only prices you already publish, quoted exactly as published. If your pricing depends on scope, the assistant should say so, explain briefly what drives the range, and offer to pass the enquiry on. An invented figure sets an expectation that a salesperson then has to argue down, which costs more goodwill than the delay of an honest answer would ever have cost.
What makes a good chatbot fallback message?
A good fallback states the boundary, gives a short reason, and offers a route onward. "I cannot help with that" fails because it only does the first part. Something like "we do not publish that because it depends on scope, but I can pass your question to the team with what you have told me" reads as deliberate policy rather than as a broken feature.
Do you need to tell visitors they are talking to an AI?
Yes, and it improves the experience rather than damaging it. Someone who knows they are talking to software reads a refusal as a policy and rephrases sensibly. Someone who believes they are talking to staff reads the same words as a brush-off and often leaves. Disclosure also sets a fair expectation about response quality, and it is increasingly expected by regulators and by ordinary users.
How often should you review chatbot transcripts?
Read a sample every week, and log every question the assistant could not answer from published material. Weekly is frequent enough to catch a new failure before it becomes a pattern, and small enough that one person can do it properly. The unanswered-question log doubles as a content backlog, because each entry is a page somebody should probably have written already.
Facing this in your
own business?
Tell us where you’re headed — we’ll map the shortest honest route.