The rushed launch that gets expensive later
The pattern repeats a lot: a company decides to automate its support, connects an AI assistant to its channels, and for a knowledge base gives it whatever's on hand — an old PDF, some past conversations, maybe nothing more than general business instructions. The assistant starts answering from day one, and during the first conversations it seems to work well. The problem shows up in the second week, when questions arrive that have no clear answer anywhere — and the assistant, instead of saying "I don't have that information," makes up something reasonable.
Why "reasonable" is the problem, not the solution
A made-up answer that sounds reasonable is more dangerous than an obviously bad one, because nobody catches it in time. A customer who gets the wrong delivery date or a warranty condition that doesn't exist has no way of knowing it's wrong — they simply act on that information, and the problem shows up later, once it's already created an expectation the company can't meet.
The root cause: automating before curating
The mistake isn't technological, it's about sequence. Launching an AI assistant before you have a knowledge base that actually covers the range of real customer questions is like opening a storefront and hiring people without training them on the company's policies — they'll do their best with common sense, which doesn't always match what the company actually does.
How to avoid it: curate first, measure after, keep expanding
The order that works is the reverse of rushing: before connecting the assistant to a live channel, build the knowledge base covering at least the most frequent real questions. Then, launch with a verification mechanism that keeps the assistant from answering on topics it doesn't cover — that it escalates to a human instead. And from there, treat every escalation caused by missing information as a pending task: expand the knowledge base with what was missing.
Escalating isn't a launch failure
Many companies have the implicit expectation that a good launch means the assistant answers "everything" from day one, and that creates the wrong pressure — the pressure to prefer a made-up answer over an escalation. It's the opposite: an assistant that escalates with good judgment when it doesn't have enough information is a sign the system is well designed, not that it's failing.
The knowledge base never "finishes"
Not even a well-curated base on launch day stays complete forever — the product changes, policies change, new questions come up. The difference between an implementation that's still working well six months later and one that degrades is having the habit of reviewing which questions couldn't be answered with confidence, and closing those gaps continuously.