Quick answerSupport, engineering, and leadership routinely use the words "hallucination," "bug," and "regression" to mean different things when discussing the same AI agent, and that gap causes real miscommunication long before any formal incident process kicks in: a support lead's casual "the agent hallucinated again" can mean anything from a genuine invented fact to a UI glitch, and leadership hears "hallucination" as a severity signal that may not match what actually happened. Fix this with a short, actively maintained internal glossary that defines these terms precisely, is referenced in day-to-day Slack and standup conversation, not just formal postmortems, and gets revisited whenever a new ambiguous case exposes a gap in the existing definitions.
A precision problem, not a survival problem
Documenting what a future team needs to rebuild an AI agent and handing off knowledge before an engineering team turns over are both about surviving a loss of institutional knowledge when specific people leave. This is a different, everyday problem that exists even with zero turnover: three teams currently employed at the same company, none of whom are going anywhere, using the same vocabulary to mean different things in a conversation happening right now.
It is also a different problem than the hallucination-versus-bug triage process, which is a technical diagnostic runbook for classifying a specific incident once you already know something went wrong. That triage process actually depends on this vocabulary being standardized first. A triage runbook that says "check if this is a hallucination or a bug" only works if everyone running it agrees on what those two words mean before they start.
Where the imprecision actually causes damage
A support agent telling their lead "the bot hallucinated on that ticket" is often using the word loosely to mean "it gave a bad answer," not the specific technical meaning of inventing information that was never in its source data. If that phrase reaches a leadership Slack channel unfiltered, it can trigger a severity response calibrated to genuine hallucination risk when the actual root cause was a UI rendering issue or a stale knowledge base entry, both of which are "bugs" in the precise sense and neither of which needed the same response.
The reverse also happens: a genuine hallucination gets logged as "just a bug" by an engineer using bug as a catch-all term for anything unexpected in the codebase's behavior, which can under-trigger the review process a real hallucination should receive.
"Regression" has its own version of this problem. Support may use it to mean "the agent got worse recently" in a general sense. Engineering typically reserves it for a specific, previously-passing behavior that a specific change broke, verifiable against a prior version. Leadership, hearing "regression" from either group, tends to assume the stricter engineering definition, which is not always what was meant.
Building a glossary that actually gets used
- Keep the list short. A glossary of five to eight precisely defined terms that people actually reference beats a comprehensive twenty-term document nobody opens. Start with hallucination, bug, regression, degradation, and incident, and expand only when a real conversation exposes a genuine gap.
- Write definitions in plain language with a concrete example, not a formal technical specification. The goal is that a support lead and an engineer read the same sentence and picture the same thing, not that the definition would survive a legal review.
- Put it somewhere people already are, pinned in the relevant Slack channel or linked directly from the incident-reporting template, not buried in a wiki page nobody navigates to during a live conversation.
- Assign an owner who updates it, ideally someone who sits in both incident reviews and cross-team standups, since the gaps in a glossary only become visible when an actual ambiguous case surfaces one.
- Revisit it after any incident where terminology confusion measurably slowed the response, treating that confusion itself as a fixable defect, the same way you would treat a process gap that delayed a technical fix.
FAQ
Isn't this what an incident postmortem template already solves? A postmortem template standardizes documentation after a formal incident is already declared. It does not help with the much more frequent informal conversations, a quick Slack message, a standup comment, that happen before anyone has decided whether something is worth a formal incident at all, which is exactly where the imprecise language does the most damage.
Who should own maintaining the glossary? Whoever runs the cross-team incident or escalation review process is usually best positioned, since they see the friction from imprecise terminology most directly and have natural authority to propose changes when a term stops working.
Does this need executive sponsorship to actually get adopted? It helps but is not required. A glossary that gets referenced consistently by whoever runs the incident review process tends to get adopted organically once people notice conversations moving faster; top-down mandates for internal documentation are often ignored regardless of sponsorship level.