Quick answerDefault to qualitative hedging language for customer facing answers, and reserve numeric confidence scores for internal tooling or expert audiences who know how to interpret a probability. Most customers read a number like 87 percent as far more precise and authoritative than the underlying model actually is, which creates false confidence rather than genuine transparency. Well written qualitative language, calibrated consistently, communicates the same underlying uncertainty without inviting a customer to over-trust a number that was never meant to be read that literally.
A number implies more precision than most models can back up
A confidence score looks objective. It is a specific figure, it can be sorted, compared, and it feels like it removes ambiguity. That is exactly the problem for most customer facing use cases: the number implies a level of calibrated precision that the underlying system usually cannot support. An agent that reports 82 percent confidence is not making a claim most customers can correctly interpret, and if the model's confidence estimates are not well calibrated against real world accuracy, an 82 that behaves like a 60 is worse than no number at all, because it actively misleads rather than simply staying vague.
Qualitative hedging, phrases like "this is likely correct but worth double-checking" or "I found a partial answer, here is what I'm less sure about," communicates uncertainty in a way most people already know how to interpret from everyday conversation. It does not pretend to a precision the system does not have, and it naturally invites the follow-up behavior you actually want: the customer treats the answer as provisional rather than final.
This is a broader question than the "I don't know" case
The specific case of an agent admitting it has no answer at all is covered in How Should an AI Agent Say "I Don't Know" Without Losing Customer Trust?. That post is about the binary moment when confidence is low enough that the agent should not attempt an answer. This question is broader: across every answer the agent does give, including the ones it is fairly confident about, how should it communicate the gradient of certainty behind the answer, not just flag the rare case where certainty collapses to zero.
It is also a narrower slice of the larger UI design question addressed in Designing a Trustworthy AI Agent UI: Sources, Confidence, and Labeling, which covers sourcing and labeling as well as confidence. This post focuses specifically on the format choice: number versus words, and when each is appropriate.
When a numeric score actually makes sense
Numeric confidence is not always the wrong choice. It tends to work when:
The audience is trained to interpret it correctly. Internal reviewers, analysts, or expert users who understand what a probability estimate does and does not mean can use a numeric score productively, for example to triage which answers need a closer look first.
The score drives an internal decision, not a customer facing statement. A confidence threshold that routes a low-confidence answer to human review before it ever reaches the customer is a legitimate and valuable use of a numeric score, precisely because the number never has to be interpreted correctly by someone outside the system.
The score is consistently calibrated and you can prove it. If you have validated that your confidence scores track real world accuracy closely, showing the number to a sophisticated audience becomes defensible. Without that validation, a numeric score is a claim you cannot actually back up if challenged.
From the team
We build production AI systems for startups.
LLM pipelines, RAG, and agent workflows that hold up under real traffic — not just in the demo.
Designing qualitative hedging that actually communicates something
Vague hedging that appears on every answer regardless of actual certainty is just as bad as an uncalibrated number, since customers quickly learn to ignore boilerplate caveats. Effective qualitative hedging should:
- Vary meaningfully with actual certainty, so a customer can tell the difference between "I'm confident, but here's a nuance" and "I found something, but you should verify this." If every answer gets the same hedge, the hedge has no information content.
- Point at the specific source of uncertainty, rather than a generic disclaimer. "This depends on your contract's specific terms, which I don't have visibility into" tells the customer something actionable; "results may vary" does not.
- Suggest a next step when the confidence is genuinely low, connecting to a human reviewer or a more authoritative source, rather than leaving the customer to figure out what to do with an uncertain answer on their own.
Testing whether your hedging language is actually working
Treat hedging calibration the same way you would treat model accuracy: measure it. Sample a set of answers across confidence levels, have a human reviewer independently assess whether the answer was correct, and check whether your hedging language tracked that reality. This is the same discipline behind building a strong evaluation set, as covered in Building a Golden Evaluation Dataset for Your AI Agent, applied specifically to the calibration of your uncertainty communication rather than just answer correctness.
FAQ
Can we show both a number and qualitative language together? It is possible, but the number should generally be de-emphasized or reserved for a details view rather than presented as the headline of the answer, since most customers will anchor on the number and skip the qualitative context around it.
Does hedging make the AI agent look less capable? Well calibrated hedging tends to build trust over time rather than undermine it, because customers learn that a confident-sounding answer from your agent really is more reliable than a hedged one. An agent that hedges on nothing eventually gets caught being wrong with full confidence, which damages trust far more than honest uncertainty does.
Should confidence communication differ between a first-time customer and a repeat one? It can be reasonable to adjust tone, but the underlying calibration should not change based on who is asking. The goal is that the same actual level of certainty always produces roughly the same signal to the customer, regardless of their history with the product.