How to Design an AI Agent That Refuses Requests It Isn't Equipped to Answer

Quick answer

A well-designed refusal boundary is precise, not a blanket disclaimer: the agent identifies specifically why a request falls outside what it's equipped to answer (regulated domain, no verified source, requires professional judgment), says so plainly instead of hedging, and redirects the customer to an appropriate resource (a licensed professional, a human on your team, an authoritative external source) rather than either guessing or issuing a generic "I can't help with that."

Refusal design is different from compliance scoping

Our posts on which compliance frameworks apply to an AI agent and AI agent legal liability for wrong information cover the regulatory and legal backdrop for why certain topics are dangerous ground. This post is about the actual product and prompt design decisions that turn "we know medical advice is risky" into an agent that reliably, gracefully declines it in the moment, without becoming so cautious it refuses ordinary questions too.

Start with an explicit boundary list, not a vibe

"Don't give medical advice" is not an instruction an agent can reliably act on, because it doesn't define the edge. Build an explicit, reviewed list with legal and domain experts: what specific categories of request are out of scope (diagnosing a symptom, interpreting a specific lab result, recommending a specific medication dosage, drafting a binding legal document, guaranteeing a specific investment outcome), and what's clearly in scope even though it's adjacent (general information about a condition, explaining what a lab value typically means in general terms, explaining what a contract clause type generally does). The boundary needs to be specific enough that two different engineers reviewing a transcript would agree on whether it was crossed.

Refuse with a reason, not a wall

The failure mode that erodes trust fastest is a flat, unexplained refusal: "I can't help with that." Customers experience this as evasive even when it's the right call. A better pattern names the actual boundary: "I can't recommend a specific dosage since that needs to account for your full medical history, which is something your doctor or pharmacist can assess directly. I can explain generally what this medication class is typically used for, if that's helpful." This does three things a bare refusal doesn't: it's honest about the limit, it doesn't make the customer feel dismissed, and it still offers whatever value is safely available.

Redirect to something real, not a disclaimer

"Please consult a professional" with nothing else is a legal fig leaf, not a helpful redirect. Where possible, give a concrete next step: a link to your own human escalation path if you have licensed staff or partners, a pointer to an authoritative external resource (a regulatory body, a well-known reputable source) for general information, or, for a B2B context, a note that this specific question should go to the customer's own legal or compliance team since it depends on facts the agent doesn't have.

Test the boundary adversarially, not just directly

Customers rarely phrase a risky request as bluntly as "diagnose my symptoms." They ask indirectly, in a follow-up to an unrelated question, or reframe a refused request in softer language to see if it gets through. Red-team the refusal boundary the same way you'd red-team any other agent behavior, covered in red-teaming an AI agent before launch: have testers specifically try to get the agent to cross the line through rephrasing, hypothetical framing ("asking for a friend"), or multi-turn erosion where each individual message looks benign but the conversation is steering toward a refused topic.

Log every refusal and review the pattern, not just the incident

Track refusal rate and refusal reason as a first-class metric, not just something you check when a customer complains. A refusal rate that's rising in a specific category might mean a genuinely risky topic is trending among your users (worth knowing), or it might mean the boundary is miscalibrated and refusing things it shouldn't (also worth knowing, and a worse customer experience than the alternative). This belongs in the same operational dashboard discussed in production quality metrics for a live AI agent, not treated as a separate, occasional audit.

Don't let over-refusal become its own failure mode

An agent that refuses too broadly, treating any question with the word "legal" or "medical" in it as out of bounds, is functionally useless and pushes customers toward less careful sources instead. The goal is precision: refuse the narrow, genuinely risky category, and stay helpful on everything adjacent to it. This tradeoff should get the same recurring, cross-functional review as the sign-off process described in legal and compliance sign-off before launching an AI agent, since getting the calibration right is a recurring judgment call, not a one-time setting.

FAQ

Should the refusal boundary be identical across all customers, or can it vary by contract or region?
It often needs to vary; a healthcare enterprise customer's contract may impose stricter limits than your default consumer product, and regional regulation (medical advice rules differ by country) can require different boundaries by geography.

How do we know if our refusal boundary is too strict or too loose?
Track both false refusals (declining something actually safe to answer, surfaced through customer complaints or support escalations) and near-misses (the agent almost answered something it shouldn't have, caught in review) as two separate metrics; both matter.

Should refusals always mention that the response comes from an AI?
Yes, transparency about AI involvement is good practice generally and specifically relevant when declining a high-stakes request, since it clarifies why the agent isn't simply answering directly.

Who should own writing and reviewing the boundary list?
A cross-functional group: legal/compliance for the regulatory edge, a domain expert (clinical, legal, financial as relevant) for what's actually safe to answer generally, and product/engineering for how it gets implemented and tested.

Does a refusal boundary need to be reviewed after launch, or is it set once?
Review it regularly, at minimum whenever the agent's scope of capability expands, since new capabilities often create new categories of risky request nobody defined a boundary for yet.