Quick answerOnce an AI agent has already given out-of-scope medical, legal, or financial advice and the customer has acted on it, the priority shifts from prevention to remediation: contain any ongoing harm immediately, reach out to the affected customer proactively rather than waiting for them to complain, document exactly what was said and what the customer did as a result, and loop in legal and, where relevant, insurance before deciding on compensation or correction. Treat this as an incident with a defined response process, not an isolated customer service ticket.
Prevention and remediation are different problems
Designing a refusal boundary is about preventing the agent from crossing into medical, legal, or financial advice it is not equipped to give. That design work matters, but no refusal boundary is perfect, and this post assumes the boundary has already failed: the agent gave the advice, and the customer already acted on it before anyone caught the mistake. At that point, better boundary design going forward does not undo what already happened, and the organization needs a separate, practiced response for the harm that is already in motion.
Step one: contain, then assess
The first move is establishing whether the harm is still unfolding or has already concluded. If the customer is about to take a further action based on the bad advice, such as a financial transaction not yet finalized or a next medical step not yet taken, containing that in-progress action is more urgent than anything else, including internal fact finding about how the mistake happened. Reach the customer directly and quickly, even before you have a full internal picture, if there is any chance further action can still be stopped.
Once containment is handled or confirmed unnecessary, assess what actually happened: what exact advice was given, what the customer did in response, and what the realistic consequences are. This assessment should pull the actual conversation transcript, not a summary, since the specific wording of out-of-scope advice often matters for both remediation and any liability exposure that follows.
Reach out before the customer has to complain
The same principle that applies to broken cross-system promises applies here with higher stakes: a customer who receives proactive, honest outreach acknowledging the mistake experiences the situation very differently than one who has to discover the harm themselves and then chase your company for an explanation. Do not wait for legal sign-off on every word before making initial contact; a short, honest acknowledgment that you are aware of the issue and actively working on it is almost always better than silence while internal process runs its course.
From the team
We build production AI systems for startups.
LLM pipelines, RAG, and agent workflows that hold up under real traffic — not just in the demo.
Document as you go, not after the fact
Build a written record of the advice given, the customer's response, your outreach, and every remediation step taken, starting from the moment the incident is identified. This documentation serves two purposes: it is what your legal and compliance teams will need if the situation escalates, and it is the raw material for the postmortem that should follow, similar in spirit to the incident postmortem and customer communication process you would run for any other production incident, adapted for the specific sensitivity of advice-related harm.
Loop in legal and insurance before deciding on compensation
Do not let the team closest to the customer relationship unilaterally decide what compensation or correction is appropriate without legal and, where applicable, insurance involvement. What feels like the right gesture of goodwill to a support team can have implications for liability admission or insurance coverage that are not obvious from a customer service perspective alone. This does not mean slowing down the human outreach in step two; it means routing the compensation and correction decision through the right internal process even while the human acknowledgment happens fast.
Close the loop by fixing the boundary that failed
Once the immediate incident is handled, feed exactly what happened back into the refusal boundary design process. A crossed boundary is direct evidence about a specific gap in how the agent was scoped, phrased, or tested, and it is far more valuable than hypothetical red-teaming for finding that particular failure mode. Treat every remediated incident as a mandatory input to the next round of boundary testing, not a closed case once the customer situation is resolved.
FAQ
Q: Should we always proactively disclose, even if the customer has not noticed the bad advice yet? In most cases yes, especially where real harm could still be prevented or mitigated. Waiting for the customer to notice on their own, when you already know, tends to make the eventual disclosure look far worse if it comes out later that you knew and stayed quiet.
Q: How fast should legal be looped in? Immediately for anything involving realistic harm, in parallel with the human outreach, not sequentially after it. The two tracks should run at the same time, with legal shaping what gets said rather than blocking the fact that something gets said quickly.
Q: Does this playbook change if the advice was technically within the disclosed limitations but still wrong? The remediation steps are largely the same, but the root cause investigation differs: a properly scoped advice type that was simply wrong is an accuracy problem to fix in the underlying content or model, not a boundary design problem.