Quick answerWhen an AI agent personalizes its behavior for an individual customer purely by accumulating their interaction history — with no retraining step and no single change anyone approved — its behavior for that specific customer can drift meaningfully over months without triggering any of the review processes built around model or prompt changes, because nothing about the underlying model or prompt actually changed. Govern this by defining what "personalization" is allowed to affect (tone and prioritization, generally) versus what it must never affect (factual accuracy, pricing, policy application), running periodic per-customer drift checks that compare current behavior against a fresh baseline, and giving customers a visible way to see and reset what the agent has inferred about them.
Why accumulated drift evades your existing change-management process
Standard AI agent governance is built around discrete, reviewable events: a new prompt version, a new model, a new tool. Whether you build or buy the underlying memory and personalization infrastructure is one of those discrete decisions, made once at build time. But once that infrastructure is live, each individual customer's accumulated interaction history can shift what the agent does for them specifically, gradually, with no equivalent discrete event to review — nobody signs off on customer #4,281's personalization state the way they'd sign off on a new prompt version, because there was no single change; there were a thousand small accumulations that were never individually reviewable.
This is a different mechanism than configuration-driven quality decay
Catching an AI agent's quality slowly decaying from a hundred small configuration changes is about your own team's accumulated prompt and config edits degrading behavior for everyone, detectable against a fixed evaluation set because the underlying system is what changed. Personalization drift is the inverse: the system and its configuration are stable, but each individual customer's experience diverges from every other customer's and from its own past baseline, driven entirely by that customer's own history. A fixed evaluation set run against the general system won't catch this, because the drift lives inside individual customer state, not in the shared configuration the eval set is designed to test.
Draw a hard boundary around what personalization can touch
Before building drift detection, decide explicitly what accumulated history is allowed to influence and what it must never influence. Tone, topic prioritization, and which of several equally correct explanations the agent leads with are reasonable candidates for personalization. Factual accuracy, pricing calculations, and policy application should never vary based on accumulated history — if two customers ask the identical factual question, accumulated personalization for one of them should never be the reason they get a materially different answer on a matter of fact or entitlement. Document this boundary explicitly, because without it, "personalization" quietly becomes the explanation offered after the fact for what was actually an ungoverned inconsistency.
Run periodic per-customer baseline comparisons, not just system-wide evals
Because drift lives at the individual level, detection needs a sampling process that periodically compares a given customer's current agent behavior against what a fresh, no-history version of the agent would produce for the same input. A meaningful, sustained divergence on anything outside the allowed personalization boundary is a signal worth investigating, even if the system-wide eval set looks perfectly healthy. This is a distinct capability from your memory infrastructure and doesn't require retraining anything — it requires running the same query twice, once with history and once without, and diffing the results on the dimensions that matter.
Give customers visibility and a reset, separate from opting out entirely
A customer being able to opt out of being remembered without opting out of AI handling altogether covers the binary choice to stop accumulation going forward. Governance of drift itself is a step further: customers should also be able to see a summary of what's currently influencing their personalized experience and reset it, independent of whether they want personalization to continue — because the fix for detected unwanted drift is often not "turn personalization off forever," it's "clear what's accumulated and start fresh."
FAQ
Who should own reviewing personalization drift, since it's not a discrete change anyone approved? Assign it to whoever owns AI agent quality generally, on a recurring cadence (monthly is a reasonable default), rather than leaving it as an ad hoc investigation that only happens after a customer complaint surfaces a problem.
Does this require a different technical architecture than standard memory infrastructure? No — it requires an additional evaluation capability layered on top of whatever memory infrastructure you already have, not a rebuild of the underlying system.
How do you tell drift apart from personalization working as intended? Personalization working as intended stays within the boundary you've defined (tone, prioritization) and remains explainable by reference to the customer's actual stated preferences or behavior. Drift is divergence outside that boundary, or divergence inside it that no longer traces back to anything the customer actually did or said.