Quick answerTrack agent quality against a fixed, unchanging evaluation set on a regular cadence, independent of any single prompt or config change, so a slow decline shows up as a trend line rather than being invisible inside dozens of individually reasonable-looking changes. This is a different failure mode from a foundation model vendor silently changing behavior, since here the model is stable and the company's own accumulated tweaks, each one approved for a good reason at the time, are the cause. The fix is a standing regression-detection practice, not a single root-cause investigation, because there usually is not one change to point to.
Why this drift is nearly invisible change by change
Each individual prompt tweak, guardrail addition, or configuration change usually goes through some review and looks like a clear improvement or a reasonable fix for the specific issue it addresses. The problem is cumulative: a prompt that has absorbed eight months of one-off patches for edge cases can become internally inconsistent, bloated, or subtly contradictory in ways no single change author would have caught, because they only ever saw their own diff, not the accumulated whole. This is structurally different from the single-vendor drift case in when your model provider silently changes behavior mid-contract, where one external event causes the shift; here, dozens of internal, individually-approved events cause it collectively.
Run a fixed evaluation set against every version, not just new ones
Maintain an evaluation set that does not change when the agent's prompts or configuration change, and run it on a fixed cadence, monthly at minimum, comparing results against the historical trend line rather than just a pass or fail threshold on the current version. A single data point tells you almost nothing; a declining trend across six months of otherwise-approved changes tells you exactly what a single before-and-after comparison would miss. This is the same infrastructure already recommended in building a golden evaluation dataset for your AI agent; the addition here is treating that dataset as a standing regression monitor across time, not just a one-time launch gate.
From the team
We build production AI systems for startups.
LLM pipelines, RAG, and agent workflows that hold up under real traffic — not just in the demo.
Give every configuration change a version and an owner
Once a decline shows up on the trend line, you need to be able to reconstruct what changed and when, which is impossible if prompt and configuration edits are not versioned and attributed. Adopt the same discipline you would expect for application code: every change to the agent's prompt, guardrails, or routing logic gets a version, a changelog entry, and an owner, so a quality investigation can correlate the decline against a real timeline of changes instead of guessing. This connects directly to versioning and rolling back AI agent prompts, which covers the mechanics of maintaining that history; the detection practice in this post is what tells you when it is time to actually go looking through it.
FAQ
How often should the fixed evaluation set itself be refreshed? Rarely, and deliberately. Refreshing it too often defeats its purpose as a stable baseline; a reasonable practice is a full refresh once or twice a year, with the historical trend line reset and clearly marked at that point so old and new baselines are never silently compared.
Isn't this the same problem as inconsistent answers to the same question? Related but not the same. Inconsistent answers can happen even with a perfectly stable configuration, due to model non-determinism; configuration creep is a directional decline over time caused by the accumulation of changes, which a stable-configuration inconsistency problem would not show on a fixed evaluation trend line.
Who should own responding to a detected decline? Whoever owns the agent's prompt and configuration day to day should own the initial investigation, but treat a confirmed multi-month decline as a finding for whatever governance group already reviews other agent quality metrics, not something resolved quietly by one engineer.