Quick answerRebuild a struggling AI agent program from scratch, rather than incrementally fixing it, when the problems trace back to foundational architecture or scope decisions made early on, not to specific bugs or gaps that a normal fix cycle can close. A useful test: list the last quarter's fixes, and check whether they are converging (each fix closes a class of problem and stays closed) or compounding (fixes routinely reopen adjacent problems, or the same category of issue keeps resurfacing in a new form). Convergence means keep fixing incrementally. Compounding, especially alongside a team that can no longer confidently predict what a given change will break, means the underlying architecture cannot support the current scope, and a rebuild will be cheaper than continuing to patch it.
This decision is not about how bad things are right now, it is about the trend
A struggling program at any given moment looks similar whether the right answer is to fix it or rebuild it: open bugs, frustrated stakeholders, missed reliability targets. The decision has to be made on the trend of the fix cycle, not the current state, because both a fixable program and a rebuild-worthy program can look equally bad on any single day. Pull the last two or three months of fix history and categorize each fix as either closing a problem for good or reopening a related problem elsewhere, in a different flow, a different edge case, a different downstream system.
A program where fixes converge, where each closed issue stays closed and the total count of open issues trends down even as new features ship, is fundamentally healthy even if it currently has a long backlog. A program where fixes compound, where the team routinely discovers that today's fix broke something adjacent, or where the same underlying problem keeps resurfacing dressed differently, is telling you the architecture cannot support what is being asked of it, and no amount of additional incremental effort will change that trend line.
The team's confidence signal is as reliable as the metrics
Ask the engineers who work on the program directly: when you make a change, how confident are you that you know what it will affect? A team that can answer specifically, this change touches these three flows, and here is how we will verify it did not break anything else, is working with a system whose behavior is legible, even if the system itself has real problems. A team that answers with some version of we're not sure, we'll find out when something breaks is describing a system whose behavior has become genuinely unpredictable to the people who built it, which is a stronger signal for rebuild than any single metric, because it means the team's own mental model of the system no longer matches its actual behavior.
This diagnostic works even when the metrics themselves are ambiguous or lag reality, since team confidence tends to erode well before it shows up cleanly in aggregate reliability numbers, and it is a much faster check to run than a full metrics review.
How this differs from the in-house-versus-vendor decision
Reconsidering an already-launched in-house AI agent in favor of a vendor is a different decision than deciding to rebuild, because it changes who builds and operates the system, not necessarily what gets built. A company can conclude a rebuild is needed and still choose to do that rebuild in-house, if the underlying reason the current system struggles is architectural rather than a fundamental mismatch between in-house capability and what the product needs. Conversely, a company might switch to a vendor without ever concluding a full rebuild was architecturally necessary, simply because the vendor now offers the capability more cost-effectively than maintaining any version in-house would.
Run these as two separate questions in sequence: first, does the current architecture, wherever it lives, need to be rebuilt or can it be incrementally fixed, and second, independent of that answer, should the company build (or rebuild) this in-house or move to a vendor. Collapsing them into one decision risks choosing a vendor to escape an architecture problem that would resurface with the vendor's system too, or rebuilding in-house when the real answer was that vendors now do this better than continued in-house investment ever will.
Pausing before rebuilding: what actually needs to stop
A rebuild decision does not necessarily mean shutting the current system off immediately, since customers depending on it need continuity, but it should mean pausing new feature investment in the current architecture immediately, redirecting that capacity toward the rebuild, rather than continuing to build new capability on a foundation already identified as compounding. Continuing to ship features on an architecture already flagged for rebuild both slows the rebuild and adds more surface area that will eventually need to be re-migrated.
This rebuild-versus-fix decision is distinct from a revenue-triggered scope rollback, where a slow revenue decline attributable to the agent's scope is the trigger for reducing what the agent does. A rebuild decision is an engineering and architecture call about whether the current system can support its current scope reliably; a scope rollback is a business call about whether the current scope is still worth supporting at all, regardless of how reliably the underlying system runs it. A struggling program can face either question independently, or both at once, and they should be evaluated on their own separate criteria rather than treated as the same underlying problem.
Once a rebuild is underway, hold it to the same bar as what to document so a future team that never built your AI agent could actually rebuild it, so the next rebuild-or-fix decision, years from now, starts from better documentation than this one did.
FAQ
How long should a rebuild-versus-fix evaluation take before making a call?
A focused review of two to three months of fix history and a direct conversation with the team building the system is usually enough to see the convergence-versus-compounding pattern clearly. If the signal is genuinely ambiguous after that review, that ambiguity is itself useful information, since a program that is clearly rebuild-worthy or clearly fixable tends to show it without an extended investigation.
Can a partial rebuild, fixing the architecture but keeping some components, work?
Yes, and it is often the right answer rather than an all-or-nothing choice: identify which specific layer is actually compounding problems, often something foundational like the orchestration or tool-use layer, and rebuild that while preserving components that have been converging fine on their own, such as a well-functioning eval pipeline or a stable integration layer.
What should the team do with the old system while the rebuild is happening?
Keep it running for existing customers with a strict, minimal maintenance-only policy, security and critical-bug fixes only, no new feature work, and set a clear cutover plan with a target date, rather than letting the old system continue to accumulate scope in parallel with the rebuild effort. A rebuild that has to also keep pace with an actively growing old system rarely finishes on schedule.