Post-Launch ROI Tracking: The Metrics Cadence That Replaces a One-Time Calculation

Quick answer

Move from a single pre-launch ROI estimate to a recurring monthly review that tracks realized cost savings, containment rate, revenue influenced, and total cost of ownership against the original business case. Reforecast the ROI model quarterly using actual production data rather than the original assumptions, and flag drift early: an agent whose realized savings fall materially below the pre-launch estimate after 90 days needs a root-cause review, not a shrug.

A pre-launch estimate is a hypothesis, not a result

The pre-launch ROI calculation that got your project greenlit was built on assumptions: expected containment rate, projected volume, an estimated cost-per-conversation. Those assumptions were the best available at the time, but none of them were measured against reality yet. Treating that number as the final word on whether the agent is "working" is a common and costly mistake. It's a forecast, and forecasts need to be checked against what actually happened.

What to track monthly once the agent is live

Realized cost savings. Actual containment rate multiplied by actual cost-per-human-conversation avoided, using real volume, not the volume projection from the business case. This number should be recalculated monthly, not assumed to hold steady.

Total cost of ownership. Inference costs, ongoing maintenance budget, vendor fees, and the production team's time spent on monitoring and tuning. Pre-launch estimates almost always underweight this line item because it's genuinely hard to predict before the agent has run in production long enough to reveal its actual support burden.

Revenue influenced, where applicable: upsell conversions, reduced cart abandonment, faster response-driven conversion lift. This is the hardest number to attribute cleanly and the one most likely to be overstated if you don't build a clean control comparison from the start.

Containment and deflection trend over time, not just a snapshot. A containment rate that looked great at launch and has quietly eroded over six months as your product or policies changed is a much more useful signal than any single-month number in isolation, and it should be read alongside your broader production quality metrics rather than as a standalone figure.

Quarterly: reforecast, don't just report

Monthly tracking tells you what happened. A quarterly reforecast tells you whether the original business case still holds. Rebuild the ROI model using trailing 90-day actuals instead of the original launch-time assumptions, and compare the new forecast to the original one. If the delta is large in either direction, that's worth a structured conversation with whoever owns the board and CEO-level metrics reporting, both because a shortfall needs explaining and because a big positive surprise is a signal to invest further rather than just a nice number to mention in passing.

What "good" drift looks like vs. what's a red flag

Some drift is expected and healthy: containment typically improves in the first two to three months as the agent's knowledge base and eval coverage mature, then plateaus. A red flag is different: cost-per-conversation rising instead of falling as volume scales, or containment declining for more than one consecutive month with no corresponding product or policy change to explain it. That pattern usually means the agent's knowledge base has drifted out of sync with a changing product, and it's worth checking against your knowledge base sync process before assuming it's a model quality issue.

Who should own this cadence

Assign monthly tracking to whoever owns production quality metrics day to day, and reserve the quarterly reforecast for a named business owner, not an engineering owner, since the reforecast is fundamentally a business-case conversation, not a technical one.

FAQ

Q: Post-Launch ROI Tracking: The Metrics Cadence That Replaces a One-Time Calculation A: Move from a single pre-launch ROI estimate to a recurring monthly review that tracks realized cost savings, containment rate, revenue influenced, and total cost of ownership against the original business case. Reforecast the ROI model quarterly using actual production data rather than the original assumptions, and flag drift early: an agent whose realized savings fall materially below the pre-launch estimate after 90 days needs a root-cause review, not a shrug.