Quick answerDefault to a single shared platform when business units serve similar customer types through similar channels, since the shared infrastructure, evaluation tooling, and guardrail library pay for themselves quickly across multiple teams. Let a business unit build its own when its product, regulatory environment, or customer base is different enough that shared infrastructure would need so many exceptions it stops being shared in any meaningful sense. This is the underlying build-one-versus-build-many architecture question, and it should be settled before you decide how a shared team operates day to day, since the operating model only matters once you have already chosen to share infrastructure at all.
This decision comes before the operating-model question, not after
Companies often jump straight to designing how a shared team should run, without first settling whether a single platform is even the right architecture for every business unit involved. That sequencing mistake produces a center of excellence built to serve units that never should have shared infrastructure in the first place. The center-of-excellence operating model question, how a shared team divides work, sets priorities, and supports multiple internal customers, only makes sense once you have already decided that sharing is the right architecture. Settle the build-one-versus-build-many question first, on its own terms.
The real test is how much would need to be an exception
A single platform earns its cost when the majority of what different business units need is genuinely the same: similar conversation patterns, similar compliance requirements, similar customer channels. The test worth applying honestly is how many business-unit-specific exceptions the shared platform would need to carry. If every unit needs its own compliance layer, its own escalation logic, and its own data isolation boundaries because of genuinely different regulatory environments, the platform stops being shared in anything but name, and the coordination overhead of forcing everyone through one system will cost more than the infrastructure savings it was meant to deliver.
From the team
We build production AI systems for startups.
LLM pipelines, RAG, and agent workflows that hold up under real traffic — not just in the demo.
Ownership of the roadmap follows the architecture choice
Once the architecture question is settled, who owns the roadmap at company-wide scale becomes a much easier question to answer, because a single-platform decision implies a single roadmap owner balancing competing business-unit needs, while a per-unit decision implies each unit owns its own roadmap with only loose coordination between them. Companies that get this backward, assigning a single roadmap owner before deciding whether the architecture is actually shared, tend to create a roadmap-ownership fight that is really a proxy for the unresolved architecture question underneath it. Resolve the architecture question explicitly, in writing, before staffing the team that will operate whatever you decide, similar to how you would size and budget a team only after the scope of what that team owns is actually clear.
FAQ
Can a company start with per-unit builds and consolidate later? Yes, and this is often the more realistic path for a company that grew through separate initiatives before anyone had the full picture. Consolidation later is easier when each unit's build kept reasonably compatible data formats and evaluation practices from the start, even if the platforms themselves stayed separate.
What is the biggest risk of choosing a single platform when units are actually too different? The platform becomes a bottleneck: any change requires negotiating across units with genuinely conflicting needs, which slows every unit down to the pace of the slowest-moving one, and eventually a frustrated unit builds its own system anyway, just without any coordination with the platform team.
Does company size matter more than product similarity for this decision? Product and customer similarity matter more. A large company with genuinely similar business units across the board is a better single-platform candidate than a smaller company whose two business units serve completely different regulatory environments and customer types.