By Sunny Webb, Director of Product, AI & Data at Pager Health℠
As a healthcare leader, you’re under pressure to move quickly on AI. You may already be deep in the build vs. buy question before you’ve answered a more foundational one: if AI influences a member’s care journey, is your organization prepared to own the accountability that comes with it?
Whether organizations build or buy, the responsibility for member outcomes remains the same. The challenge isn’t simply deploying AI. It’s operationalizing AI in a way that’s reliable, governable, and useful in real clinical and member workflows – while also meeting clinical, legal, regulatory, and security standards that health plans are held to.
That question tends to separate organizations generating real outcomes from those managing stalled pilots.
Pilots lie. Production doesn’t.
Too many AI decisions are shaped by pilot quality. A polished pilot can make a system look production-ready, having been tested under ideal conditions with clean data, narrow workflows, and scripted success paths. Gartner estimates 60% of organizations will fail to realize the anticipated value of their AI investments. Other research shows AI project failure rates as high as 80%.
Care navigation doesn’t happen under ideal conditions. It happens across fragmented systems, incomplete provider data, changing eligibility, and operational silos that result in limited cross-functional visibility and coordination between teams.
There are real consequences when the chosen path of building or buying is wrong:
- Delays in care due to misrouting or disconnected engagement
- Inaccurate provider data sends members out of network, where costs run several times higher
- Inconsistent experiences erode member confidence
Most AI failures trace back to system design: fragmented and unreliable data sources, AI that doesn’t fit real clinical and operational workflows, and limited capacity to monitor performance once a system goes live. Organizations deploy pilots because they impressed stakeholders, then later discover that production environments surface the assumptions the pilot was built to avoid.
The accountability gap
Healthcare has always operated with accountability built into the system.
When a nurse makes a mistake while talking with a patient, there are consequences. The nurse’s judgment is scrutinized. Accreditation requirements come into question. Accountability is clear because the stakes are real.
But when an AI model makes a mistake, accountability becomes much less clear.
Who owns the outcome? The foundation model vendor? The implementation team? Your organization? AI introduces multiple layers of responsibility, which makes accountability harder to define when things go wrong.
The industry is still working through those questions. What’s already clear, however, is that you can’t treat AI-generated recommendations as exempt from the standards applied everywhere else in care delivery.
Whether organizations build or buy, accountability can’t be outsourced. The responsibility for member outcomes remains with the organization deploying the technology.
Scripted and narrow scopes combined with poorly governed AI layered on top of minimally tested models aren’t simply operational risks. In healthcare, they can become systemic failures that affect thousands of members before anyone recognizes a problem. Production-grade AI is substantially different from piloted AI. Details matter, as do the underlying guardrails, governance models, and monitoring – built to understand intent, context aware, and capable of escalating appropriately.
That’s why production readiness matters so much. The objective isn’t simply to deploy AI. The objective is to deploy systems you can trust, monitor, govern, and ultimately stand behind when outcomes matter.
What “production-ready” care navigation requires
Healthcare organizations don’t deploy AI models. They deploy accountability systems that happen to include AI.
Getting a model to generate an answer is different from getting that answer to a member reliably, accurately, and safely. Production-ready care navigation requires:
- Real-time provider data that’s continuously validated – the No Surprises Act mandates 90-day provider verification, yet 40% of directory errors persist after a year
- Scheduling integration that works without EHR connectivity, which is too operationally heavy for most organizations to implement at scale
- Clinical guardrails for high-risk scenarios
- Conversational and deterministic models that are context aware and recognize intent to improve engagement and safety
- Human escalation pathways built into the workflow from the start
- Monitoring infrastructure that surfaces degradation in production before it becomes a systemic problem
Each of those components is a sustained operational commitment. Most organizations budget for the model only. The surrounding infrastructure tends to be underestimated until it becomes the reason your pilot stalls.
The honest build vs. buy checklist
Building internally can be the right path for your organization. It makes sense when AI is a genuine strategic differentiator, when there’s meaningful proprietary data that creates an advantage, and when your organization is prepared to own the full operational burden over time. That burden includes data integration, workflow orchestration, governance, human-in-the-loop design, and continuous iteration as your members’ needs and system conditions change.
In healthcare, those conditions are difficult to meet at scale because the resource requirements are substantial, the timeline from your pilot to meaningful deployment is typically measured in years, and your ongoing operational responsibility continues after launch.
A practical build-vs.-buy decision usually comes down to four questions:
- Is this a true source of strategic differentiation for your organization?
- Do you have the data, workflow, and budgetary readiness to support production deployment?
- Is your organization prepared to own governance and operational responsibility over the long term?
- Are you optimizing for control, or for speed, predictability, and time to value?
If your organization’s answers point toward buying, the benefit isn’t simply faster deployment. It’s the opportunity to leverage infrastructure, governance processes, operational workflows, and lessons learned that would otherwise need to be developed and maintained internally.
The most important evaluation criteria extend beyond model performance. Organizations should understand how vendors approach governance, monitoring, human oversight, escalation pathways, and ongoing performance management once systems are operating in production.
Deployment timelines may be measured in months rather than years, allowing your teams to focus on the clinical and operational priorities you’re best positioned to own.
What it looks like when it actually works
The impact of production-ready care navigation can be significant. In Pager Health deployments, we’ve seen reductions in cost of care exceeding 25% and 93% of members directed to preferred in-network providers.1
These outcomes aren’t the result of a better model alone. They’re the result of systems built to work under real conditions, with real workflows, real consequences, and real accountability in scope from the start.
In healthcare, trust has always required accountability. AI should be held to the same standard.
The future of care navigation will belong to organizations whose systems continue to perform under real-world conditions, whose governance structures withstand scrutiny, and whose leaders are prepared to stand behind the recommendations their technology makes when members need help most.
1. Figures based on internal performance metrics