To realize the potential of AI in incidental findings management, trust must be built directly into the system itself
Across healthcare, the conversation about artificial intelligence has moved quickly from possibility to urgency. Imaging volumes are rising. Clinical documentation is growing more complex. Staffing constraints show no signs of easing. AI systems capable of interpreting narrative clinical text are increasingly seen as part of the solution.
But deploying any generalized AI tool in a clinical setting without rigorous supervision carries real risk. There is a meaningful difference between what an AI system appears capable of doing and what it has been validated to do in practice — and in a clinical environment, conflating the two is not acceptable. Governance committees, clinical leaders, and health system administrators are only beginning to reckon with that distinction.
Why Human Validation Is Still Necessary in Most Clinical AI Workflows
There are three primary AI models in use today, each with distinct strengths and limits.
Pattern-based Natural Language Processing (NLP) performs well on structured, high-frequency tasks but loses reliability when clinical language becomes ambiguous or contextually layered.
Large Language Models (LLMs) bring flexible, context-sensitive interpretation across varied clinical language. In an analysis of more than 2,000 real-world radiology reports from over 40 hospital systems, LLMs achieved accuracies of 81–85% without domain-specific tuning — strong baseline performance. But because LLMs are probabilistic, their outputs vary and require continuous verification to ensure reliability holds across diverse documentation styles and over time.
Computational Linguistics (CL) operates on deterministic, rule-based logic where every decision follows a traceable reasoning path. That transparency is a genuine clinical asset — every conclusion can be explained, reviewed, and audited. The CL model in the same study demonstrated higher precision and consistency within the specific clinical use case evaluated.
Yet any model working independently still requires verification before care can safely advance, introducing operational burden at scale. So how can we ensure an AI output can be acted on without human review?
The Operational Cost of the Validation Burden
Understanding why that question matters requires understanding what happens when it goes unaddressed.
Every AI-generated output that a clinician must verify before acting on contributes to what is known as the Validation Burden — the cumulative human effort required to confirm findings prior to action. In incidental findings and screening programs, this includes verifying extracted data, assembling clinical context, determining the appropriate guideline-based next step, and activating a care plan.
For individual cases, the effort is modest. Across hundreds of thousands of radiology reports in a health system, it compounds quickly — becoming one of the most significant operational constraints early detection programs face. When every finding requires verification, the promise of automation becomes, in practice, the burden of supervision.
Building Trust into the IT Infrastructure
To address this challenge, we propose an "Architecture of Trust" framework that embeds validation directly into the infrastructure itself — rather than treating it as a downstream manual step.
The approach works through deliberate independence. Two fundamentally different but complementary AI systems — a deterministic CL engine and a probabilistic LLM — analyze the same clinical report separately, without access to each other's conclusions. Their outputs are then compared. When both systems independently arrive at the same conclusion and that agreement meets a predefined performance threshold, the finding can be computationally validated and downstream workflows proceed automatically: guideline-based recommendations, care team notifications, and longitudinal tracking updates.
When the systems disagree, or when agreement falls below the required threshold, the case is automatically routed to human review. That routing is objective, consistent, and fully auditable — no judgment call required at the point of inference.
The performance implications are significant. When the CL model and LLM independently agreed, the error rate within that subset fell below one percent — approaching zero across several model configurations. Independent agreement functions as a measurable signal of reliability.
Why This Matters
Computational validation provides a framework for applying automation more safely and consistently within clinical workflows. It establishes clear, predefined criteria for when automation is warranted, produces a traceable reasoning path for every decision, and ensures human review is applied objectively according to pre-established safety and accuracy thresholds — a must-have in clinical settings.
A Foundation for What Comes Next
Healthcare AI is no longer a future consideration — it is an active deployment challenge. The organizations navigating it most effectively are those that have moved beyond evaluating AI on benchmark performance alone.
The Architecture of Trust offers a framework grounded in the principle that clinical AI must demonstrate its reliability, not assert it. Deterministic models provide the transparency and traceability governance requires. Probabilistic models provide the linguistic flexibility real-world clinical documentation demands. Computational validation — applied at the moment of inference rather than retrospectively — creates the conditions under which automation can be both safe and scalable.
What this framework ultimately enables is a more measurable, auditable, and scalable approach to AI clinical deployment.
To learn more, read the white paper "Reducing the Validation Burden with Safe and Trustworthy Automation. "