Generative AI in healthcare has reached a turning point as adoption continues to grow, but health systems face critical challenges with scaling the technology.
Establishing effective policies around governance and security and building trust around generative AI solutions in healthcare continue to be major hurdles.
New startup Qualified Health aims to work with health systems to tackle these foundational issues by building an AI infrastructure for healthcare. The company's leadership team is stacked with experts across healthcare and AI including clinicians, AI engineers, policy experts and healthcare operators as it expands its partnerships with health systems.
"We see ourselves very simply as infrastructure for generative AI in healthcare," Justin Norden, founder and CEO at Qualified Health and previously a partner at GSR Ventures told Fierce Healthcare in an exclusive interview. "What does that mean? It's everything from delivering the technology, training and support to get started with these tools and then scale them safely across the organization."
Qualified Health's platform simplifies implementations, expedites measurable ROI and centralizes the orchestration, evaluation and governance of AI solutions, according to the company.
"We see our role as we are that infrastructure where we're getting the basics in place, we're getting the data management in place, we're getting the pipes in place, if you will, and on top of that infrastructure, we can put in place best-in-class evaluation methods," Norden said.
The startup banked $30 million in a hefty seed funding round backed by investors SignalFire, Frist Cressey Ventures, Town Hall Ventures, Healthier Capital, Intermountain Ventures and Flare Capital along with a list of notable angel investors.
Two years ago, when ChatGPT launched, the conversation around generative AI in healthcare began to shift, said Norden, who is an Adjunct Professor at Stanford Medicine in the Department of Biomedical Informatics Research, where he teaches the Generative AI and Medicine course.
"Lots of people were thinking about point solutions, quick applications and were building demos, but what was really challenging for systems was that people were having trouble taking those into full production. People were having trouble actually launching them, scaling them and getting the value. One of those key areas that was lacking was just building trust in these tools," Norden said.
Providers lack transparency into algorithm performance and the accuracy of the results. They also lack the ability to monitor updates to AI deployments and face challenges integrating GenAI tools into their workflows, Norden noted.
Without robust infrastructure to govern and monitor AI systems, healthcare organizations are forced to choose between innovation and risk.
Qualified Health launched as a Public Benefit Corporation (PBC) to partner with health systems as these organizations deploy generative AI solutions. The company's infrastructure solution provides guardrails for active AI governance and continuous monitoring of algorithm performance.
Working with health systems' IT teams, Qualified Health's platform provides "enforceable" governance equipped with role-based access controls, risk alerts, data privacy protections and safeguards against AI hallucinations.
"Governance is a topic that comes up a lot where state of the art is a PDF document saying, 'Please do this and please don't do that.' How do you build that into the tools that they're actually using?" Norden said.
Qualified Health's tools allow healthcare organizations to convert their AI guidelines into enforceable policies.
The platform also provides post-deployment monitoring with complete observability into application performance and usage and provide human-in-the-loop evaluation and escalation systems, according to the company.
This infrastructure allows healthcare organizations to scale deployments confidently, track ROI and provide audit trails for compliance.
And, the company's platform provides tools for healthcare AI agent creation to enable healthcare teams to rapidly create and deploy AI agents to automate workflows. Health systems are deploying AI agents to drive cost savings, generate revenue and improve care quality.
The startup will use the $30 million in funding to grow it engineering team as it continues building enterprise software for health systems. "We're based in Palo Alto, we're hiring expensive ML/AI talent and this capital is to hire the best team to go do this," Norden said.
He added, "This isn't to spend $30 million training a new large language model. Our opinion is that's a bit of a red herring. The Googles and OpenAIs and Anthropics will spend the hundreds of millions of dollars training fundamentally these new models. We're building the infrastructure to wrangle and support and put the pieces together in healthcare."
The co-founding team of Qualified Health includes Beau Norgeot, Ph.D., Chief AI Officer at the startup, who was previously chief data officer at Lucid Lane and Staff VP of AI at Elevance Health and Nirav Shah, M.D., a senior scholar at Stanford University whose expertise spans public health, digital health and generative AI, public and private health insurance, and clinical operations.
Kedar Mate, M.D., also is a co-founder and Chief Medical Officer of Qualified Health, and he recently led the Institute for Healthcare Improvement as its president and CEO. Shantanu Phatakwala serves as Chief Commercial Officer at Qualified Health and brings diverse experience spanning operations, technology, strategy and entrepreneurship. Shantanu was the Chief Data Science Officer at Haven, a healthcare venture formed by Amazon, Berkshire Hathaway, and JP Morgan Chase, and he was a founding member of Evolent Health.
Norden was the CEO and co-founder of Trustworthy AI, a company acquired by Waymo that built software to develop, test and deploy safety-critical autonomous systems.
"There is a lot of learning to be had from the autonomous vehicle world. It's not enough just to build an AI model, test it once and then deploy it and cross your fingers and hope for the best, which candidly, has been the status quo in healthcare, even just how we think about approvals and technology," he said. "As we're moving into this next phase with getting AI deployments where you need to see how people are using these systems as fundamentally people interact now with these tools, and ultimately, we want to measure the outcomes. But, we don't have the infrastructure in place to capture that data, to measure that data, see how people are using these tools."
Federal regulators also have highlighted the need for post-deployment monitoring of AI tools as the technology is rapidly adopted in healthcare.
"There have been conversations around health systems needing to measure and mitigate bias with these tools, measure performance, make sure these things are working. There's no infrastructure to do that today, So, we're coming in as a partner to help them do that," Norden said.
Qualified Health is currently working with a "handful" of health systems, Norden said.
"We're partnering with them to build these solutions. The way we see our job is how can we bring together their voices and their needs and then make those conversations happen so they can learn from each other. And, how can we bring regulators and academics to those conversations because the clear cut rules for AI don't exist yet today in healthcare," he said.
"We want to be a partner to show what 'good' AI deployment looks like and demonstrate what 'good' evaluation looks like for these systems and then work together with them," he added.
Healthcare is in the early innings of deploying generative AI technologies. Qualified Health is focused on addressing gaps in trust, access to safe tools and data security to help health systems accelerate innovation while maintaining control.
"We're at the beginning of a fundamental shift in how healthcare is delivered. Qualified Health is uniquely positioned to become the cornerstone that enables safe, effective AI deployment across the entire healthcare ecosystem. The opportunity ahead is immense," Amir Dan Rubin, CEO of Healthier Capital, one of Qualified Health's investors, said in a statement.