UMass Chan Medical School, MITRE plan to open assurance lab to test healthcare AI

UMass Chan Medical School and MITRE announced plans to open an assurance lab to test healthcare artificial intelligence technologies.

The new R&D center will be located at the Mass Chan Medical School, just west of Boston in Worcester, Massachusetts, and run in partnership with MITRE Corporation to address health AI safeguards, support startup testing and boost regional workforce training, according to the organizations.

The state has pledged $555,000 to support the new health AI assurance lab. The lab will be supported through the Technology & Innovation Ecosystem Awards Program managed by the Innovation Institute at the Massachusetts Technology Collaborative, along with $137,000 in matching investments from the private sector.

The lab will use cutting-edge tools to develop infrastructure and processes for the evaluation of AI-enabled solutions in healthcare, the organizations said in a press release. And, the lab will evaluate technologies using a framework developed by MITRE and defined by the Coalition for Health AI (CHAI), a national consortium.  

Developers of AI-enabled solutions will bring their products to the assurance lab for assessment of their features and expected uses. The organizations said this process will "foster trustworthy and responsible use."

"The lab will contribute to emerging national standards for the evaluation of digital health technologies which can support commercial applications for healthcare innovators in the region. The funds will also support the construction of physical spaces that will allow for collaboration, workforce training, and R&D focused on security infrastructure to ensure the safety of health AI products before they’re put into general use," the organizations said.

The AI assurance lab at UMass Chan Medical School will be the first step in creating what MITRE and many other healthcare stakeholders envision as a "national network of AI assurance labs," said Doug Robbins, vice president at MITRE.

MITRE, a nonprofit corporation, recently opened a new AI assurance and discovery lab to test government uses of AI for potential risks. Robbins said the organization expects to connect both labs as part of the national network of assurance labs.

CHAI, launched in the spring of 2022, includes representatives from more than 1,500 member organizations including hospital systems, tech companies, government agencies and advocacy groups. Its founding members include Duke AI Health, Google, Johns Hopkins University, Mayo Clinic, Microsoft, MITRE, SAS, Stanford Medicine, University of California (UC), Berkeley and UC San Francisco. 

CHAI aspires to contribute to best practices with the testing, deployment and evaluation of AI systems and supports the development of assurance labs to test and monitor AI technologies.

“Given the immense potential of AI to transform everyday life, we want to be mindful of its overall impact,” said Patrick Larkin, director of the Innovation Institute at MassTech and a member of the AI Strategic Taskforce.

“That means supporting investments that ultimately provide startups and established companies with the necessary tools, methods, processes, infrastructure, and a simulated real-world environment to develop and refine their AI-driven solutions in a controlled setting. This will help promote safety and efficacy, while facilitating collaboration, data sharing, and the development of cutting-edge technologies that will further position Massachusetts as a global leader in health AI," Larkin said.

“We are committed to researching and developing health AI tools to strengthen our state’s leadership in healthcare,” said Secretary Yvonne Hao at the executive office of economic development and co-chair of the state’s AI Strategic Taskforce.

“The new Health AI Assurance Laboratory is an incredible opportunity to grow the AI ecosystem by building strategic partnerships and providing career opportunities to students in data analysis, informatics, and machine learning. Through the leadership of the world-class researchers at UMass Chan and MITRE, this new lab will help us build a stronger health AI industry in Massachusetts and accelerate responsible innovation for the global marketplace," Hao said.