The nonprofit Coalition for Health AI has released details of its long-discussed model card registry, a central repository for AI model cards they say will be standardized by them for the benefit of industry.
CHAI is working to build the model card registry for AI purchasers, like health systems, to get a basic understanding of a model’s training data, fairness metrics and intended use. Inclusion on the registry equates to a CHAI “stamp of approval” for AI vendors that have correctly filled out a CHAI model card.
The model registry does not solve the problem of validating the model, which requires evaluating the model’s performance against a locally representative data set, among other technical tests. Health systems undertaking AI procurement will still need to collect more information on how the model would perform in their setting.
“The goal here is to just make sure that the content reported in the nutrition label, everything is where it's supposed to be, that the instructions of the model card were followed,” Merage Ghane, director of responsible AI at CHAI, said in an interview. AI model cards are colloquially referred to as "nutrition labels" because they describe the contents of AI models in a standard format.
Ghane said the model registry, like the model card itself, will give health systems concrete information to answer standard questions about a model besides looking at a sales pitch or marketing materials. The central repository of model cards will also act as an industry resource for reliable information on responsible AI vendors.
“[For pilot users] it led to a productive first conversation, rather than a first conversation that feels like a full pitch, and then you're left with more questions, and you actually have answers,” Ghane said.
For developers, the model card registry will be a pool of potential customers, CHAI touts.
The model card registry itself is being built with global tech services company Avanade, which specializes in enhancing the experience of Microsoft products. The model card registry will be free and open to use, and initial funding came from donations by Avanade and CHAI.
CHAI is building the software that will review the contents of the AI model cards for completion and consistency with responsible AI standards laid out in the model card, including compliance with HTI-1. The review will be mostly automated so that information on AI vendors can be quickly uploaded to the registry, or kicked back to the vendors for further details.
CHAI is in the process of troubleshooting the software and determining how a human will be involved in the review. Ghane said that as CHAI works through the troubleshooting stages of the platform, a human will be more involved in reviewing which model cards are flagged for incompleteness and which are passed through the software. Ghane said CHAI has an engineer who works full-time on the model registry.
CHAI has signed on a slew of health systems and vendors that will support the initial launch of the product: Cleveland Clinic, Kaiser Permanente, Memorial Sloan Kettering, Mercy, Mount Sinai Health System, Providence, Rush University System for Health, Sharp HealthCare, Stanford Medicine, UMass Memorial and University of Texas Health System.
“For the those procuring these solutions, it'll sort of streamline those initial stages so everyone gets the same information up front, knows what they're looking at, knows how to ask the questions, and sort of work through that procurement process in a more streamlined way and a focus on what's important to know from the outset,” Ghane said.
CHAI also aims to build a gold standard sample model card for each AI use case to serve as a benchmark for the industry. Ghane said the organization has worked on discharge summary and sepsis use cases.
CHAI’s model card has a check box to denote if the model has been externally validated through an independent third party, including CHAI-certified AI assurance labs. CHAI’s grand AI assurance lab vision is a work in progress.
The company has not given a public update about the status of the assurance labs since October, when it released its draft framework for its certification criteria for assurance labs. At the time, CHAI leadership said it had 32 candidates to host the assurance labs and that it would have two sites running by the end of 2024.