UPDATED 12:00 p.m. ET Jan. 9
CHAI model card now open source on GitHub
The Coalition for Health AI (CHAI) released an open access version of its model card for healthcare AI on Thursday.
The card, which has been compared to a nutrition label for algorithms, is available for free on GitHub, a popular open source platform. The files include a fillable template of the card, instructions, an example, resources and references.
CHAI unveiled its draft model card in October during the HLTH conference in Las Vegas along with a completed example by member company Aidoc. It announced that the draft version of the card was available to CHAI member organizations in November.
The model card can be used by any health AI developer to offer users more transparency on the creation and contents of the algorithm. The card includes information on known risks and limitations, the data that trained the algorithm, bias mitigation approaches and any ongoing maintenance.
The model card can also be used for some predictive algorithms hosted in electronic health records to comply with HTI-1, a rule issued by HHS' Assistant Secretary of Technology Policy (ASTP).
CHAI is accepting public feedback on the card and encourages stress testing. Comments are due to CHAI by Jan. 22.
CHAI is perhaps best known for its efforts to establish a nationwide network of assurance labs, which would test the claims made by AI developers. Former President of the American Medical Association, Jesse Ehrenfeld, M.D., recently published an editorial in Springer Nature that expressed his doubt about the concept.
“While the creation of a nationwide network of health AI assurance laboratories seems compelling, it is fraught with foundational challenges and insurmountable constraints that make it as feasible as building a sandcastle on the beach during a category five hurricane,” Ehrenfeld and co-author Keith Woeltje wrote.
The editorial was published in early December.
LAS VEGAS—The Coalition for Health AI (CHAI) unveiled the first applied model card, completed by AI imaging company Aidoc. CHAI announced the completion of its draft model card template Oct. 18.
The CHAI model card working group unveiled the first applied model card at the CHAI Global Summit on Saturday, held in conjunction with the HLTH 2024 conference.
Model cards are designed to give stakeholders visibility into how a model works and what it does along with potential technical limitations and risks.
The contents of the model card are still under consideration. CHAI will collect feedback from members on the model card, and the public will have a comment period.
In the following example, Aidoc lists relevant information about its intracranial hemorrhage (ICH) model, which helps radiologists identify high-priority images. Aidoc has numerous FDA-cleared algorithms, and it used documentation submitted to the FDA to complete the model card, a representative of the company said at the summit. It took the company approximately three to four hours to complete the model card.
The model card is intended for use beyond algorithms cleared by the FDA.
At the top of the card, the AI vendor must list the release stage and date, where the model is available and contact information to submit inquiries or report an issue. After this key information, the model card has a summary section, which the CHAI model card creators stressed should include the most important content.
Image attribution: CHAI
In its current conception, the model card would be filled out by the developer of an AI model so that a buyer, such as a health system, could easily identify its uses and limitations. CHAI intends to release instructional documents alongside the model cards to assist AI developers with completion.
The model card would be voluntary to complete. However, CHAI built the model card to comply with the standards of the Department of Health and Human Services’ HTI-1 rule, which requires models used within certified electronic health records to have such a label.
The model card is separate from CHAI’s AI assurance lab framework.
The next section of the model card is the uses and directions section, which lists how the model should be used, what clinical staff should use it and the limitations of its use. The Aidoc ICH model card explains that the solution will display a compressed preview image when it notifies clinicians. The model card indicates the image should not be used for diagnostic purposes and should only be used for prioritization of images.
The AI system facts are listed next. This section includes the outcomes and outputs of the model, the model type, the foundation model, if used, and the input data source.
The Aidoc model was trained on DICOM images from 2015-22, and the majority of images were from 2021-22, the model card says. The images were obtained from across the U.S. and central Europe in diverse settings such as urban and rural community hospitals and academic medical centers.
The model card also lists Aidoc’s bias mitigation approach for the model, which includes analyzing data distribution according to age and gender. Aidoc noted on the model card that DICOM images do not include information about race or ethnicity but said that the company takes steps in other ways to mitigate racial and ethnic bias. The training and validation data included patients of a variety of health statuses, comorbidities and ages.
The model card notes that Aidoc does multiple retrainings of the model throughout its lifecycle. The model card includes information on ongoing maintenance, continuous monitoring and Aidoc’s transparency, intelligibility and accountability efforts.
The warnings section includes the model’s known risks and limitations. For the ICH model, Aidoc says the solution is intended to be used with other patient information and professional judgment. The clinical risk level for the model is low.
The CHAI community debated how to keep the model cards condensed but display relevant information.
During the discussions, though, Summit attendees agreed that the model cards should have a clear indication of whether patients were involved in the creation and testing of the product.