LAS VEGAS—The Coalition for Health AI (CHAI) unveiled the first applied model card, completed by AI imaging company Aidoc. CHAI announced the completion of its draft model card template Oct. 18.
The CHAI model card working group unveiled the first applied model card at the CHAI Global Summit on Saturday, held in conjunction with the HLTH 2024 conference.
Model cards are designed to give stakeholders visibility into how a model works and what it does along with potential technical limitations and risks.
The contents of the model card are still under consideration. CHAI will collect feedback from members on the model card, and the public will have a comment period.
In the following example, Aidoc lists relevant information about its intracranial hemorrhage (ICH) model, which helps radiologists identify high-priority images. Aidoc has numerous FDA-cleared algorithms, and it used documentation submitted to the FDA to complete the model card, a representative of the company said at the summit. It took the company approximately three to four hours to complete the model card.
The model card is intended for use beyond algorithms cleared by the FDA.
At the top of the card, the AI vendor must list the release stage and date, where the model is available and contact information to submit inquiries or report an issue. After this key information, the model card has a summary section, which the CHAI model card creators stressed should include the most important content.
Image attribution: CHAI
In its current conception, the model card would be filled out by the developer of an AI model so that a buyer, such as a health system, could easily identify its uses and limitations. CHAI intends to release instructional documents alongside the model cards to assist AI developers with completion.
The model card would be voluntary to complete. However, CHAI built the model card to comply with the standards of the Department of Health and Human Services’ HTI-1 rule, which requires models used within certified electronic health records to have such a label.
The model card is separate from CHAI’s AI assurance lab framework.
The next section of the model card is the uses and directions section, which lists how the model should be used, what clinical staff should use it and the limitations of its use. The Aidoc ICH model card explains that the solution will display a compressed preview image when it notifies clinicians. The model card indicates the image should not be used for diagnostic purposes and should only be used for prioritization of images.
The AI system facts are listed next. This section includes the outcomes and outputs of the model, the model type, the foundation model, if used, and the input data source.
The Aidoc model was trained on DICOM images from 2015-22, and the majority of images were from 2021-22, the model card says. The images were obtained from across the U.S. and central Europe in diverse settings such as urban and rural community hospitals and academic medical centers.
The model card also lists Aidoc’s bias mitigation approach for the model, which includes analyzing data distribution according to age and gender. Aidoc noted on the model card that DICOM images do not include information about race or ethnicity but said that the company takes steps in other ways to mitigate racial and ethnic bias. The training and validation data included patients of a variety of health statuses, comorbidities and ages.
The model card notes that Aidoc does multiple retrainings of the model throughout its lifecycle. The model card includes information on ongoing maintenance, continuous monitoring and Aidoc’s transparency, intelligibility and accountability efforts.
The warnings section includes the model’s known risks and limitations. For the ICH model, Aidoc says the solution is intended to be used with other patient information and professional judgment. The clinical risk level for the model is low.
The CHAI community debated how to keep the model cards condensed but display relevant information.
During the discussions, though, Summit attendees agreed that the model cards should have a clear indication of whether patients were involved in the creation and testing of the product.