Industry Voices—The keys to improving breast cancer AI are in your hospital PACS

In recent years, artificial intelligence applications for breast cancer have gotten a lot of attention in mainstream media. It’s easy to understand why. Given the fear that a cancer diagnosis raises, it can be reassuring to hear that AI might offer another tool in the healthcare team’s toolbox.

Currently, there are at least 16 FDA-cleared mammography algorithms, with capabilities including cancer detection, cancer risk prediction and quality control.

What doesn’t always come through amid the buzz is that AI technology has some distance to go before it lives up to its full cancer-fighting potential. A recent meta-analysis of AI applications in mammography pointed out issues with poor quality, bias and untraceable data. Researchers concluded that “current evidence on the use of AI systems in breast cancer screening is a long way from having the quality and quantity required for its implementation into clinical practice.”

What’s slowing progress? There are a number of culprits, but a key contributor is the lack of high-quality data from racially, geographically and socioeconomically diverse populations. The majority of large AI training cohorts are available in Europe, where there are national registries and screening programs. But data sets in the U.S. tend to be both single-center and more difficult to access. In fact, researchers found in 2020 that cohorts from just three states—California, Massachusetts and New York—were disproportionately used to train clinical deep learning algorithms, “with little to no representation from the remaining 47 states.”

As these findings indicate, large portions of the country are “data deserts” in that their populations have little if any representation in the medical AI literature to date. As more and more practices come to rely on AI to aid in mammography interpretation, a lack of diverse data sets presents a serious risk to ensuring that breast AI reduces disparities rather than exacerbating them.

For Black Americans, progress cannot come too soon. Despite screening rates now comparable to white women, Black and African women experience substantially higher mortality rates from breast cancer. While many are optimistic that AI can help address this inequity, that cannot happen without adequate representation in training data. Until then, there is a very real risk that mammography AI will underperform on many populations.

This underperformance can result in both missed cancers and, perhaps, increased recalls and unnecessary diagnostic work-ups. This is to say nothing of the lost opportunity to identify imaging biomarkers in underrepresented populations that may help better screen women at greatest risk to alleviate their mortality burden. 


Decentralized AI training can help
 

Of course, one of the primary barriers to building large, diverse data sets in the U.S. is siloed data within healthcare organizations. Providers, as stewards of their patient’s health data, are justifiably reticent to let data leave the safety and security of their own picture archiving and communication systems (PACS). AI researchers, therefore, have a hard time centralizing diverse data in the volumes necessary to prevent bias in their models.

However, an emerging AI training technology called federated learning holds promise to change this paradigm. Federated learning makes it possible for researchers to access data where they reside, without the data ever having to travel between organizations. Instead, algorithms do the traveling—“visiting” data for training, then going back to researchers with weighted scores. Federated learning removes the need to share sensitive patient data across institutions and hence is considered secure by design.

This concept opens up a world of possibilities for collaborative research on biomedical data—if provider organizations open their doors to it. Imagine a multisite federated network of mammographic data (including both standard full-field digital mammograms and digital breast tomosynthesis) that is diverse across all requisite parameters: age, race, sex, socioeconomic status, geography, mammography device, mammographic findings, cancer diagnosis and cancer outcomes. With data set contributions from sites that assure representation of marginalized communities—including members of the America’s Essential Hospitals alliance and certified Critical Access Hospitals—this network can address the data desert problem and help ensure that AI is developed with data that matches the diversity of our patients.

By bringing their algorithms to the data via federated learning, AI developers can validate their models on large real-world data sets to identify bias and/or performance deficiencies. Once uncovered, teams can gain deeper access to the cohorts with poor model performance to improve their AI, building responsible technologies that perform well independent of patient characteristics.


Make your hospital’s data part of the solution


This network of diverse data is actually being assembled right now, with participation from users of both open-source and commercial data platforms. Further recruitment and onboarding are planned for sites that serve underrepresented communities, both urban and rural, which are generally absent from current AI cohorts.

In the near future, provider organizations that want to help improve AI will be able to simply publish a summary of their data to a cataloglike interface. AI researchers can browse this same catalog and contact data owners to access their data via federated learning or license them via other compliant data-sharing agreements.

This innovative model can go far in accelerating the timeline to achieving responsible AI through the identification and mitigation of bias. In this new landscape, one of the most helpful things hospitals can do is to recognize the value of their diverse data, and leverage opportunities to make it available for the greater good.

Aaron Mintz, M.D., is an assistant professor of radiology at Washington University School of Medicine and the medical director of Flywheel, a biomedical research data platform.