AI System Outperformed Radiologist When Screening for Breast Cancer

A new study found an artificial intelligence (AI) system is capable of surpassing human experts in breast cancer prediction.

Published on January 1, 2020, this AI system outperformed all of the humans who read mammography screenings. 

This study’s results in the area under the receiver operating characteristic curve (AUC-ROC) for the AI system was greater than the AUC-ROC for the average radiologist, by an absolute margin of 11.5 percent.

This is an important finding since the current mammogram screening programs are affected by high rates of false positives and false negatives.

To assess this AI system’s performance in the clinical setting, these researchers curated a large representative dataset from the UK and a large enriched dataset from the USA.

They ran a simulation in which the AI system participated in the double-reading process that is used in the UK, and found that the AI system maintained non-inferior performance and reduced the workload of the 2nd mammography reader by 88 percent. 

The results showed an absolute reduction of 5.7 percent in the USA and 1.2 percent in the UK in false positives and 9.4 percent and 2.7 percent in false negatives.

This study’s results are important since breast cancer is the 2nd leading cause of death from cancer in women, but early detection and treatment can considerably improve outcomes. And the optimal use of the AI system within clinical workflows remains to be determined. 

The specificity advantage exhibited by the AI system suggests that it could help to reduce recall rates and unnecessary biopsies. 

The improvement in sensitivity exhibited in the US data shows that the AI system may be capable of detecting cancers earlier than the standard of care. In the USA, each mammogram is interpreted by a single radiologist.

An analysis of the localization performance of the AI system suggests it holds early promise for flagging suspicious regions for review by experts. Notably, the additional cancers identified by the AI system tended to be invasive, rather than in situ disease.

Most medical and governmental organizations around the world recommend breast cancer screening for all women starting between the ages of 40 and 50. In the USA and UK combined, over 42 million mammogram exams are performed each year.

The AI system used in this study was a collaboration with Google, DeepMind, Cancer Research UK Imperial Centre, Northwestern University and Royal Surrey County Hospital, and consisted of an ensemble of 3 deep learning models, each operating on a different level of analysis (individual lesions, individual breasts, and the full case). Each model produces a cancer risk score between 0 and 1 for the entire mammography case. 

The final prediction of the system was the mean of the predictions from the three independent models.

Google’s Chief Financial Officer Ruth Porat shared her optimism around potential technological breakthroughs in this area in a post during October 2019, reflecting on her personal experience with breast cancer.

In summary, these researchers said ‘This robust assessment of the AI system paves the way for clinical trials to improve the accuracy and efficiency of breast cancer screening.’

‘But getting there will require continued research, prospective clinical studies and regulatory approval to understand and prove how software systems inspired by this research could improve patient care.’

These researchers disclosed various industry relationships.

Breast cancer vaccine news published by Precision Vaccinations.