May 15, 2023 – No matter where you look, applications of machine learning in artificial intelligence are being harnessed to change the status quo. This is particularly true in healthcare, where technological advances are accelerating drug discovery and identification of potential new treatments.
But these developments do not come without red flags. They also put a magnifying glass on preventable differences in the burden of disease, injury and violence, and the chances of achieving optimal health, all of which disproportionately affect People of color and other disadvantaged communities.
The question now is whether AI applications will broaden or help narrow health disparities, especially when it comes to developing clinical algorithms that doctors use to detect and diagnose disease, predict outcome, and guide treatment strategies.
said Paul Yi, assistant professor of diagnostic radiology and nuclear medicine at the University of Maryland School of Medicine, and director of the University of Maryland’s Medical Intelligence Imaging Center (UM2ii).
“For medicine, getting a wrong diagnosis literally means life or death depending on the situation,” Ye said.
Yi is a co-author of a study published last month in the journal Nature medicine He and his colleagues attempted to discover whether medical imaging datasets used in data science competitions help or hinder the ability to recognize biases in AI models. These competitions feature computer scientists and clinicians collecting data from around the world, with teams competing to create the best clinical algorithms, many of which have been adopted into practice.
The researchers used a popular data science competition website called Kaggle for medical imaging competitions held between 2010 and 2022. They then evaluated the datasets to see if demographic variables had been reported. Finally, they looked at whether the competition included demographic-based performance as part of the evaluation criteria for the algorithms.
Of the 23 datasets included in the study, Yee said, “the majority — 61% — reported no demographic data at all.” Nine contests provided demographic data (mostly on age and gender), and one reported on race and ethnicity.
“None of these data science competitions, regardless of whether or not they reported on demographics, assessed these biases, namely, answer accuracy in male versus female, or white versus black versus Asian patients,” Ye said. subtext? “If we don’t have the demographics, we can’t measure the biases,” he explained.
Mathematical hygiene, checks and balances
said Bertalan Miesko, MD, PhD, director of the Institute of Medical Futurists in Budapest, Hungary.
One approach, which Meskó referred to as “algorithmic hygiene,” is similar to the approach a group of researchers at Emory University in Atlanta took when they created a racially diverse granular dataset — EMory BrEast Imaging dataset (EMBED) – Consists of 3.4 million images for mammographic screening and diagnosis of breast cancer. Forty-two percent of the 11,910 unique patients represented were self-reported African American women.
“The fact that our database is diverse is kind of a direct byproduct for our patients,” said Harry Trivedi, MD, assistant professor in the departments of radiology, imaging sciences, and biomedical informatics at Emory University School of Medicine and co-director. From the Health Innovation and Translational Informatics (HITI) Lab.
“So far, the vast majority of datasets used to develop a deep learning model do not include that demographic information,” Trivedi said. “But it’s been really important at EMBED and all future datasets we’re developing to have that information available because without it, it’s impossible to know how and when your model might be biased or the model you’re testing might be biased.”
He said, “You can’t turn a blind eye to it.”
Importantly, bias can be introduced at any stage of the AI development cycle, not just at its beginning.
“Developers can use statistical tests that allow them to detect whether the data used to train the algorithm differs significantly from the actual data they encounter in real-life settings,” said Misko. “This could indicate biases due to the training data.”
Another approach is “bias removal,” which helps eliminate differences between groups or individuals based on individual traits. Meskó referred to IBM’s Open Source AI Fairness 360 Toolkitwhich is a comprehensive set of metrics and algorithms that researchers and developers can access to use to reduce bias in their datasets and AI.
Checks and balances are similarly important. For example, this could include “algorithmic decisions being checked by humans and vice versa. That way, they can hold each other accountable and help mitigate bias,” Misko said..
Keep humans in the loop
Speaking of checks and balances, should patients be concerned that a machine is replacing a doctor’s judgment or making potentially dangerous decisions due to the loss of an important piece of data?
Trivedi mentioned that AI research guidelines are under development that specifically focus on rules to consider when testing and evaluating models, especially those that are open source. Also, the Food and Drug Administration and the Department of Health and Human Services are trying to regulate Algorithm development and validation With the aim of improving accuracy, transparency and fairness.
Like medicine itself, AI is not a one-size-fits-all solution, and perhaps checks and balances, consistent assessment, and a concerted effort to build diverse and comprehensive data sets can eventually address and help overcome pervasive health disparities.
At the same time, “I think we’re still a long way from completely removing the human element and not including clinicians in the process,” says Kelly Michelson, MD, MPH, and director of the Center for Bioethics and Medical Humanities at Northwestern University. Feinberg Medicine and attending physician at Ann & Robert H. Lurie Children’s Hospital in Chicago.
“There are really some amazing opportunities for AI to reduce disparities,” she said, also noting that AI is not just “this one big thing.”
“AI means a lot of different things in a lot of different places,” says Mickelson. “And the way it is used is different. It’s important to realize that the issues around bias and the impact on health disparities will be different depending on what kind of AI you’re talking about.”