After consulting with a machine learning tool, providers were able to more accurately determine whether patients had a complete response to chemotherapy.
In a small but multi-institutional study, an artificial intelligence-based system improved providers’ assessments of whether patients with bladder cancer had a complete response to chemotherapy before a radical cystectomy (bladder removal surgery).
Yet the researchers caution that AI isn’t a replacement for human expertise and that their tool shouldn’t be used.
“If you use the tool smartly, it can help you,” said Lubomir Hadjiyski, Ph.D. professor of radiology at the University of Michigan Medical School and the study's senior author.
Image credit: Justine Ross/Michigan Medicine
When patients develop bladder cancer, surgeons often remove the entire bladder to keep cancer from returning or spreading to other organs or areas. However, more evidence is building that surgery may not be necessary if a patient has zero evidence of disease after chemotherapy.
However, it’s difficult to determine whether the lesion left after treatment is simply tissue that’s become necrotic or scarred due to treatment or whether cancer remains. The researchers wondered if AI could help.
“The big question was when you have such an artificial device next to you, how is it going to affect the physician?” Hadjiyski said. “Is it going to help? Is it going to confuse them? Is it going to raise their performance, or will they simply ignore it?”
Fourteen physicians from different specialties – including radiology, urology, and oncology – and two fellows and a medical student looked at pre-and post-treatment scans of 157 bladder tumors. The providers gave ratings for three measures that assessed the level of response to chemotherapy and a recommendation for the next treatment to be done for each patient (radiation or surgery).
Then the providers looked at a score calculated by the computer. Lower scores indicated a lower likelihood of complete response to chemo and vice versa for higher scores. The providers could revise their ratings or leave them unchanged. Their final ratings were compared against samples of the tumors taken during their bladder removal surgeries to gauge accuracy.
Across different specialties and experience levels, providers saw improvements in their assessments with the AI system. Those with less experience had even more gains, so much so that they were able to make diagnoses at the same level as the experienced participants.
“That was the distinct part of that study that showed interesting observations about the audience,” Hadjiyski said.
The tool helped providers from academic institutions more than those that worked at health centers focused solely on clinical care.
The study is part of an NIH-funded project, led by Hadjiyski and Ajjai Alva, M.D., an associate professor of internal medicine at U-M, to develop and evaluate biomarker-based tools for treatment response decision support of bladder cancer.
Over the course of more than two decades of conducting AI-based studies to assess different types of cancer and their treatment response, Hadjiyski says he’s observed that machine learning tools can be useful as a second opinion to assist physicians in decision making, but they can also make mistakes.
“One interesting thing that we figured out is that the computer makes mistakes on a different subset of cases than a radiologist would,” he added. “Which means that if the tool is used correctly, it gives a chance to improve but not replace the physician’s judgment.”