The headline you have been handed this year is that artificial intelligence now reads a scan better than the specialist who trained for it. That headline is true. It is also incomplete in a way that matters, because the property that makes these models good at the reading is the same property that makes them fail quietly for some patients and not others. You cannot keep the first half and discard the second. They are the same fact seen from two sides.
The cancer found a year early
Start with the wins, because they are real and I do not want to talk you out of them. A computational pathology model called Virchow, trained on 1.5 million stained tissue slides, built a pan-cancer detector that reached a specimen-level AUC of 0.95 across seventeen cancer types, including seven rare ones (Nature Medicine, 2024). AUC is a ranking measure, not an accuracy, and 0.95 is genuinely strong for a single model spanning that many tissues. It is not, I will note, the "above 97 percent" that traveled through the press. The honest number is 0.95, which is quieter and still impressive. Ask of any such claim the only question worth asking: how big is it, really, and against what.
The one that should get your attention is the pancreas. A Mayo Clinic model called REDMOD was run over nearly 1,500 abdominal CT scans, including scans from people later found to have pancreatic cancer, scans that had been read as normal at the time. By Mayo's own account of the study (published in Gut, 2026), it flagged 73 in every 100 of those pre-diagnostic cancers, at a median of about sixteen months before the clinical diagnosis, against 39 in 100 for specialists reading the same images unaided. On scans taken more than two years out the gap widened, 68 in 100 versus 23. Pancreatic cancer is almost always found late: more than 85 in every 100 cases are diagnosed after the disease has already spread, and fewer than 15 in 100 patients are alive five years on. A tool that sees it on a normal-looking pancreas is not a small thing.
I will hold one caveat here, because it is the whole discipline of reading a study. Finding a cancer sixteen months earlier is not the same as saving a life sixteen months' worth. Detect something earlier and you automatically get to count more months of "survival" from diagnosis even if the patient dies on the exact same day they always would have. That is lead-time bias, and it is why the same Mayo group has moved to a prospective trial, AI-PACED, to see whether earlier flags actually change outcomes or just change the clock. The senior author, Ajit Goenka, said in Mayo's announcement of the study that the model "can identify the signature of cancer from a normal-appearing pancreas." I believe him. Whether that signature, found early, buys survival is a different sentence, and it has not been written yet.
Cleared by the thousand, used by few
The scale is the part people underrate. The FDA's own running list of authorized AI-enabled devices had reached 1,524 by the end of March 2026, and 1,163 of them, about three in four, are for radiology (that dated tally is The Imaging Wire's reading of the list). Two caveats keep the number honest. Around 96 in 100 of those devices cleared through the 510(k) pathway, which asks whether a device is similar to one already on sale, not whether it improves outcomes. And clearance is not use: only about 30 percent of radiologists report using any AI at all. A cleared model sitting unused helps no one.
Where the scale does reach patients, the eye is the clearest case. Autonomous systems now screen for diabetic retinopathy in a primary-care office with no ophthalmologist in the building, and roughly half of people with diabetes skip their annual eye check, so a camera a nurse can run is a way to reach the people the system misses. The evidence behind these tools is not all one grade, and it matters which is which. The first such system the FDA cleared, IDx-DR, posted 87.2 percent sensitivity in its peer-reviewed pivotal trial (npj Digital Medicine, 2018); a newer system, AEYE-DS, reports around 93 percent, but that number is the manufacturer's own reported figure, not an independent trial. Take the two accordingly.
The signal no radiologist can see
Now the other side of the same fact. In 2022 a team led by Judy Wawira Gichoya showed that a model can predict a patient's self-reported race from a chest X-ray, a mammogram, a neck radiograph, a chest CT (Lancet Digital Health, 2022). No radiologist can do this. There is no known feature on the film that encodes race to a human eye. The models did it anyway, and kept doing it when the images were degraded, or cropped to one-ninth their size, or blurred until they barely read as X-rays. Nobody has been able to say what the machine is looking at.
That would be a curiosity if it stopped there. It does not. The problem is that a model reaching for a signal we cannot see may be using that signal as a shortcut for the diagnosis, and shortcuts break unevenly. A 2024 study in Nature Medicine, by Yuzhe Yang, Gichoya, Marzyeh Ghassemi and colleagues, confirmed that medical imaging models do lean on these demographic shortcuts in disease classification, and that the ones leaning hardest are the least fair across subgroups. Ghassemi, describing the work to MIT News, put the asymmetry plainly: "Many popular machine learning models have superhuman demographic prediction capacity; radiologists cannot detect self-reported race from a chest X-ray."
Black women, underdiagnosed worst of all
Put a number on the cost, because the number is the argument. A 2025 analysis in Science Advances took CheXzero, a model that reads chest X-rays about as well as radiologists do, and measured who it failed. The model identified a patient's race at an AUC of about 0.78, well above chance; the radiologists it was compared against did barely better than a coin toss. And it underdiagnosed the groups you would fear it would, missing real disease more often in women, in younger patients, in Black patients, with the widest gaps at the intersections, Black women worst of all. This held across all five datasets the authors tested, including one from Spain and one from Vietnam, so it is not one hospital's quirk. Some of the signal is not even biology: image-acquisition settings, the scanner and its processing, carry demographic traces, and controlling for them narrows the bias (Nature Communications, 2024), which tells you part of what the model "learned" was our own uneven data.
There is a real counter-argument and I will give it its full weight. These shortcuts can be engineered out, the argument goes: train the model to stop encoding demographics and the fairness gap closes. The same Nature Medicine team tested exactly that, and the honest result is a trap. Scrubbing the shortcuts did make the models fairer on the data they were tuned on, but that fairness often failed to survive a move to a new hospital, and the models that leaned least on demographics to begin with were frequently the ones that generalized most fairly. So the fix is real, and it is local. It buys you fairness on your own dataset and does not promise it on the next one, and none of it tells you what the machine was reading in the first place.
So hold both. A model that finds a pancreatic tumor a year before your doctor can is running on the same capacity as a model that decides, on grounds no one can name, that your chest is healthy because of who you are. The averages will look excellent either way, because an average of a group is exactly the place a subgroup goes to hide. If you write code you already know the shape of this: a system optimized hard on a target finds whatever correlates with that target in the training data, invited or not, and it does not tell you which. Medicine is only now learning what that sentence costs.