Radiologist Plus AI vs. Algorithm Alone: Shanghai Settles the Debate, FDA Pays for Evidence

02 · FDA Regulations

FDA Pours Millions Into Verifying AI Reports [2]

If the Shanghai results show that radiologist oversight pays off, the next piece of news shows how regulators want to institutionalize that oversight. The US FDA has awarded Cognita Imaging $1.29 million under an 18-month research contract to develop a method for evaluating the quality of radiology reports generated by large language models.

The heart of the project is an "LLM-as-jury" approach — rather than relying on a single model's assessment, several independent LLMs evaluate the same report, aiming to produce a more reliable, averaged verdict on its quality. Once the methodology is developed and tested, the team plans to apply it at scale — analyzing roughly one million patient studies drawn from a diverse US population, spanning different facilities, imaging equipment types, and clinical conditions.

Crucially, humans aren't disappearing from the process at all — radiologists will review cases where the models' assessments diverge significantly, in order to distinguish errors in the AI report itself from errors in the evaluation mechanism. This shows that even a regulator building automated quality-control tools treats human oversight as an indispensable piece of the puzzle — exactly as the Shanghai trial suggested.