Kusmirek, Joanna E.

Abstract Objectives Large language models (LLMs) offer potential in natural language processing tasks in healthcare. Due to the need for high accuracy, understanding their limitations is essential. The purpose of this study was to evaluate the performance of LLMs in classifying radiology reports for the presence of pulmonary embolism (PE) under various conditions, including different prompt designs and data perturbations. Materials and Methods In this retrospective, institutional review board approved study, we evaluated 3 Google’s LLMs including Gemini-1.5-Pro, Gemini-1.5-Flash-001, and Gemini-1.5-Flash-002, in classifying 11 999 pulmonary CT angiography radiology reports for PE. Ground truth labels were determined by concordance between a computer vision-based PE detection (CVPED) algorithm and multiple LLM runs under various configurations. Discrepancies between algorithms’ classifications were aggregated and manually reviewed. We evaluated the effects of prompt design, data perturbations, and repeated analyses across geographic cloud regions. Performance metrics were calculated. Results Of 11 999 reports, 1296 (10.8%) were PE-positive. Accuracy across LLMs ranged between 0.953 and 0.996. The highest recall rate for a prompt modified after a review of the misclassified cases (up to 0.997). Few-shot prompting improved recall (up to 0.99), while chain-of-thought generally degraded performance. Gemini-1.5-Flash-002 demonstrated the highest robustness against data perturbations. Geographic cloud region variability was minimal for Gemini-1.5+-Pro, while the Flash models showed stable performance. Discussion and Conclusion LLMs demonstrated high performance in classifying radiology reports, though results varied with prompt design and data quality. These findings underscore the need for systematic evaluation and validation of LLMs for clinical applications, particularly in high-stakes scenarios.

The radiotracer [ 68 Ga]Ga-NOTA-CTP was evaluated in healthy and diseased CD1 mice with myocardial infarction following ligation of the left anterior descending artery.

Abstract PurposeOur study aimed to use chest CT-angiogram (CTA) to assess if right ventricular (RV) dilation, quantified as an increased RV:LV (left ventricle) ratio, is associated with adverse outcomes in the novel coronavirus (COVID-19) infection.MethodsWe reviewed clinical, laboratory, and chest CTA findings in COVID-19 patients (n=100), and two control groups: normal subjects (n=10) and subjects with organizing pneumonia (n=10). On a chest CTA, we measured basal dimensions of the RV and LV in a focused 4-chamber view; and dimensions of pulmonary artery (PA) and aorta (AO) at the PA bifurcation level. ResultsAmong the COVID-19 cohort, the mean age (±SD) was 55.1±14.9 years and 55% were female. A higher RV:LV ratio was correlated with adverse outcomes, defined as ICU admission, intubation, or death. In patients with adverse outcomes, the RV:LV ratio was 1.06±0.10, vs 0.95±0.15 in patients without adverse outcomes. Among the adverse outcomes group, compared to the control subjects with organizing pneumonia, the lung parenchymal damage was lower (22.6±9.0 vs 32.7±6.6), yet the RV:LV ratio was higher (1.06±0.14 vs 0.89±0.07). In ROC analysis, RV:LV ratio had an AUC= 0.707 with an optimal cut-off of RV:LV 1.1 as a predictor of adverse outcomes. In a validation cohort (n=25), an RV:LV ≥1.1 as a cut-off predicted adverse outcomes with an odds ratio of 76:1.ConclusionIn COVID-19 patients, RV:LV ratio ≥1.1 on CTA-chest is correlated with adverse outcomes. RV dilation in COVID-19 is out of proportion to parenchymal lung damage, pointing towards a vascular and/or thrombotic injury in the lungs.