Wood, Nadia G

Abstract Objectives Large language models (LLMs) offer potential in natural language processing tasks in healthcare. Due to the need for high accuracy, understanding their limitations is essential. The purpose of this study was to evaluate the performance of LLMs in classifying radiology reports for the presence of pulmonary embolism (PE) under various conditions, including different prompt designs and data perturbations. Materials and Methods In this retrospective, institutional review board approved study, we evaluated 3 Google’s LLMs including Gemini-1.5-Pro, Gemini-1.5-Flash-001, and Gemini-1.5-Flash-002, in classifying 11 999 pulmonary CT angiography radiology reports for PE. Ground truth labels were determined by concordance between a computer vision-based PE detection (CVPED) algorithm and multiple LLM runs under various configurations. Discrepancies between algorithms’ classifications were aggregated and manually reviewed. We evaluated the effects of prompt design, data perturbations, and repeated analyses across geographic cloud regions. Performance metrics were calculated. Results Of 11 999 reports, 1296 (10.8%) were PE-positive. Accuracy across LLMs ranged between 0.953 and 0.996. The highest recall rate for a prompt modified after a review of the misclassified cases (up to 0.997). Few-shot prompting improved recall (up to 0.99), while chain-of-thought generally degraded performance. Gemini-1.5-Flash-002 demonstrated the highest robustness against data perturbations. Geographic cloud region variability was minimal for Gemini-1.5+-Pro, while the Flash models showed stable performance. Discussion and Conclusion LLMs demonstrated high performance in classifying radiology reports, though results varied with prompt design and data quality. These findings underscore the need for systematic evaluation and validation of LLMs for clinical applications, particularly in high-stakes scenarios.

Abstract Yoga is increasing in popularity in the United States and across the globe. However, most yoga programs are provided outside the worksite; although many companies offer worksite wellness programs, at present there is limited documentation regarding the potential benefits of participating in a worksite yoga program. Therefore, the purpose of this project was to examine the potential effect of a worksite yoga program on self-acceptance, quality of life, and perceived stress. A prospective cohort pilot study that examined a structured worksite yoga program was designed and tailored to individuals new to yoga. The 8-week Yoga Foundations program was conducted at an academic medical center's worksite wellness center with 86 subjects. Outcome measures were the 36-item Self-Acceptance Scale; a six-item quality-of-life measure that assesses overall, social, mental, physical, emotional, and spiritual well-being; and the ten-item Perceived Stress Scale. Participants demonstrated significant improvement in their overall self-acceptance (p < 0.001), quality of life (p < 0.001), and perceived stress (p < 0.001) levels. They also highly rated the yoga instructors and the weekly format of the program. Participation in a Yoga Foundations program was associated with improvements in self-acceptance, quality of life and stress levels in worksite wellness center members. Future studies should use randomized designs and examine other wellness domains to learn more about the potential benefits of worksite yoga programs.