Citations

Retrieval-Augmented Generation (RAG) offers a promising strategy to harness large language models (LLMs) for delivering up-to-date, accurate clinical guidance while reducing physicians’ cognitive burden, yet its effectiveness hinges on query clarity and structure. We propose an adaptive Self-Query Retrieval (SQR) framework that integrates three refinement modules—PICOT (Population, Intervention, Comparison, Outcome, Time), SPICE (Setting, Population, Intervention, Comparison, Evaluation), and Iterative Query Refinement (IQR)—to automatically restructure and iteratively enhance clinical questions until they meet predefined retrieval-quality thresholds. Implemented on Gemini-1.0 Pro, we benchmarked SQR using thirty postoperative rhinoplasty queries, evaluating responses for accuracy and relevance on a three-point Likert scale and for retrieval quality via precision, recall, and F1 score; statistical significance was assessed by one-way ANOVA with Tukey post-hoc testing. The full SQR pipeline achieved 87% accuracy (Likert 2.4 ± 0.7) and 100% relevance (Likert 3.0 ± 0.0), significantly outperforming a non-refined RAG baseline (50% accuracy, 80% relevance; p < 0.01 and p = 0.03). Precision, recall, and F1 rose from 0.17, 0.39 and 0.24 to 0.53, 1.00, and 0.70, respectively, while PICOT-only and SPICE-only variants yielded intermediate improvements. These findings demonstrate that automated structuring and iterative enhancement of queries via SQR substantially elevate LLM-based clinical decision support, and its model-agnostic architecture enables rapid adaptation across specialties, data sources, and LLM platforms.

Abstract Background/ Objectives: Agentic AI represents a promising evolution of AI technology applied to healthcare, with systems increasingly capable of operating autonomously to achieve defined clinical goals. However, the literature lacks conceptual clarity between “AI agents” and “agentic AI”, and few studies have rigorously explored their clinical applications. Therefore, this study aims to conduct a novel systematic review addressing this gap by examining agentic AI systems in healthcare settings, characterizing their applications, features, outcomes, and limitations, and clarifying the conceptual distinctions between AI agents and agentic AI systems using predefined and objective criteria. Methods A comprehensive search was conducted across PubMed, Embase, Cochrane, Scopus, and Google Scholar on April 6th, 2025. Studies were included if they involved AI systems in healthcare settings that demonstrated the following agentic features: autonomous operation, goal-directed behavior, and initiating action. Data on the clinical tasks achieved by the agents, key findings, features, and limitations were collected from the included studies. Screening and extraction followed PRISMA guidelines, with Risk of Bias assessed using ROBINS-I and Cochrane's Risk of Bias tools. Results Of 984 retrieved records, seven studies met the inclusion criteria, spanning domains such as emergency medicine, oncology, radiology, and rehabilitation. Multi-agent architecture was frequently used to decompose and coordinate complex workflows. Among the included studies, the AIs showed high accuracy in diagnosing cancer patients, conducting treatment plans, sending alerts, coaching messages, analysing image data, and adapting to challenging experimental scenarios. While demonstrating potential for improved efficiency, task accuracy, and patient engagement, significant limitations were noted: narrow task scope, lack of physical agency, limited clinical validation, and barriers to integration into real-world healthcare systems. Only one system had been deployed in a patient-facing trial setting. Conclusion The current literature suggests an emerging role and application of Agentic AI, holding promise with the potential to revolutionize diagnostics, triage, treatment planning, and patient management. However, real-world implementation and evaluations in the literature are limited. Future research must address critical validation, regulation, ethics, and clinical integration challenges to realize their full potential. Clear operational definitions and frameworks for evaluating agency are essential to support safe and effective deployment of these systems.

Background: Data accessibility remains a significant barrier in healthcare AI due to privacy constraints and logistical challenges. Synthetic data, which mimics real patient information while remaining both realistic and non-identifiable, offers a promising solution. Large Language Models (LLMs) create new opportunities to generate high-fidelity clinical conversations between patients and physicians. However, the value of this synthetic data depends on careful evaluation of its realism, accuracy, and practical relevance. Objective: To assess the performance of four leading LLMs: ChatGPT 4.5, ChatGPT 4o, Claude 3.7 Sonnet, and Gemini Pro 2.5 in generating synthetic transcripts of patient–physician interactions in plastic surgery scenarios. Methods: Each model generated transcripts for ten plastic surgery scenarios. Transcripts were independently evaluated by three clinically trained raters using a seven-criterion rubric: Medical Accuracy, Realism, Persona Consistency, Fidelity, Empathy, Relevancy, and Usability. Raters were blinded to the model identity to reduce bias. Each was rated on a 5-point Likert scale, yielding 840 total evaluations. Descriptive statistics were computed, and a two-way repeated measures ANOVA was used to test for differences across models and metrics. In addition, transcripts were analyzed using automated linguistic and content-based metrics. Results: All models achieved strong performance, with mean ratings exceeding 4.5 across all criteria. Gemini 2.5 Pro received mean scores (5.00 ± 0.00) in Medical Accuracy, Realism, Persona Consistency, Relevancy, and Usability. Claude 3.7 Sonnet matched the scores in Persona Consistency and Relevancy and led in Empathy (4.96 ± 0.18). ChatGPT 4.5 also achieved perfect scores in Relevancy, with high scores in Empathy (4.93 ± 0.25) and Usability (4.96 ± 0.18). ChatGPT 4o demonstrated consistently strong but slightly lower performance across most dimensions. ANOVA revealed no statistically significant differences across models (F(3, 6) = 0.85, p = 0.52). Automated analysis showed substantial variation in transcript length, style, and content richness: Gemini 2.5 Pro generated the longest and most emotionally expressive dialogues, while ChatGPT 4o produced the shortest and most concise outputs. Conclusions: Leading LLMs can generate medically accurate, emotionally appropriate synthetic dialogues suitable for educational and research use. Despite high performance, demographic homogeneity in generated patients highlights the need for improved diversity and bias mitigation in model outputs. These findings support the cautious, context-aware integration of LLM-generated dialogues into medical training, simulation, and research.

The increasing integration of large language models (LLMs) into healthcare presents significant opportunities, but also critical risks related to patient safety, accuracy, and ethical alignment. Despite these concerns, no standardized framework exists for systematically evaluating and stress testing LLM behavior in clinical decision-making. The PIEE cycle—Planning and Preparation, Information Gathering and Prompt Generation, Execution, and Evaluation—is a structured red-teaming framework developed specifically to address artificial intelligence (AI) safety risks in healthcare decision-making. PIEE enables clinicians and informatics teams to simulate adversarial prompts, including jailbreaking, social engineering, and distractor attacks, to stress-test language models in real-world clinical scenarios. Model performance is evaluated using specific metrics such as true positive and false positive rates for detecting harmful content, hallucination rates measured through adapted TruthfulQA scoring, safety and reliability assessments, bias detection via adapted BBQ benchmarks, and ethical evaluation using structured Likert-based scoring rubrics. The framework is illustrated using examples from plastic surgery, but is adaptable across specialties, and is intended for use by all medical providers, regardless of their backgrounds or familiarity with artificial intelligence. While the framework is currently conceptual and validation is ongoing, PIEE provides a practical foundation for assessing the clinical reliability and ethical robustness of LLMs in medicine.

(1) Burn injuries demand multidisciplinary, evidence-based care, yet the extensive literature complicates timely decision making. Retrieval-augmented generation (RAG) synthesizes research while addressing inaccuracies in pretrained models. However, citation bias in sourcing for RAG often prioritizes highly cited studies, overlooking less-cited but valuable research. This study examines RAG’s performance in burn management, comparing citation levels to enhance evidence synthesis, reduce selection bias, and guide decisions. (2) Two burn management datasets were assembled: 30 highly cited (mean: 303) and 30 less-cited (mean: 21). The Gemini-1.0-Pro-002 RAG model addressed 30 questions, ranging from foundational principles to advanced surgical approaches. Responses were evaluated for accuracy (5-point scale), readability (Flesch–Kincaid metrics), and response time with Wilcoxon rank sum tests (p < 0.05). (3) RAG achieved comparable accuracy (4.6 vs. 4.2, p = 0.49), readability (Flesch Reading Ease: 42.8 vs. 46.5, p = 0.26; Grade Level: 9.9 vs. 9.5, p = 0.29), and response time (2.8 vs. 2.5 s, p = 0.39) for the highly and less-cited datasets. (4) Less-cited research performed similarly to highly cited sources. This equivalence broadens clinicians’ access to novel, diverse insights without sacrificing quality. As plastic surgery evolves, RAG’s inclusive approach fosters innovation, improves patient care, and reduces cognitive burden by integrating underutilized studies. Embracing RAG could propel the field toward dynamic, forward-thinking care.

e13154 Background: Triple-negative breast cancer (TNBC) is a less common but highly aggressive subtype of breast cancer with poor prognosis and limited therapeutic options. Despite advancements in oncology, significant survival disparities persist among racial, histologic, and socio-demographic groups. This study investigates these disparities and identifies factors contributing to unequal outcomes, providing actionable insights for improving care. Methods: Data on TNBC cases from 2000 to 2021 were extracted from the Surveillance, Epidemiology, and End Results (SEER) database, encompassing 17 registries and 76,265 patients. Patients were categorized by stage, marital status, age, race, and histology. Kaplan-Meier survival analysis was used to estimate median survival with 95% confidence intervals (CI), and the log-rank test assessed differences across groups. Cox proportional hazards regression determined the magnitude of disparities. A p-value < 0.05 was considered statistically significant. Results: Among the 76,265 patients analyzed, those with distant-stage disease had the poorest median survival (51 months; 95% CI: 46–60), compared to localized-stage patients (128 months; 95% CI: 127–129). The youngest cohort (15–19 years) exhibited the lowest survival (43 months), while individuals aged 80–84 had a median survival of 79 months (95% CI: 76–84). South Asians, particularly Pakistani individuals, had the shortest survival among racial groups (83 months). Single patients had lower survival (116 months) compared to married or divorced individuals. Histologically, median survival was 113 months (95% CI: 103–120) for epithelial neoplasms and 78 months for mucoepidermoid neoplasms. All observed differences were statistically significant (p < 0.05). Conclusions: Localized-stage TNBC, individuals aged 40–44, and patients from White or Black racial groups exhibited better survival outcomes, as did those with cystic, mucinous, or serous histologies compared to epithelial neoplasms. Addressing these disparities through tailored treatment strategies and improved access to care is critical. Limitations of this study include the lack of treatment-level data and the potential influence of unmeasured confounders. Stage Marital Status Age Group at Diagnosis Distant Localized Single Divorced 15-19 40-44 80-84 Median Survival(LCL-UCL) 51(46-60) 128(127-129) 116(114-118) 127(124-129) 43(14-NA) 132(130-135) 79(76-84) Neoplasm Histology Race Epithelial Muco-epidermoid Cystic, Mucinous& Serous Pakistani OtherAsian Black White Median Survival(LCL-UCL) 113 (103-120) 78(63-NA) 142(123-NA) 83 (63-NA) 97(91-108) 123 (121-125) 126 (125-127) LCL= Lower Confidence Limit, UCL= Upper Confidence Limit.

e13637 Background: Large language models (LLMs) like GPT-4 (OpenAI) and Claude Opus (Anthropic) showed high accuracy in medical multiple-choice exams, but data on their oncology-specific clinical reasoning and performance is limited. This study evaluates their accuracy and clinical reasoning on breast oncology multiple-choice questions (MCQs) from the American Society of Clinical Oncology (ASCO) question bank. Methods: Using OpenAI and Anthropic Application Programming Interface (APIs), questions were tested without additional prompts under consistent settings (GPT-4 and Claude Opus; Temperature = 0, Tokens = Max). Each question was tested three times to assess precision. Then, Chain-of-thought (COT) prompting was applied to promote LLMs stepwise reasoning to increase their accuracy. Accuracy before and after COT prompting was compared. Incorrect responses were reviewed by board-certified medical oncologists speclized in breast cancer, who scored reasoning clarity, bias, and clinical relevance. Qualitative feedback was descriptively analyzed. Results: A total of 273 breast oncology MCQs were evaluated across the two LLMs. GPT-4 achieved an initial accuracy of 81.3% (222/273; 95% CI: 76.3%-85.5%), compared to Claude Opus, which achieved an accuracy of 79.5% (217/273; 95% CI: 74.3%-83.9%). The Chi-squared test for difference between the models before chain-of-thought (COT) prompting yielded a p-value of 0.59, indicating no statistically significant difference in accuracy between GPT-4 and Claude Opus prior to COT prompting. After COT prompting, the performance of the two models diverged significantly. GPT-4 saw a net decline in accuracy, decreasing by a net of 1 correct answer, resulting in an overall accuracy of 80.95% (221/273). In contrast, Claude Opus experienced a notable improvement, with a net of 19 additional correct answers, leading to an accuracy of 86.4% (236/273). A statistical analysis using a Chi-squared test revealed a difference in accuracy between the two models of 5.5% (p = 0.08), demonstrating that the improvement in accuracy for Claude Opus after COT prompting was borderline statistically significant compared to GPT-4. Thematic analysis of oncologists' feedback revealed that the most common reasons for incorrect answers were reliance on outdated guidelines, misinterpretation of clinical trial data, and failure to consider multidisciplinary or patient-specific approaches in clinical decision-making. Conclusions: Although AI models can achieve high scores on multiple-choice exams, they still require human supervision. These models rely on potentially outdated training data and lack the ability to individualize patient care or apply data from clinical trials. especially in unique or unconventional/non textbook scenarios.

e16237 Background: Alcohol use (AU) and Liver cell carcinoma (LCC) are significant contributors to adult mortality worldwide. Examining trends and disparities in AU and LCC-related deaths is essential for understanding public health challenges and formulating targeted interventions. This study investigates national mortality patterns related to AU and LCC in the United States from 1999 to 2023, emphasizing demographic, geographic, and temporal disparities. Methods: This descriptive analysis used the CDC WONDER database (1999–2023) to identify AU-related deaths (ICD-10 codes E24.4, F10, G32.2, G62.1, G72.1, I42.6, K70, K85.2, K86.0, R78.0, T51, X65, Y15) and LCC (ICD-10 code C22.0) in adults aged 25 and older. Age-adjusted mortality rates (AAMRs) per 1,000,000 individuals were calculated, stratified by gender, race/ethnicity, census region, urban-rural classification, and state. Trends were analyzed using Joinpoint regression to compute annual percent changes (APCs) and 95% confidence intervals (CIs). Statistical significance was set at p < 0.05, and analyses adhered to STROBE guidelines. Results: From 1999 to 2023, a total of 21,007 deaths were attributed to LCC and AU (Men: 18,836 Women: 2,171). AAMRs showed a stable trend from 1999 to 2003 (APC: 0.0; 95% CI: -8.7 to 4.9) but increased significantly from 2003 to 2019 (APC: 6.1; 95% CI: 5.7 to 7.4), stabilizing from 2019 to 2023 (APC: -0.6; 95% CI: -3.2 to 1.3). Men's AAMRs decreased from 1999 to 2003 (APC: -1.1; 95% CI: -10.4 to 3.4), then rose sharply from 2003 to 2019 (APC: 6.1; 95% CI: 5.6 to 7.1). For women, AAMRs increased significantly, particularly between 2013 and 2017 (APC: 12.4; 95% CI: -9.5 to 25.1). Non-Hispanic Black and White populations experienced notable increases in AAMRs, while Hispanic people showed an initial decline followed by an increase post-2004 (APC: 4.7; 95% CI: 3.7 to 7.9). The West had the highest regional AAMR (5.3; 95% CI: 5.2 to 5.5), while the Northeast recorded the lowest (2.1; 95% CI: 2.0 to 2.2). New Mexico had the highest state AAMR (8.4; 95% CI: 7.4 to 9.4), and Mississippi the lowest (1.2; 95% CI: 0.9 to 1.5). AAMRs rose more sharply in non-metropolitan areas from 1999-2020 (APC: 6.6; 95% CI: 5.9 to 7.8) compared to metropolitan regions from 2004 to 2020 (APC: 5.4; 95% CI: 4.9 to 6.5). Conclusions: AU and LCC-related mortality rates have increased significantly since 2003, with distinct gender, racial, and geographic disparities. Men and rural populations faced higher mortality increases, with notable variations across states and regions. These findings underscore the need for focused public health efforts to mitigate these disparities and address the growing burden of LCC and AU-related mortality.