Forte, Antonio J.

Background: Generative AI and synthetic media have enabled realistic human Embodied Conversational Agents (ECAs) or avatars. A subset of this technology replicates faces and voices to create realistic likenesses. When combined with avatars, these methods enable the creation of “digital twins” of physicians, offering patients scalable, 24/7 clinical communication outside the immediate clinical environment. This study evaluated surgical patient perceptions of an AI-generated surgeon avatar for postoperative education. Methods: We conducted a pilot feasibility study with 30 plastic surgery patients at Mayo Clinic, USA (July–August 2025). A bespoke interactive surgeon avatar was developed in Python using the HeyGen IV model to reproduce the surgeon’s likeness. Patients interacted with the avatar through natural voice queries, which were mapped to predetermined, pre-recorded video responses covering ten common postoperative topics. Patient perceptions were assessed using validated scales of usability, engagement, trust, eeriness, and realism, supplemented by qualitative feedback. Results: The avatar system reliably answered 297 of 300 patient queries (99%). Usability was excellent (mean System Usability Scale score = 87.7 ± 11.5) and engagement high (mean 4.27 ± 0.23). Trust was the highest-rated domain, with all participants (100%) finding the avatar trustworthy and its information believable. Eeriness was minimal (mean = 1.57 ± 0.48), and 96.7% found the avatar visually pleasing. Most participants (86.6%) recognized the avatar as their surgeon, although many still identified it as artificial; voice resemblance was less convincing (70%). Interestingly, participants with prior exposure to deepfakes demonstrated consistently higher acceptance, rating usability, trust, and engagement 5–10% higher than those without prior exposure. Qualitative feedback highlighted clarity, efficiency, and convenience, while noting limitations in realism and conversational scope. Conclusions: The AI-generated physician avatar achieved high patient acceptance without triggering uncanny valley effects. Transparency about the synthetic nature of the technology enhanced, rather than diminished, trust. Familiarity with the physician and institutional credibility likely played a key role in the high trust scores observed. When implemented transparently and with appropriate safeguards, synthetic physician avatars may offer a scalable solution for postoperative education while preserving trust in clinical relationships.

Abstract Background Outcomes of patients with acute pyelonephritis (AP) treated in a hospital-at-home setting have not been comprehensively evaluated in the United States. Methods We performed a multicenter, retrospective cohort study of adults diagnosed with and managed for AP in Mayo Clinic's Advanced Care at Home (ACH) program between July 2020 and January 2025. Collected data included demographics, Charlson Comorbidity Index (CCI), genitourinary comorbidities, severity of illness (SOI), and risk of mortality (ROM) scores, as well as pyelonephritis-related complications. Outcomes included length of stay (LOS), escalation of care, and 30-day postdischarge emergency department (ED) visits, readmissions, and mortality. Results A total of 165 patients met inclusion criteria. Median age was 67 years; SOI scores were moderate in 33.3%, major in 52.1%, and extreme in 8.5%. ROM scores were moderate in 30.3%, major in 38.2%, and extreme in 6.7%. Median CCI score was 5, and all patients had preexisting genitourinary conditions. On admission, 30.9% met sepsis criteria, acute kidney injury was present in 47.3%, and bacteremia developed in 33.3%. Median LOS in the ACH program was 3.1 days. Only 4.8% required escalation to a brick-and-mortar hospital. Readmission occurred in 17.0%, and 4.8% had ED visits. No in-program deaths occurred. Conclusions This multicenter retrospective study shows that AP, including cases with high illness severity and complex comorbidities, can be managed safely and effectively in a hospital-at-home setting with careful patient selection and robust infrastructure to support timely escalation when needed.

Background: During postoperative recovery, patients and their caregivers often lack crucial information, leading to numerous repetitive inquiries that burden healthcare providers. Traditional discharge materials, including paper handouts and patient portals, are often static, overwhelming, or underutilized, leading to patient overwhelm and contributing to unnecessary ER visits and overall healthcare overutilization. Conversational chatbots offer a solution, but Natural Language Processing (NLP) systems are often inflexible and limited in understanding, while powerful Large Language Models (LLMs) are prone to generating “hallucinations”. Objective: To combine the deterministic framework of traditional NLP with the probabilistic capabilities of LLMs, we developed the AI Virtual Assistant (AIVA) Platform. This system utilizes a retrieval-augmented generation (RAG) architecture, integrating Gemini 2.0 Flash with a medically verified knowledge base via Google Vertex AI, to safely deliver dynamic, patient-facing postoperative guidance grounded in validated clinical content. Methods: The AIVA Platform was evaluated through 750 simulated patient interactions derived from 250 unique postoperative queries across 20 high-frequency recovery domains. Three blinded physician reviewers assessed formal system performance, evaluating classification metrics (accuracy, precision, recall, F1-score), relevance (SSI Index), completeness, and consistency (5-point Likert scale). Safety guardrails were tested with 120 out-of-scope queries and 30 emergency escalation scenarios. Additionally, groundedness, fluency, and readability were assessed using automated LLM metrics. Results: The system achieved 98.4% classification accuracy (precision 1.0, recall 0.98, F1-score 0.9899). Physician reviews showed high completeness (4.83/5), consistency (4.49/5), and relevance (SSI Index 2.68/3). Safety guardrails successfully identified 100% of out-of-scope and escalation scenarios. Groundedness evaluations demonstrated strong context precision (0.951), recall (0.910), and faithfulness (0.956), with 95.6% verification agreement. While fluency and semantic alignment were high (BERTScore F1 0.9013, ROUGE-1 0.8377), readability was 11th-grade level (Flesch–Kincaid 46.34). Conclusion: The simulated testing demonstrated strong technical accuracy, safety, and clinical relevance in simulated postoperative care. Its architecture effectively balances flexibility and safety, addressing key limitations of standalone NLP and LLMs. While readability remains a challenge, these findings establish a solid foundation, demonstrating readiness for clinical trials and real-world testing within surgical care pathways.

Retrieval-augmented generation (RAG) quality depends on how source documents are segmented before indexing; fixed-length chunks can split concepts or add noise, reducing precision. We evaluated whether proposition, semantic, and adaptive chunking improve accuracy and relevance for safer clinical decision support. Using a curated domain knowledge base with Gemini 1.0 Pro, we built four otherwise identical RAG pipelines that differed only in the chunking strategy: adaptive length, proposition, semantic, and a fixed token-dependent baseline. Thirty common postoperative rhinoplasty questions were submitted to each pipeline. Outcomes included medical accuracy and clinical relevance (3-point Likert scale) and retrieval precision, recall, and F1; group differences were tested with ANOVA and Tukey post hoc analyses. Adaptive chunking achieved the highest accuracy—87% (Likert 2.37 ± 0.72) versus baseline 50% (1.63 ± 0.72; p = 0.001)—and the highest relevance (93%, 2.90 ± 0.40). Retrieval metrics were strongest with adaptive (precision 0.50, recall 0.88, F1 0.64) versus baseline (0.17, 0.40, 0.24). Proposition and semantic strategies improved all metrics relative to baseline, though less than adaptive. Aligning chunks to logical topic boundaries yielded more accurate, relevant answers without modifying the language model, offering a model-agnostic, data-source-neutral lever to enhance the safety and utility of LLM-based clinical decision support.

Background: The incidence of lymphedema may be as high as 65% among breast cancer patients, varying according to the diagnostic method and locoregional treatment. Therefore, investigations of preventive methods are highly welcomed. Our goal was to conduct a systematic review of the literature about the use of axillary reverse mapping to prevent lymphedema during breast cancer treatment. We hypothesized that identification of arm-draining lymph nodes may decrease the incidence of lymphedema. Methods: On October 7, 2019, we conducted a systematic review of studies in PubMed, Cochrane Clinical Answers, and Cochrane Central Register of Controlled Trials, without time frame or language limitations, on the use of axillary reverse mapping to prevent lymphedema during breast cancer treatment. We excluded articles that investigated other uses of lymphoscintigraphy, such as lymphedema diagnosis or treatment evaluation. Results: Of 104 potential articles found in the literature, 5 studies fulfilled the eligibility criteria. A total of 501 patients were included. Reverse mapping was done with radiography or single-photon emission computed tomography/computed tomography. Moreover, the examination was applied with different treatments, such as axillary lymph node dissection or radiotherapy, allowing preservation of noncompromised lymph nodes. Axillary reverse mapping also had prognostic value and was used with sentinel lymph node biopsy to identify patients at higher risk for lymphedema. Only one study compared the incidence of lymphedema between patients who received standard care or reverse mapping, showing significant benefits of its use. Conclusions: Use of axillary reverse mapping to prevent lymphedema is feasible and seems to provide valuable information for breast cancer treatment. Future studies that compare reverse mapping and standard care are still necessary.

Background: Artificial Intelligence (AI)-driven Decision Support Systems (DSSs) promise improvements in diagnostic accuracy and clinical workflow efficiency, but their adoption is hindered by inadequate confidence calibration, limited transparency, and poor alignment with real-world decision processes, which limit clinician trust and lead to high override rates. Methods: We developed and validated a dynamic scoring framework to enhance trust in AI-generated diagnoses by integrating AI confidence scores, semantic similarity measures, and transparency weighting into the override decision process using 6689 cardiovascular cases from the MIMIC-III dataset. Override thresholds were calibrated and validated across varying transparency and confidence levels, with override rate as the primary acceptance measure. Results: The implementation of this framework reduced the override rate to 33.29%, with high-confidence predictions (90–99%) overridden at a rate of only 1.7%, and low-confidence predictions (70–79%) at a rate of 99.3%. Minimal transparency diagnoses had a 73.9% override rate compared to 49.3% for moderate transparency. Statistical analyses confirmed significant associations between confidence, transparency, and override rates (p < 0.001). Conclusions: These findings suggest that enhanced transparency and confidence calibration can substantially reduce override rates and promote clinician acceptance of AI diagnostics. Future work should focus on clinical validation to optimize patient safety, diagnostic accuracy, and efficiency.

Retrieval-Augmented Generation (RAG) offers a promising strategy to harness large language models (LLMs) for delivering up-to-date, accurate clinical guidance while reducing physicians’ cognitive burden, yet its effectiveness hinges on query clarity and structure. We propose an adaptive Self-Query Retrieval (SQR) framework that integrates three refinement modules—PICOT (Population, Intervention, Comparison, Outcome, Time), SPICE (Setting, Population, Intervention, Comparison, Evaluation), and Iterative Query Refinement (IQR)—to automatically restructure and iteratively enhance clinical questions until they meet predefined retrieval-quality thresholds. Implemented on Gemini-1.0 Pro, we benchmarked SQR using thirty postoperative rhinoplasty queries, evaluating responses for accuracy and relevance on a three-point Likert scale and for retrieval quality via precision, recall, and F1 score; statistical significance was assessed by one-way ANOVA with Tukey post-hoc testing. The full SQR pipeline achieved 87% accuracy (Likert 2.4 ± 0.7) and 100% relevance (Likert 3.0 ± 0.0), significantly outperforming a non-refined RAG baseline (50% accuracy, 80% relevance; p < 0.01 and p = 0.03). Precision, recall, and F1 rose from 0.17, 0.39 and 0.24 to 0.53, 1.00, and 0.70, respectively, while PICOT-only and SPICE-only variants yielded intermediate improvements. These findings demonstrate that automated structuring and iterative enhancement of queries via SQR substantially elevate LLM-based clinical decision support, and its model-agnostic architecture enables rapid adaptation across specialties, data sources, and LLM platforms.

Abstract Background/ Objectives: Agentic AI represents a promising evolution of AI technology applied to healthcare, with systems increasingly capable of operating autonomously to achieve defined clinical goals. However, the literature lacks conceptual clarity between “AI agents” and “agentic AI”, and few studies have rigorously explored their clinical applications. Therefore, this study aims to conduct a novel systematic review addressing this gap by examining agentic AI systems in healthcare settings, characterizing their applications, features, outcomes, and limitations, and clarifying the conceptual distinctions between AI agents and agentic AI systems using predefined and objective criteria. Methods A comprehensive search was conducted across PubMed, Embase, Cochrane, Scopus, and Google Scholar on April 6th, 2025. Studies were included if they involved AI systems in healthcare settings that demonstrated the following agentic features: autonomous operation, goal-directed behavior, and initiating action. Data on the clinical tasks achieved by the agents, key findings, features, and limitations were collected from the included studies. Screening and extraction followed PRISMA guidelines, with Risk of Bias assessed using ROBINS-I and Cochrane's Risk of Bias tools. Results Of 984 retrieved records, seven studies met the inclusion criteria, spanning domains such as emergency medicine, oncology, radiology, and rehabilitation. Multi-agent architecture was frequently used to decompose and coordinate complex workflows. Among the included studies, the AIs showed high accuracy in diagnosing cancer patients, conducting treatment plans, sending alerts, coaching messages, analysing image data, and adapting to challenging experimental scenarios. While demonstrating potential for improved efficiency, task accuracy, and patient engagement, significant limitations were noted: narrow task scope, lack of physical agency, limited clinical validation, and barriers to integration into real-world healthcare systems. Only one system had been deployed in a patient-facing trial setting. Conclusion The current literature suggests an emerging role and application of Agentic AI, holding promise with the potential to revolutionize diagnostics, triage, treatment planning, and patient management. However, real-world implementation and evaluations in the literature are limited. Future research must address critical validation, regulation, ethics, and clinical integration challenges to realize their full potential. Clear operational definitions and frameworks for evaluating agency are essential to support safe and effective deployment of these systems.

Background: Data accessibility remains a significant barrier in healthcare AI due to privacy constraints and logistical challenges. Synthetic data, which mimics real patient information while remaining both realistic and non-identifiable, offers a promising solution. Large Language Models (LLMs) create new opportunities to generate high-fidelity clinical conversations between patients and physicians. However, the value of this synthetic data depends on careful evaluation of its realism, accuracy, and practical relevance. Objective: To assess the performance of four leading LLMs: ChatGPT 4.5, ChatGPT 4o, Claude 3.7 Sonnet, and Gemini Pro 2.5 in generating synthetic transcripts of patient–physician interactions in plastic surgery scenarios. Methods: Each model generated transcripts for ten plastic surgery scenarios. Transcripts were independently evaluated by three clinically trained raters using a seven-criterion rubric: Medical Accuracy, Realism, Persona Consistency, Fidelity, Empathy, Relevancy, and Usability. Raters were blinded to the model identity to reduce bias. Each was rated on a 5-point Likert scale, yielding 840 total evaluations. Descriptive statistics were computed, and a two-way repeated measures ANOVA was used to test for differences across models and metrics. In addition, transcripts were analyzed using automated linguistic and content-based metrics. Results: All models achieved strong performance, with mean ratings exceeding 4.5 across all criteria. Gemini 2.5 Pro received mean scores (5.00 ± 0.00) in Medical Accuracy, Realism, Persona Consistency, Relevancy, and Usability. Claude 3.7 Sonnet matched the scores in Persona Consistency and Relevancy and led in Empathy (4.96 ± 0.18). ChatGPT 4.5 also achieved perfect scores in Relevancy, with high scores in Empathy (4.93 ± 0.25) and Usability (4.96 ± 0.18). ChatGPT 4o demonstrated consistently strong but slightly lower performance across most dimensions. ANOVA revealed no statistically significant differences across models (F(3, 6) = 0.85, p = 0.52). Automated analysis showed substantial variation in transcript length, style, and content richness: Gemini 2.5 Pro generated the longest and most emotionally expressive dialogues, while ChatGPT 4o produced the shortest and most concise outputs. Conclusions: Leading LLMs can generate medically accurate, emotionally appropriate synthetic dialogues suitable for educational and research use. Despite high performance, demographic homogeneity in generated patients highlights the need for improved diversity and bias mitigation in model outputs. These findings support the cautious, context-aware integration of LLM-generated dialogues into medical training, simulation, and research.

The increasing integration of large language models (LLMs) into healthcare presents significant opportunities, but also critical risks related to patient safety, accuracy, and ethical alignment. Despite these concerns, no standardized framework exists for systematically evaluating and stress testing LLM behavior in clinical decision-making. The PIEE cycle—Planning and Preparation, Information Gathering and Prompt Generation, Execution, and Evaluation—is a structured red-teaming framework developed specifically to address artificial intelligence (AI) safety risks in healthcare decision-making. PIEE enables clinicians and informatics teams to simulate adversarial prompts, including jailbreaking, social engineering, and distractor attacks, to stress-test language models in real-world clinical scenarios. Model performance is evaluated using specific metrics such as true positive and false positive rates for detecting harmful content, hallucination rates measured through adapted TruthfulQA scoring, safety and reliability assessments, bias detection via adapted BBQ benchmarks, and ethical evaluation using structured Likert-based scoring rubrics. The framework is illustrated using examples from plastic surgery, but is adaptable across specialties, and is intended for use by all medical providers, regardless of their backgrounds or familiarity with artificial intelligence. While the framework is currently conceptual and validation is ongoing, PIEE provides a practical foundation for assessing the clinical reliability and ethical robustness of LLMs in medicine.