Prabha, Srinivagasam

Background: During postoperative recovery, patients and their caregivers often lack crucial information, leading to numerous repetitive inquiries that burden healthcare providers. Traditional discharge materials, including paper handouts and patient portals, are often static, overwhelming, or underutilized, leading to patient overwhelm and contributing to unnecessary ER visits and overall healthcare overutilization. Conversational chatbots offer a solution, but Natural Language Processing (NLP) systems are often inflexible and limited in understanding, while powerful Large Language Models (LLMs) are prone to generating “hallucinations”. Objective: To combine the deterministic framework of traditional NLP with the probabilistic capabilities of LLMs, we developed the AI Virtual Assistant (AIVA) Platform. This system utilizes a retrieval-augmented generation (RAG) architecture, integrating Gemini 2.0 Flash with a medically verified knowledge base via Google Vertex AI, to safely deliver dynamic, patient-facing postoperative guidance grounded in validated clinical content. Methods: The AIVA Platform was evaluated through 750 simulated patient interactions derived from 250 unique postoperative queries across 20 high-frequency recovery domains. Three blinded physician reviewers assessed formal system performance, evaluating classification metrics (accuracy, precision, recall, F1-score), relevance (SSI Index), completeness, and consistency (5-point Likert scale). Safety guardrails were tested with 120 out-of-scope queries and 30 emergency escalation scenarios. Additionally, groundedness, fluency, and readability were assessed using automated LLM metrics. Results: The system achieved 98.4% classification accuracy (precision 1.0, recall 0.98, F1-score 0.9899). Physician reviews showed high completeness (4.83/5), consistency (4.49/5), and relevance (SSI Index 2.68/3). Safety guardrails successfully identified 100% of out-of-scope and escalation scenarios. Groundedness evaluations demonstrated strong context precision (0.951), recall (0.910), and faithfulness (0.956), with 95.6% verification agreement. While fluency and semantic alignment were high (BERTScore F1 0.9013, ROUGE-1 0.8377), readability was 11th-grade level (Flesch–Kincaid 46.34). Conclusion: The simulated testing demonstrated strong technical accuracy, safety, and clinical relevance in simulated postoperative care. Its architecture effectively balances flexibility and safety, addressing key limitations of standalone NLP and LLMs. While readability remains a challenge, these findings establish a solid foundation, demonstrating readiness for clinical trials and real-world testing within surgical care pathways.

Retrieval-augmented generation (RAG) quality depends on how source documents are segmented before indexing; fixed-length chunks can split concepts or add noise, reducing precision. We evaluated whether proposition, semantic, and adaptive chunking improve accuracy and relevance for safer clinical decision support. Using a curated domain knowledge base with Gemini 1.0 Pro, we built four otherwise identical RAG pipelines that differed only in the chunking strategy: adaptive length, proposition, semantic, and a fixed token-dependent baseline. Thirty common postoperative rhinoplasty questions were submitted to each pipeline. Outcomes included medical accuracy and clinical relevance (3-point Likert scale) and retrieval precision, recall, and F1; group differences were tested with ANOVA and Tukey post hoc analyses. Adaptive chunking achieved the highest accuracy—87% (Likert 2.37 ± 0.72) versus baseline 50% (1.63 ± 0.72; p = 0.001)—and the highest relevance (93%, 2.90 ± 0.40). Retrieval metrics were strongest with adaptive (precision 0.50, recall 0.88, F1 0.64) versus baseline (0.17, 0.40, 0.24). Proposition and semantic strategies improved all metrics relative to baseline, though less than adaptive. Aligning chunks to logical topic boundaries yielded more accurate, relevant answers without modifying the language model, offering a model-agnostic, data-source-neutral lever to enhance the safety and utility of LLM-based clinical decision support.

Background: Artificial Intelligence (AI)-driven Decision Support Systems (DSSs) promise improvements in diagnostic accuracy and clinical workflow efficiency, but their adoption is hindered by inadequate confidence calibration, limited transparency, and poor alignment with real-world decision processes, which limit clinician trust and lead to high override rates. Methods: We developed and validated a dynamic scoring framework to enhance trust in AI-generated diagnoses by integrating AI confidence scores, semantic similarity measures, and transparency weighting into the override decision process using 6689 cardiovascular cases from the MIMIC-III dataset. Override thresholds were calibrated and validated across varying transparency and confidence levels, with override rate as the primary acceptance measure. Results: The implementation of this framework reduced the override rate to 33.29%, with high-confidence predictions (90–99%) overridden at a rate of only 1.7%, and low-confidence predictions (70–79%) at a rate of 99.3%. Minimal transparency diagnoses had a 73.9% override rate compared to 49.3% for moderate transparency. Statistical analyses confirmed significant associations between confidence, transparency, and override rates (p < 0.001). Conclusions: These findings suggest that enhanced transparency and confidence calibration can substantially reduce override rates and promote clinician acceptance of AI diagnostics. Future work should focus on clinical validation to optimize patient safety, diagnostic accuracy, and efficiency.

Retrieval-Augmented Generation (RAG) offers a promising strategy to harness large language models (LLMs) for delivering up-to-date, accurate clinical guidance while reducing physicians’ cognitive burden, yet its effectiveness hinges on query clarity and structure. We propose an adaptive Self-Query Retrieval (SQR) framework that integrates three refinement modules—PICOT (Population, Intervention, Comparison, Outcome, Time), SPICE (Setting, Population, Intervention, Comparison, Evaluation), and Iterative Query Refinement (IQR)—to automatically restructure and iteratively enhance clinical questions until they meet predefined retrieval-quality thresholds. Implemented on Gemini-1.0 Pro, we benchmarked SQR using thirty postoperative rhinoplasty queries, evaluating responses for accuracy and relevance on a three-point Likert scale and for retrieval quality via precision, recall, and F1 score; statistical significance was assessed by one-way ANOVA with Tukey post-hoc testing. The full SQR pipeline achieved 87% accuracy (Likert 2.4 ± 0.7) and 100% relevance (Likert 3.0 ± 0.0), significantly outperforming a non-refined RAG baseline (50% accuracy, 80% relevance; p < 0.01 and p = 0.03). Precision, recall, and F1 rose from 0.17, 0.39 and 0.24 to 0.53, 1.00, and 0.70, respectively, while PICOT-only and SPICE-only variants yielded intermediate improvements. These findings demonstrate that automated structuring and iterative enhancement of queries via SQR substantially elevate LLM-based clinical decision support, and its model-agnostic architecture enables rapid adaptation across specialties, data sources, and LLM platforms.

The increasing integration of large language models (LLMs) into healthcare presents significant opportunities, but also critical risks related to patient safety, accuracy, and ethical alignment. Despite these concerns, no standardized framework exists for systematically evaluating and stress testing LLM behavior in clinical decision-making. The PIEE cycle—Planning and Preparation, Information Gathering and Prompt Generation, Execution, and Evaluation—is a structured red-teaming framework developed specifically to address artificial intelligence (AI) safety risks in healthcare decision-making. PIEE enables clinicians and informatics teams to simulate adversarial prompts, including jailbreaking, social engineering, and distractor attacks, to stress-test language models in real-world clinical scenarios. Model performance is evaluated using specific metrics such as true positive and false positive rates for detecting harmful content, hallucination rates measured through adapted TruthfulQA scoring, safety and reliability assessments, bias detection via adapted BBQ benchmarks, and ethical evaluation using structured Likert-based scoring rubrics. The framework is illustrated using examples from plastic surgery, but is adaptable across specialties, and is intended for use by all medical providers, regardless of their backgrounds or familiarity with artificial intelligence. While the framework is currently conceptual and validation is ongoing, PIEE provides a practical foundation for assessing the clinical reliability and ethical robustness of LLMs in medicine.

(1) Burn injuries demand multidisciplinary, evidence-based care, yet the extensive literature complicates timely decision making. Retrieval-augmented generation (RAG) synthesizes research while addressing inaccuracies in pretrained models. However, citation bias in sourcing for RAG often prioritizes highly cited studies, overlooking less-cited but valuable research. This study examines RAG’s performance in burn management, comparing citation levels to enhance evidence synthesis, reduce selection bias, and guide decisions. (2) Two burn management datasets were assembled: 30 highly cited (mean: 303) and 30 less-cited (mean: 21). The Gemini-1.0-Pro-002 RAG model addressed 30 questions, ranging from foundational principles to advanced surgical approaches. Responses were evaluated for accuracy (5-point scale), readability (Flesch–Kincaid metrics), and response time with Wilcoxon rank sum tests (p < 0.05). (3) RAG achieved comparable accuracy (4.6 vs. 4.2, p = 0.49), readability (Flesch Reading Ease: 42.8 vs. 46.5, p = 0.26; Grade Level: 9.9 vs. 9.5, p = 0.29), and response time (2.8 vs. 2.5 s, p = 0.39) for the highly and less-cited datasets. (4) Less-cited research performed similarly to highly cited sources. This equivalence broadens clinicians’ access to novel, diverse insights without sacrificing quality. As plastic surgery evolves, RAG’s inclusive approach fosters innovation, improves patient care, and reduces cognitive burden by integrating underutilized studies. Embracing RAG could propel the field toward dynamic, forward-thinking care.

Current clinical care relies heavily on complex, rule-based systems for tasks like diagnosis and treatment. However, these systems can be cumbersome and require constant updates. This study explores the potential of the large language model (LLM), LLaMA 2, to address these limitations. We tested LLaMA 2′s performance in interpreting complex clinical process models, such as Mayo Clinic Care Pathway Models (CPMs), and providing accurate clinical recommendations. LLM was trained on encoded pathways versions using DOT language, embedding them with SentenceTransformer, and then presented with hypothetical patient cases. We compared the token-level accuracy between LLM output and the ground truth by measuring both node and edge accuracy. LLaMA 2 accurately retrieved the diagnosis, suggested further evaluation, and delivered appropriate management steps, all based on the pathways. The average node accuracy across the different pathways was 0.91 (SD ± 0.045), while the average edge accuracy was 0.92 (SD ± 0.122). This study highlights the potential of LLMs for healthcare information retrieval, especially when relevant data are provided. Future research should focus on improving these models’ interpretability and their integration into existing clinical workflows.

Background: Effective pain management is crucial for both comfort and outcomes, yet predicting and managing this pain is difficult. This study aimed to analyze postoperative pain in patients undergoing hand surgery at the Mayo Clinic Florida, examining how patient characteristics and anxiety affect pain outcomes. Methods: We conducted a single-arm clinical trial at Mayo Clinic Florida, recruiting patients undergoing hand surgery. Preoperative pain and anxiety were assessed using the Pain Catastrophizing Scale (PCS) and State-Trait Anxiety Inventory (STAI). Postoperatively, patients used an iPhone app to record pain levels and medication use every four hours. Devices were collected three days after surgery. We analyzed the relationship between demographics, PCS, STAI scores, and pain levels using linear and logistic regression models. All statistical tests were two-sided with significance set at p < 0.05, analyzed with R4.2.2. Results: Data were collected from 62 patients (62.9% women, 37.1% men) undergoing hand surgery. Participants were mainly White (90.3%), with 50% being in the middle-aged adult group. Most had low anxiety levels (80.6% STAI-S, 82.3% STAI-T) and low catastrophizing (61.3% PCS). Postoperative pain was low, with median scores between 1.0 and 2.0 over three days. Demographics, anxiety, and catastrophizing were not significant predictors of pain levels. Logistic regression showed time as a significant factor, with pain levels peaking on Day 3. Conclusions: Postoperative pain after hand surgery was generally low, with time being a significant predictor of increased pain. Demographic factors, anxiety, and catastrophizing did not significantly affect pain levels. Pain management should emphasize time-sensitive interventions and ongoing monitoring.

The marine benthic zone is host to a vast array of species that are vital to both ecosystem function and biodiversity conservation. Understanding these underwater communities is essential for ecological research, conservation efforts, and sustainable resource management. However, traditional surveying techniques face significant challenges in the underwater environment. This research extends strategies for underwater biological monitoring by offering a reliable and useful method for detecting benthos in marine benthic zones. Ultimately, by promoting conservation initiatives and sustainable management techniques in marine settings, the proposed Single Shot Detector (SSD) based system may speed up underwater surveys, make data collection easier, and further our knowledge of benthic ecosystems.