Citations

Background/Objective: Hypertension is the most prevalent chronic disease in primary care and a leading cause of cardiovascular morbidity and mortality. Despite existing guidelines, therapeutic inertia and suboptimal control persist. Large language models (LLMs) offer a potential valuable addition to augment clinical decision-making, yet their reliability for guideline-driven tasks remains unverified. This study evaluated the accuracy and safety of hypertension management recommendations generated by three LLMs compared to expert responses. Methods: Fifty-one clinical vignettes representing 17 core hypertension management concepts were constructed by hypertension experts. Each case was submitted to three LLMs (GPT-4, Gemini, MedLM) and a hypertension expert also wrote the “gold standard” answers. Three blinded expert reviewers rated each response on a 4-point accuracy scale, a binary safety (safe/unsafe) scale, and attempted to identify the source (LLM vs. expert) providing the response. Ratings were analyzed using mean scores, percentages of accurate and safe responses, and inter-rater agreement. Results: GPT-4 had the highest accuracy (83%) and safety (86%) scores among LLMs but remained inferior to expert responses (92% accuracy, 93% safety). Gemini and MedLM performed significantly worse (accuracy: 64% and 35%; safety: 73% and 39%, respectively). GPT-4 generated the most guideline-concordant responses (46%) among the three LLMs (Gemini 35%, MedLM 14%), but remains lower than experts’ responses (68%). Evaluators misidentified LLM responses as expert-written in 10 to 25% of cases, particularly with GPT-4. Inter-rater reliability for accuracy ratings was highest for expert-generated responses (ICC 0.81), with progressively lower agreement for GPT-4 (0.76), Gemini (0.70), and MedLM (0.68). A similar pattern was observed for safety and source discrimination ratings. The agreement was strongest for safety assessments and weakest for source discrimination. Conclusion: Among three tested LLMs, GPT-4 demonstrated closer agreement to expert decisions thereby showing greater potential for supporting hypertension management. However, current LLMs’ versions frequently produce inaccurate or unsafe recommendations and remain inferior to expert judgment. Human-in-the-loop supervision remains essential when deploying LLMs for clinical decision-making.

BACKGROUND: Hypertension, the leading cause of cardiovascular mortality, remains suboptimally controlled. Large language models (LLMs) could improve hypertension control by augmenting clinical decision-making, but their reliability for guideline-driven tasks is unverified. This study evaluated the accuracy and safety of hypertension management recommendations generated by 3 LLMs. METHODS: Fifty-one vignettes were constructed and submitted to the LLMs (GPT-4, Gemini, Medical Large Language Model [by Google; MedLM]) and a hypertension expert to generate the responses. Three blinded reviewers rated each response on a 4-point accuracy scale, a binary safety (safe/unsafe) scale, and attempted to identify the source (LLM versus expert) providing the response. RESULTS: GPT-4 had the highest accuracy (83%) and safety (86%) scores among LLMs but remained inferior to expert responses (92% accuracy, 93% safety). Gemini and MedLM performed significantly worse (accuracy: 64% and 35%; safety: 73% and 39%, respectively). GPT-4 generated the most guideline-concordant responses (46%) among the 3 LLMs (Gemini 35%, MedLM 14%) but was lower than expert responses (68%). Interrater reliability for accuracy ratings was higher for LLM-generated responses (GPT-4 [intraclass correlation coefficient, 0.30], Gemini [intraclass correlation coefficient, 0.61], and MedLM [intraclass correlation coefficient, 0.58]), with lower agreement for expert responses (intraclass correlation coefficient, 0.23). A similar pattern was observed for safety and source discrimination ratings. The agreement was strongest for safety assessments and weakest for source discrimination. CONCLUSIONS: Among the 3 tested LLMs, GPT-4 demonstrated closer agreement to expert decisions, thereby showing greater potential for supporting hypertension management. Despite their potential, current LLM versions are inferior to expert recommendations. Human-in-the-loop supervision remains essential when deploying LLMs for clinical decision-making.

Artificial intelligence (AI), including machine learning, deep learning, reinforcement learning, and generative AI, has the potential to advance critical care nephrology (CCN) by enhancing prediction accuracy, improving diagnostic capabilities, supporting clinical decision making, and streamlining workflow processes. Current applications in CCN include AKI prediction, nephrotoxin surveillance, intradialytic hypotension forecasting, and AI-guided dialysis and continuous KRT management, with performance often exceeding traditional models. However, the effect on patient-centered outcomes such as mortality, dialysis dependence, and cost-effectiveness remains uncertain. Emerging techniques, such as conformal prediction for calibrated risk estimates, causal inference for intervention modeling, and reinforcement learning for adaptive ultrafiltration, show promise in enhancing reliability, interpretability, and individualized care. Generative AI and large language models extend these applications to clinical documentation, reasoning, and patient education, while raising new challenges, including hallucinations, regulatory oversight, and clinician trust. Persistent barriers such as data heterogeneity, limited external validation, alert fatigue, and economic constraints hinder broad adoption. This review synthesizes the current evidence and outlines four priorities for advancing AI in CCN: ( 1 ) rigorous multicenter validation focused on clinical outcomes, ( 2 ) integration of uncertainty quantification and causal modeling into AI tools, ( 3 ) development of clinician-centered interfaces that minimize cognitive load, and ( 4 ) establishment of transparent, adaptive regulatory and governance frameworks. Realizing the promise of AI in CCN will require multidisciplinary collaboration, fairness and generalizability testing, and sustainable implementation strategies that align technologic innovation with measurable improvements in patient care.

We appreciate Dr Akcay's insightful comments. His observations provide valuable context and highlight important considerations for future research. We hope our work will serve as a foundation for further investigation into sex-associated risks in peritoneal dialysis and contribute to optimizing high-quality patient care for all individuals. Looking ahead, future studies should integrate dialysis-specific measures, explore mechanistic pathways, and incorporate patient-centered outcomes to better characterize sex-associated differences in peritoneal dialysis.