Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA.
Citations 21
Generative AI and large language model evaluation of kidney donation websites: Benchmarking health literacy, sentiment, and digital engagement to optimize donor recruitment
Background Kidney transplantation offers substantial clinical and economic benefits for patients with end-stage kidney disease (ESKD), yet organ shortages persist. Enhancing public awareness and health literacy regarding kidney donation is essential for effective donor recruitment. While online patient education materials are primary drivers of public perception, their readability, digital engagement, and accessibility remain underexplored barriers. Methods We analyzed the most prominent kidney donation websites identified through U.S.-based Google searches, including 11 primary organizations and 16 affiliated subdomains, for a total of 27 websites. Readability was benchmarked using Flesch-Kincaid, SMOG, and Gunning Fog indices. To assess qualitative metrics, we deployed a generative AI framework utilizing a Large Language Model (Claude) to conduct automated sentiment analysis and tone evaluation, validated by human review. We systematically mapped digital engagement features, including multimedia, interactive tools, and multilingual support, to determine content comprehensiveness. Results Websites consistently provided accurate information with a generally positive or neutral tone. Average readability exceeded recommended levels, with a combined mean grade of 12.3; 34% of websites were written at a college-level reading standard. Consensus across content was high. Multimedia elements were widely used, but engagement features were limited; only 30% of sites included extensive testimonials, and interactive tools were absent. AI-based analysis enabled standardized and reproducible evaluation, highlighting opportunities to improve accessibility, tone, and inclusivity. Conclusion Current U.S. kidney donation digital resources present a barrier to health equity due to excessive reading complexity and static engagement models. AI provides a scalable, reproducible framework to audit and optimize patient education materials. Future initiatives must leverage AI-guided content optimization to bridge the literacy gap, potentially increasing donor registration and access to transplantation.
AI-generated explanations in kidney transplantation: accuracy vs. readability and implications for patient education
Background Effective patient education is critical for informed decision-making and adherence in kidney transplantation. Generative artificial intelligence (AI), particularly large language models (LLMs), has the potential to enhance patient education in kidney transplantation; however, its factual accuracy and readability remain incompletely characterized. Methods We evaluated the performance of the GPT-5.1 (2025) model in generating plain-language explanations for 100 clinically relevant kidney transplantation terms. Explanations were generated using a standardized prompt (first round) and a revised prompt explicitly requesting an eighth-grade reading level or lower (second round). Accuracy was assessed by expert reviewers using a 5-point Likert scale, while readability was evaluated using the Flesch Reading Ease (higher score indicated easier readability of the text) and Flesch–Kincaid Grade Level (higher score indicated higher education level required to understand the text) score. The study was conducted in November 2025. Results All AI-generated explanations demonstrated high accuracy, with no clinically significant errors. In the first round, the mean Flesch Reading Ease score was 23.6 ± 23.4, indicating very difficult readability, and 46% of explanations required a college-level reading ability (mean Flesch–Kincaid Grade Level 13.4 ± 4.8). Following prompt revision, readability improved substantially. The mean Flesch Reading Ease score increased to 62.4 ± 7.5, corresponding to standard readability, and all explanations were written at a middle school level or below (mean Flesch–Kincaid Grade Level 6.3 ± 1.1). Conclusion GPT-5.1 generated highly accurate explanations of kidney transplantation terms across prompting strategies. Explicit readability-focused prompting substantially improved readability without compromising accuracy, underscoring the importance of prompt design when deploying LLMs for patient-centered education in transplantation.
Digital transformation of nephrology POCUS education—Integrating a multiagent, artificial intelligence, and human collaboration-enhanced curriculum with expert feedback
Background The digital transformation in medical education is reshaping how clinical skills, such as point-of-care ultrasound (POCUS), are taught. In nephrology fellowship programs, POCUS is essential for enhancing diagnostic accuracy, guiding procedures, and optimizing patient management. To address these evolving demands, we developed an artificial intelligence (AI)-driven POCUS curriculum using a multiagent approach that integrates human expertise with advanced AI models, thereby elevating educational standards and better preparing fellows for contemporary clinical practice. Methods In April 2024, the Mayo Clinic Minnesota Nephrology Fellowship Program initiated a novel AI-assisted process to design a comprehensive POCUS curriculum. This process integrated multiple advanced AI models—including GPT-4.0, Claude 3.0 Opus, Gemini Advanced, and Meta AI with Llama 3—to generate initial drafts and iteratively refine content. A panel of blinded nephrology POCUS experts subsequently reviewed and modified the AI-generated material to ensure both clinical relevance and educational rigor. Results The curriculum underwent 12 iterative revisions, incorporating feedback from 29 communications across AI models. Key features of the final curriculum included expanded core topics, diversified teaching methods, enhanced assessment tools, and integration into inpatient and outpatient nephrology rotations. The curriculum emphasized quality assurance, POCUS limitations, and essential clinical applications, such as fistula/graft evaluation and software integration. Alignment with certification standards further strengthened its utility. AI models contributed significantly to the curriculum's foundational structure, while human experts provided critical clinical insights. Conclusion This curriculum, enhanced through a multiagent approach that combines AI and human collaboration, exemplifies the transformative potential of digital tools in nephrology education. The innovative framework seamlessly integrates advanced AI models with expert clinical insights, providing a scalable model for medical curriculum development that is responsive to evolving educational demands. The synergy between technological innovation and human expertise holds promising implications for advancing fellowship training. Future studies should evaluate its impact on clinical competencies and patient outcomes across diverse practice environments.
Public interest in chronic kidney disease and dialysis: a 20-year data analysis
Background/Objectives: It is often challenging for patients to understand treatment options, their mechanisms of action, and the potential side effects of each treatment option for glomerular disorders. This study explored the ability of ChatGPT to simplify these treatment options to enhance patient understanding. Methods: GPT-4 was queried on sixty-seven glomerular disorders using two distinct queries for a general explanation and an explanation adjusted for an 8th grade level or lower. Accuracy was rated on a scale of 1 (incorrect) to 5 (correct and comprehensive). Readability was measured using the average of the Flesch–Kincaid Grade (FKG) and SMOG indices, along with the Flesch Reading Ease (FRE) score. The understandability score (%) was determined using the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P). Results: GPT-4’s general explanations had an average readability level of 12.85 ± 0.93, corresponding to the upper end of high school. When tailored for patients at or below an 8th-grade level, the readability improved to a middle school level of 8.44 ± 0.72. The FRE and PEMAT-P scores also reflected improved readability and understandability, increasing from 25.73 ± 6.98 to 60.75 ± 4.56 and from 60.7% to 76.8% (p < 0.0001 for both), respectively. The accuracy of GPT-4’s tailored explanations was significantly lower compared to the general explanations (3.99 ± 0.39 versus 4.56 ± 0.66, p < 0.0001). Conclusions: ChatGPT shows significant potential for enhancing the readability and understandability of glomerular disorder therapies for patients, but at a cost of reduced comprehensiveness. Further research is needed to refine the performance, evaluate the real-world impact, and ensure the ethical use of ChatGPT in healthcare settings.
Identification of kidney-related medications using AI from self-captured pill images
Background Professional opinion polling has become a popular means of seeking advice for complex nephrology questions in the #AskRenal community on X. ChatGPT is a large language model with remarkable problem-solving capabilities, but its ability to provide solutions for real-world clinical scenarios remains unproven. This study seeks to evaluate how closely ChatGPT's responses align with current prevailing medical opinions in nephrology. Methods Nephrology polls from X were submitted to ChatGPT-4, which generated answers without prior knowledge of the poll outcomes. Its responses were compared to the poll results (inter-rater) and a second set of responses given after a one-week interval (intra-rater) using Cohen's kappa statistic (κ). Subgroup analysis was performed based on question subject matter. Results Our analysis comprised two rounds of testing ChatGPT on 271 nephrology-related questions. In the first round, ChatGPT's responses agreed with poll results for 163 of the 271 questions (60.2%; κ = 0.42, 95% CI: 0.38–0.46). In the second round, conducted to assess reproducibility, agreement improved slightly to 171 out of 271 questions (63.1%; κ = 0.46, 95% CI: 0.42–0.50). Comparison of ChatGPT's responses between the two rounds demonstrated high internal consistency, with agreement in 245 out of 271 responses (90.4%; κ = 0.86, 95% CI: 0.82–0.90). Subgroup analysis revealed stronger performance in the combined areas of homeostasis, nephrolithiasis, and pharmacology (κ = 0.53, 95% CI: 0.47–0.59 in both rounds), compared to other nephrology subfields. Conclusion ChatGPT-4 demonstrates modest capability in replicating prevailing professional opinion in nephrology polls overall, with varying performance levels between question topics and excellent internal consistency. This study provides insights into the potential and limitations of using ChatGPT in medical decision making.
AI-Driven Patient Education in Chronic Kidney Disease: Evaluating Chatbot Responses against Clinical Guidelines
Chronic kidney disease (CKD) patients can benefit from personalized education on lifestyle and nutrition management strategies to enhance healthcare outcomes. The potential use of chatbots, introduced in 2022, as a tool for educating CKD patients has been explored. A set of 15 questions on lifestyle modification and nutrition, derived from a thorough review of three specific KDIGO guidelines, were developed and posed in various formats, including original, paraphrased with different adverbs, incomplete sentences, and misspellings. Four versions of AI were used to answer these questions: ChatGPT 3.5 (March and September 2023 versions), ChatGPT 4, and Bard AI. Additionally, 20 questions on lifestyle modification and nutrition were derived from the NKF KDOQI guidelines for nutrition in CKD (2020 Update) and answered by four versions of chatbots. Nephrologists reviewed all answers for accuracy. ChatGPT 3.5 produced largely accurate responses across the different question complexities, with occasional misleading statements from the March version. The September 2023 version frequently cited its last update as September 2021 and did not provide specific references, while the November 2023 version did not provide any misleading information. ChatGPT 4 presented answers similar to 3.5 but with improved reference citations, though not always directly relevant. Bard AI, while largely accurate with pictorial representation at times, occasionally produced misleading statements and had inconsistent reference quality, although an improvement was noted over time. Bing AI from November 2023 had short answers without detailed elaboration and sometimes just answered “YES”. Chatbots demonstrate potential as personalized educational tools for CKD that utilize layman’s terms, deliver timely and rapid responses in multiple languages, and offer a conversational pattern advantageous for patient engagement. Despite improvements observed from March to November 2023, some answers remained potentially misleading. ChatGPT 4 offers some advantages over 3.5, although the differences are limited. Collaboration between healthcare professionals and AI developers is essential to improve healthcare delivery and ensure the safe incorporation of chatbots into patient care.
Outcomes of peritoneal dialysis in cirrhosis: A systematic review and meta-analysis
Background: Cirrhosis and end-stage kidney disease (ESKD) are significant global health concerns, contributing to high mortality and morbidity. Haemodialysis (HD) is frequently used to treat ESKD in patients with cirrhosis. However, it often presents challenges such as haemodynamic instability during dialysis sessions, leading to less than optimal outcomes. Peritoneal dialysis (PD), while less commonly used in cirrhotic patients, raises concerns about the risks of peritonitis and mortality. Our systematic review and meta-analysis aimed to assess outcomes in PD patients with cirrhosis. Methods: We executed a comprehensive search in Ovid MEDLINE, EMBASE and Cochrane databases up to 25 September 2023. The search focused on identifying studies examining mortality and other clinical outcomes in ESKD patients with cirrhosis receiving PD or HD. In addition, we sought studies comparing PD outcomes in cirrhosis patients to those without cirrhosis. Data from each study were aggregated using a random-effects model and the inverse-variance method. Results: Our meta-analysis included a total of 13 studies with 15,089 patients. Seven studies compared ESKD patients on PD with liver cirrhosis (2753 patients) against non-cirrhosis patients (9579 patients). The other six studies provided data on PD (824 patients) versus HD (1943 patients) in patients with cirrhosis and ESKD. The analysis revealed no significant difference in mortality between PD and HD in ESKD patients with cirrhosis (pooled odds ratio (OR) of 0.77; 95% confidence interval (CI), 0.53–1.14). In PD patients with cirrhosis, the pooled OR for peritonitis compared to non-cirrhosis patients was 1.10 (95% CI: 1.03–1.18). The pooled ORs for hernia and chronic hypotension in cirrhosis patients compared to non-cirrhosis controls were 2.48 (95% CI: 0.08–73.04) and 17.50 (95% CI: 1.90–161.11), respectively. The pooled OR for transitioning from PD to HD among cirrhotic patients was 1.71 (95% CI: 0.76–3.85). Mortality in cirrhosis patients on PD was comparable to non-cirrhosis controls, with a pooled OR of 1.05 (95% CI: 0.53–2.10). Conclusions: Our meta-analysis demonstrates that PD provides comparable mortality outcomes to HD in ESKD patients with cirrhosis. In addition, the presence of cirrhosis does not significantly elevate the risk of mortality among patients undergoing PD. While there is a higher incidence of chronic hypotension and a slightly increased risk of peritonitis in cirrhosis patients on PD compared to those without cirrhosis, the risks of hernia and the need to transition from PD to HD are comparable between both groups. These findings suggest PD as a viable and effective treatment option for ESKD patients with cirrhosis.
Evaluating ChatGPT's efficacy in assessing the safety of non-prescription medications and supplements in patients with kidney disease
Background This study investigated the efficacy of ChatGPT-3.5 and ChatGPT-4 in assessing drug safety for patients with kidney diseases, comparing their performance to Micromedex, a well-established drug information source. Despite the perception of non-prescription medications and supplements as safe, risks exist, especially for those with kidney issues. The study's goal was to evaluate ChatGPT's versions for their potential in clinical decision-making regarding kidney disease patients. Method The research involved analyzing 124 common non-prescription medications and supplements using ChatGPT-3.5 and ChatGPT-4 with queries about their safety for people with kidney disease. The AI responses were categorized as “generally safe,” “potentially harmful,” or “unknown toxicity.” Simultaneously, these medications and supplements were assessed in Micromedex using similar categories, allowing for a comparison of the concordance between the two resources. Results Micromedex identified 85 (68.5%) medications as generally safe, 35 (28.2%) as potentially harmful, and 4 (3.2%) of unknown toxicity. ChatGPT-3.5 identified 89 (71.8%) as generally safe, 11 (8.9%) as potentially harmful, and 24 (19.3%) of unknown toxicity. GPT-4 identified 82 (66.1%) as generally safe, 29 (23.4%) as potentially harmful, and 13 (10.5%) of unknown toxicity. The overall agreement between Micromedex and ChatGPT-3.5 was 64.5% and ChatGPT-4 demonstrated a higher agreement at 81.4%. Notably, ChatGPT-3.5's suboptimal performance was primarily influenced by a lower concordance rate among supplements, standing at 60.3%. This discrepancy could be attributed to the limited data on supplements within ChatGPT-3.5, with supplements constituting 80% of medications identified as unknown. Conclusion ChatGPT's capabilities in evaluating the safety of non-prescription drugs and supplements for kidney disease patients are modest compared to established drug information resources. Neither ChatGPT-3.5 nor ChatGPT-4 can be currently recommended as reliable drug information sources for this demographic. The results highlight the need for further improvements in the model's accuracy and reliability in the medical domain.