Erickson, Stephen B.

Background This study investigated the efficacy of ChatGPT-3.5 and ChatGPT-4 in assessing drug safety for patients with kidney diseases, comparing their performance to Micromedex, a well-established drug information source. Despite the perception of non-prescription medications and supplements as safe, risks exist, especially for those with kidney issues. The study's goal was to evaluate ChatGPT's versions for their potential in clinical decision-making regarding kidney disease patients. Method The research involved analyzing 124 common non-prescription medications and supplements using ChatGPT-3.5 and ChatGPT-4 with queries about their safety for people with kidney disease. The AI responses were categorized as “generally safe,” “potentially harmful,” or “unknown toxicity.” Simultaneously, these medications and supplements were assessed in Micromedex using similar categories, allowing for a comparison of the concordance between the two resources. Results Micromedex identified 85 (68.5%) medications as generally safe, 35 (28.2%) as potentially harmful, and 4 (3.2%) of unknown toxicity. ChatGPT-3.5 identified 89 (71.8%) as generally safe, 11 (8.9%) as potentially harmful, and 24 (19.3%) of unknown toxicity. GPT-4 identified 82 (66.1%) as generally safe, 29 (23.4%) as potentially harmful, and 13 (10.5%) of unknown toxicity. The overall agreement between Micromedex and ChatGPT-3.5 was 64.5% and ChatGPT-4 demonstrated a higher agreement at 81.4%. Notably, ChatGPT-3.5's suboptimal performance was primarily influenced by a lower concordance rate among supplements, standing at 60.3%. This discrepancy could be attributed to the limited data on supplements within ChatGPT-3.5, with supplements constituting 80% of medications identified as unknown. Conclusion ChatGPT's capabilities in evaluating the safety of non-prescription drugs and supplements for kidney disease patients are modest compared to established drug information resources. Neither ChatGPT-3.5 nor ChatGPT-4 can be currently recommended as reliable drug information sources for this demographic. The results highlight the need for further improvements in the model's accuracy and reliability in the medical domain.

ABSTRACT Background and hypothesis The MENTOR trial (MEmbranous Nephropathy Trial Of Rituximab) showed that rituximab was noninferior to cyclosporine in inducing complete or partial remission of proteinuria and was superior in maintaining proteinuria remission. However, the cost of rituximab may prohibit first-line use for some patients and health-care payers. Methods A Markov model was used to determine the incremental cost-effectiveness ratio (ICER) of rituximab compared with cyclosporine for the treatment membranous nephropathy from the perspective of a health-care payer with a lifetime time horizon. The model was informed by data from the MENTOR trial where possible; additional parameters including cost and utility inputs were obtained from the literature. Sensitivity analyses were performed to evaluate the impact of reduced-cost biosimilar rituximab. Results Rituximab for the treatment of membranous nephropathy was cost effective (assuming a willingness-to-pay threshold of $50 000 per quality-adjusted life year (QALY) gained; in $US 2021) compared with cyclosporine, with an ICER of $8373/QALY over a lifetime time horizon. The incremental cost of rituximab therapy was $28 007 with an additional 3.34 QALYs compared with cyclosporine. Lower cost of rituximab biosimilars resulted in a more favorable ICER, and in some cases resulted in rituximab being dominant (lower cost and great benefit) compared to cyclosporine. Conclusions Despite the greater cost of rituximab, it may be a cost-effective option for the treatment of membranous nephropathy when compared with cyclosporine. The cost-effectiveness of rituximab is further improved with the use of less expensive biosimilars.