Rudrapatna, Vivek A

Importance Accurate prediction of patient mortality after kidney transplant is an unmet need. Objective To develop and validate an integrative prediction model that predicts short- and long-term patient mortality for kidney recipients. Design, Setting, and Participants This international cohort study included patients who underwent transplants between 2004 and 2023 from 14 academic medical centers from Europe and the US. The derivation cohort consisted of 1566 consecutive adult kidney recipients in a deeply phenotyped cohort prospectively recruited in 1 French center between 2004 and 2014. External validation cohorts consisted of 10 951 kidney recipients from 7 centers in France, 3 centers in Europe, and 3 centers in North America who underwent kidney transplants between 2006 and 2023. Data were analyzed from January 2023 to June 2025. Main Outcome and Measures All-cause mortality was assessed, and 121 candidate prognostic factors, comprising demographics and clinical, biological, imaging, and immunological parameters, were collected. Results Among 12 517 kidney transplant recipients, including 1566 in the derivation cohort (mean [SD] age, 50.05 [14.31] years; 942 male [60.15%]) and 10 951 in validation cohorts (mean [SD] age, 53.32 [13.97] years; 6766 male [61.78%]), 2486 patients (19.9%) died after a median (IQR) follow-up of 5.08 (2.97-7.00) years. Fourteen prognostic factors (including clinical, biological, and imaging risk factors) were independently associated with patient death (eg, patient age: hazard ratio per 1-year increase in age, 1.07 [95% CI, 1.06-1.08]; P < .001) and were combined into a risk prediction model (mBox). The model exhibited accurate calibration and discrimination in the derivation cohort, with C statistics of 0.82 (95% CI, 0.77-0.87) and 0.80 (95% CI, 0.78-0.82) at 1 and 10 years, respectively, after the transplant. Abbreviated models were developed and validated to ensure model generalizability. Performance of abbreviated models was confirmed in external validation cohorts from France (C statistic, 0.76 [95% CI, 0.73-0.78]), Europe (C statistic, 0.74 [95% CI, 0.72-0.76]), the US (C statistic, 0.74 [95% CI, 0.70-0.78]), the Greater Paris University Hospital database (C statistic, 0.79 [95% CI, 0.77-0.81]), and the University of California at San Francisco database (C statistic, 0.70 [95% CI, 0.65-0.74]). The model was also validated in a series of subpopulations (eg, women vs men: C statistic, 0.81 [95% CI, 0.77-0.84] vs 0.79 [95% CI, 0.77-0.82]) and clinical scenarios (eg, before COVID-19 era: C statistic, 0.79 [95% CI, 0.77-0.81]) with good and stable performance. Conclusions and Relevance In this study, an accurate prediction model for mortality among kidney recipients, computable at the time of transplant, was developed and externally validated. Results suggest that this model may help stratify patient risk of death, allowing for improved medical decisions.

Objective Closed-source large language models (LLMs) like generative pre-trained transformer 4o (GPT-4o) have shown promise for clinical information extraction but are potentially limited by cost, data security concerns and inflexibility. Open-source models are an attractive alternative with various adaptation strategies with no consensus on best practices. This study aims to rigorously identify optimal adaptation strategies for open-source models and evaluate their performance relative to closed-source alternatives. Methods and Analysis We studied three LLM adaptation strategies: chain-of-thought prompting, few-shot prompting and fine-tuning. Our target for information extraction was the Mayo Endoscopic Subscore (MES). We applied those strategies in all combinations to six open-source models (8–70 billion parameters) using an annotated set of colonoscopy procedure reports from the University of California, San Francisco (N=608) and San Francisco General Hospital (N=217). We analysed the relationship of these strategies to several performance metrics with a mixed-effects model, accounting for the variability between centres and LLMs. GPT-4o served as a closed-source oracle and provided in-depth commentary on the cost-effectiveness of these options. Results Quantised low-rank adaptation (QLoRA) statistically improves ( p §amp;lt; 0.001 ) ) the performance of open-source LLMs by 9.1–15.7 percentage points across accuracy, precision recall and annotation eligibility accuracy. However, GPT-4o with prompt engineering outperforms the best open-source model by 4.9%–11.2%. A simple cost-effectiveness analysis suggests that GPT-4o is more affordable compared with open-source alternatives. Conclusion GPT-4o is currently the most efficient LLM for MES extraction. If unavailable, QLoRA-optimised open-source models are a competitive alternative. However, results also suggest that current instruction-following LLMs including GPT-4o do not fully follow user-provided instructions, leaving room for improvement. More work is needed to achieve consistent, near-perfect performance in clinical information extraction by LLMs.

Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLMs) like Bidirectional Encoder Representations from Transformers (BERT) have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event (AE) detection. We adapted a new clinical LLM, University of California – San Francisco (UCSF)‐BERT, to identify serious AEs (SAEs) occurring after treatment with a non‐steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. We annotated 928 outpatient IBD notes corresponding to 928 individual patients with IBD for all SAE‐associated hospitalizations occurring after treatment with a non‐steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of eight candidate models, UCSF‐BERT achieved the highest numerical performance on identifying drug‐SAE pairs from this corpus (accuracy 88–92%, macro F1 61–68%), with 5–10% greater accuracy than previously published models. UCSF‐BERT was significantly superior at identifying hospitalization events emergent to medication use ( P < 0.01). LLMs like UCSF‐BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared with prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multicenter data and newer architectures like Generative pre‐trained transformer (GPT). Our findings support the potential value of using large language models to enhance pharmacovigilance.

Abstract Background The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. Methods We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. Results Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). Conclusions We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.