Citations

Objectives/Goals: This study aims to evaluate the performance of a third-party artificial intelligence (AI) product in predicting diagnosis-related groups (DRGs) in a community healthcare system. We highlight a use case illustrating how clinical practice leverages AI-predicted information in unexpected yet advantageous ways and assess the AI predictions accuracy and practical application. Methods/Study Population: DRGs are crucial for hospital reimbursement under the prospective payment model. The Mayo Clinic Health System (MCHS), a network of clinics and hospitals serving a substantial rural population in Minnesota and Wisconsin, has recently adopted an AI algorithm developed by Xsolis (an AI-focused healthcare solution provider). This algorithm, a 1D convolutional neural network, predicts DRGs based on clinical documentation. To assess the accuracy of AI-generated DRG predictions for inpatient discharges, we analyzed data from 930 patients hospitalized at MCHS Mankato between March 2 and May 13, 2024. The Xsolis platform provided the top three DRG predictions for the first 48 hours of each inpatient stay. The accuracy of these predictions was then compared against the final billed DRG codes from the hospital’s records. Results/Anticipated Results: In our validation set, Xsolis achieved a top-3 DRG prediction accuracy of 71% at 24 hours and 81% at 48 hours, which is lower than the originally reported accuracy of 81.1% and 83.3%, respectively. Interestingly, discussions with clinical practice leaders revealed that the most valuable information derived from the AI predictions was the expected geometric mean length of stay (GMLOS), which Xsolis was perceived to predict accurately. In the Medicare system, each DRG is associated with an expected GMLOS, a critical factor for efficient hospital flow planning. A subsequent analysis comparing predicted GMLOS with the actual length of stay showed variances of -0.10 days on day 1 and 0.14 days on day 2, indicating a high degree of accuracy and aligning with clinical practice perceptions. Discussion/Significance of Impact: Our research underscores that clinical practice can leverage AI predictions in unexpected yet beneficial ways. While initially focused on DRG prediction, the associated GMLOS emerged as more significant. This suggests that AI algorithm validation should be tailored to specific clinical needs rather than relying solely on generalized benchmarks.

Abstract In the U.S. inpatient payment system, the Diagnosis-Related Group (DRG) is pivotal, but its assignment process is inefficient. The study introduces DRG-LLaMA, an advanced large language model (LLM) fine-tuned on clinical notes to enhance DRGs assignment. Utilizing LLaMA as the foundational model and optimizing it through Low-Rank Adaptation (LoRA) on 236,192 MIMIC-IV discharge summaries, our DRG-LLaMA-7B model exhibited a noteworthy macro-averaged F1 score of 0.327, a top-1 prediction accuracy of 52.0%, and a macro-averaged Area Under the Curve (AUC) of 0.986, with a maximum input token length of 512. This model surpassed the performance of prior leading models in DRG prediction, showing a relative improvement of 40.3% and 35.7% in macro-averaged F1 score compared to ClinicalBERT and CAML, respectively. Applied to base DRG and complication or comorbidity (CC)/major complication or comorbidity (MCC) prediction, DRG-LLaMA achieved a top-1 prediction accuracy of 67.8% and 67.5%, respectively. Additionally, our findings indicate that DRG-LLaMA's performance correlates with increased model parameters and input context lengths.

Introduction: Richter's syndrome (RS) represents transformation of chronic lymphocytic leukemia (CLL) into a highly aggressive lymphoma with dismal prognosis. Transcriptomic alterations have been described in CLL but most studies focused on peripheral blood samples with minimal data on RS-involved tissue. Moreover, transcriptomic features of RS have not been well defined in the era of CLL novel therapies. In this study we investigated transcriptomic profiles of CLL/RS-involved nodal tissue using samples from a clinical trial cohort of refractory CLL and RS patients treated with Pembrolizumab (NCT02332980). Methods: Nodal samples from 9 RS and 4 CLL patients in MC1485 trial cohort were reviewed and classified as previously published (Ding et al, Blood 2017). All samples were collected prior to Pembrolizumab treatment. Targeted gene expression profiling of 789 immune-related genes were performed on FFPE nodal samples using Nanostring nCounter® Analysis System (NanoString Technologies, Seattle, WA). Differential expression analysis was performed using NanoStringDiff. Genes with 2 fold-change in expression with a false-discovery rate less than 5% were considered differentially expressed. Results: The details for the therapy history of this cohort were illustrated in Figure 1a. All patients exposed to prior ibrutinib before the tissue biopsy had developed clinical progression while receiving ibrutinib. Unsupervised hierarchical clustering using the 300 most variable genes in expression revealed two clusters: C1 and C2 (Figure 1b). C1 included 4 RS and 3 CLL treated with prior chemotherapy without prior ibrutinib, and 1 RS treated with prior ibrutinib. C2 included 1 CLL and 3 RS received prior ibrutinib, and 1 RS treated with chemotherapy. The segregation of gene expression profiles in samples was largely driven by recent exposure to ibrutinib. In C1 cluster (majority had no prior ibrutinb), RS and CLL samples were clearly separated into two subgroups (Figure 1b). In C2 cluster, CLL 8 treated with ibrutinib showed more similarity in gene expression to RS, than to other CLL samples treated with chemotherapy. In comparison of C2 to C1, we identified 71 differentially expressed genes, of which 34 genes were downregulated and 37 were upregulated in C2. Among the upregulated genes in C2 (majority had prior ibrutinib) are known immune modulating genes including LILRA6, FCGR3A, IL-10, CD163, CD14, IL-2RB (figure 1c). Downregulated genes in C2 are involved in B cell activation including CD40LG, CD22, CD79A, MS4A1 (CD20), and LTB, reflecting the expected biological effect of ibrutinib in reducing B cell activation. Among the 9 RS samples, we compared gene profiles between the two groups of RS with or without prior ibrutinib therapy. 38 downregulated genes and 10 upregulated genes were found in the 4 RS treated with ibrutinib in comparison with 5 RS treated with chemotherapy. The top upregulated genes in the ibrutinib-exposed group included PTHLH, S100A8, IGSF3, TERT, and PRKCB, while the downregulated genes in these samples included MS4A1, LTB and CD38 (figure 1d). In order to delineate the differences of RS vs CLL, we compared gene expression profiles between 5 RS samples and 3 CLL samples that were treated with only chemotherapy. RS samples showed significant upregulation of 129 genes and downregulation of 7 genes. Among the most significantly upregulated genes are multiple genes involved in monocyte and myeloid lineage regulation including TNFSF13, S100A9, FCN1, LGALS2, CD14, FCGR2A, SERPINA1, and LILRB3. Conclusion: Our study indicates that ibrutinib-resistant, RS-involved tissues are characterized by downregulation of genes in B cell activation, but with PRKCB and TERT upregulation. Furthermore, RS-involved nodal tissues display the increased expression of genes involved in myeloid/monocytic regulation in comparison with CLL-involved nodal tissues. These findings implicate that differential therapies for RS and CLL patients need to be adopted based on their prior therapy and gene expression signatures. Studies using large sample size will be needed to verify this hypothesis. Figure Disclosures Zhao: Merck: Current Employment. Blumenschein:Merck: Current Employment. Yearley:Merck: Current Employment. Wang:Novartis: Research Funding; Incyte: Research Funding; Innocare: Research Funding. Parikh:Verastem Oncology: Honoraria; GlaxoSmithKline: Honoraria; Pharmacyclics: Honoraria, Research Funding; MorphoSys: Research Funding; Ascentage Pharma: Research Funding; Genentech: Honoraria; AbbVie: Honoraria, Research Funding; Merck: Research Funding; TG Therapeutics: Research Funding; AstraZeneca: Honoraria, Research Funding; Janssen: Honoraria, Research Funding. Kenderian:Sunesis: Research Funding; MorphoSys: Research Funding; Humanigen: Consultancy, Patents & Royalties, Research Funding; Gilead: Research Funding; BMS: Research Funding; Tolero: Research Funding; Lentigen: Research Funding; Juno: Research Funding; Mettaforge: Patents & Royalties; Torque: Consultancy; Kite: Research Funding; Novartis: Patents & Royalties, Research Funding. Kay:Astra Zeneca: Membership on an entity's Board of Directors or advisory committees; Acerta Pharma: Research Funding; Juno Theraputics: Membership on an entity's Board of Directors or advisory committees; Dava Oncology: Membership on an entity's Board of Directors or advisory committees; Oncotracker: Membership on an entity's Board of Directors or advisory committees; Sunesis: Research Funding; MEI Pharma: Research Funding; Agios Pharma: Membership on an entity's Board of Directors or advisory committees; Bristol Meyer Squib: Membership on an entity's Board of Directors or advisory committees, Research Funding; Tolero Pharmaceuticals: Membership on an entity's Board of Directors or advisory committees, Research Funding; Abbvie: Research Funding; Pharmacyclics: Membership on an entity's Board of Directors or advisory committees, Research Funding; Rigel: Membership on an entity's Board of Directors or advisory committees; Morpho-sys: Membership on an entity's Board of Directors or advisory committees; Cytomx: Membership on an entity's Board of Directors or advisory committees. Braggio:DASA: Consultancy; Bayer: Other: Stock Owner; Acerta Pharma: Research Funding. Ding:DTRM: Research Funding; Astra Zeneca: Research Funding; Abbvie: Research Funding; Merck: Membership on an entity's Board of Directors or advisory committees, Research Funding; Octapharma: Membership on an entity's Board of Directors or advisory committees; MEI Pharma: Membership on an entity's Board of Directors or advisory committees; alexion: Membership on an entity's Board of Directors or advisory committees; Beigene: Membership on an entity's Board of Directors or advisory committees.

Introduction: Novel targeted therapies have revolutionized treatments for both upfront and relapsed CLL. However, effective therapies have not been established for progressive CLL after developing resistance to both BTK inhibitor and BCL-2 inhibitors. Similarly, Richter's transformation (RS) remains an area of unmet clinical need. Hypothesizing that progressive CLL and RS possess significant but differential molecular abnormalities, we aimed to dissect the initial genomic profiles in a clinical cohort of patients seen in our routine clinical practice using targeted DNA sequencing and transcriptome analysis. Methods: Twelve CLL patients who had lymph node biopsies for evaluation of clinical progression at Mayo Clinic were included in this study. RNA sequencing and targeted DNA sequencing on a focused oncology panel of 1,711 genes were performed on Illumina HiSeq through Tempus Labs, Inc. (Chicago, IL). Differentially expressed (DE) genes were identified using edgeR at the cutoff of FDR <=0.05 and fold-change >=2. Somatic mutations and copy number alternations (CNAs) were identified through Mutect2 and PatternCNV, respectively. CN loss was defined to have log2(ratio) <=-0.93 and CN gain to have log2(ratio) >= 0.585. Pathway analysis was done through the Enrichr package. Results: Among the 12 patients, 7 had confirmed RS (large cell lymphoma histology) and 5 had CLL based on pathological findings. The clinical and therapeutic history for each patient is illustrated in Fig 1A. FISH results from 11 of the patients showed chromosomal abnormalities including del(13q), del(11q), del(17p) and tri(12) (Fig 1A). Hierarchical clustering of expression profiles from the top 10,000 most variable genes in expression revealed two major clusters (Fig 1A). Cluster C1 contains five RS cases, of which case S207 did not receive ibrutinib, S256 was on high-dose steroid and anti-CD20 post-ibrutinib therapy for >1 year before biopsy, and case S025 was treated for < 6 months with ibrutinib. These three form a subgroup separate from the other two RS in C1 who had 8 and 42 months of continuous ibrutinib therapy pre-biopsy, respectively. C2 contains two RS and five CLL cases. Within C2, CLL and RS cases showed similar transcriptomic profiles. Over 90% of the DE genes showed an overall increase of expression in C1, particularly within the 3-sample subgroup (Fig 1B). These genes are enriched in pathways known to play key roles in CLL, including PI3K-AKT signaling, NOTCH signaling and RAS/MAPK signaling, while genes up-regulated in C2 are enriched in WNT signaling. The results suggest marked differences in gene expression profiles and indicate molecular heterogeneity among RS and CLL patients. In addition, targeted DNA sequencing data were analyzed to identify somatic mutations and CNAs. The analysis confirmed the FISH results of del(13q), del(11q), del(17p) or tri(12) in 8 of the cases (Fig 1A). The mutation profiles are similar between C1 and C2 for the recurrently mutated genes published in CLL. However, we identified recurrent CNAs that occurred preferentially in C1 (covering 9 genes) or C2 (48 genes) (Fig 1C). Manual inspections of the reads coverage revealed that 5 of the 9 genes are from chr1q21.1 to chr1q22 that showed clear CN gains largely in C1. Of the 48 genes, 6 are from chrXp11 that showed CN losses, and 27 are from chr17q12 to chr17q25 that showed CN gains. Therefore, the majority of the 57 genes are from large-scale CNAs. Notably, for the CNAs mostly occurring in C2, the affected genes are enriched in WNT signaling, Cell cycle/protein kinase activity and chromatin modifying enzymes. Collectively, the results support the variability of transcriptomic signatures and genetic lesions between RS and CLL. Conclusion: This study revealed a noticeable heterogeneity in gene expression and genetic variation between RS and CLL, albeit in a small cohort. The vast majority of the DE genes showed elevated expression in C1 containing only RS cases, and these genes are enriched in PI3K-AKT signaling, NOTCH signaling, and RAS/MAPK signaling. The pattern is more pronounced for a subgroup in C1 who were overall less exposed to ibrutinib. For somatic variation, there is a strong tendency of occurrence of recurrent CNAs towards C2 cohort, mainly involving gains of chr17q and deletions of chrXp11. Given these advances we are encouraged to further pursue the genetic abnormalities with defined roles in aggressive and transformed CLL. Disclosures Liu: Eisal: Research Funding; Genentech: Research Funding; GRAIL: Research Funding; Menarini Silicon Biosystems: Research Funding; Merck: Research Funding; Seattle Genetics: Research Funding; Tesaro: Research Funding. Parikh:Genentech: Honoraria; Ascentage Pharma: Research Funding; AbbVie: Honoraria, Research Funding; Verastem Oncology: Honoraria; Pharmacyclics: Honoraria, Research Funding; GlaxoSmithKline: Honoraria; Merck: Research Funding; MorphoSys: Research Funding; TG Therapeutics: Research Funding; AstraZeneca: Honoraria, Research Funding; Janssen: Honoraria, Research Funding. Wang:Incyte: Research Funding; Innocare: Research Funding; Novartis: Research Funding. Kenderian:Gilead: Research Funding; Humanigen: Consultancy, Patents & Royalties, Research Funding; MorphoSys: Research Funding; Lentigen: Research Funding; Novartis: Patents & Royalties, Research Funding; Torque: Consultancy; Mettaforge: Patents & Royalties; BMS: Research Funding; Tolero: Research Funding; Sunesis: Research Funding; Juno: Research Funding; Kite: Research Funding. Kay:Tolero Pharmaceuticals: Membership on an entity's Board of Directors or advisory committees, Research Funding; Juno Theraputics: Membership on an entity's Board of Directors or advisory committees; Cytomx: Membership on an entity's Board of Directors or advisory committees; Morpho-sys: Membership on an entity's Board of Directors or advisory committees; Agios Pharma: Membership on an entity's Board of Directors or advisory committees; Astra Zeneca: Membership on an entity's Board of Directors or advisory committees; Sunesis: Research Funding; MEI Pharma: Research Funding; Acerta Pharma: Research Funding; Rigel: Membership on an entity's Board of Directors or advisory committees; Oncotracker: Membership on an entity's Board of Directors or advisory committees; Abbvie: Research Funding; Bristol Meyer Squib: Membership on an entity's Board of Directors or advisory committees, Research Funding; Dava Oncology: Membership on an entity's Board of Directors or advisory committees; Pharmacyclics: Membership on an entity's Board of Directors or advisory committees, Research Funding. Braggio:DASA: Consultancy; Bayer: Other: Stock Owner; Acerta Pharma: Research Funding. Ding:Astra Zeneca: Research Funding; DTRM: Research Funding; Beigene: Membership on an entity's Board of Directors or advisory committees; alexion: Membership on an entity's Board of Directors or advisory committees; MEI Pharma: Membership on an entity's Board of Directors or advisory committees; Merck: Membership on an entity's Board of Directors or advisory committees, Research Funding; Octapharma: Membership on an entity's Board of Directors or advisory committees; Abbvie: Research Funding.