Citations

428 Background: Prostate cancer management presents a significant healthcare burden, with the need to efficiently triage patients for treatment. Our objective is to leverage large language models to predict physician-recommended treatment plans from unstructured clinical notes. By accurately predicting treatment plans, we aim to risk stratify and triage patients effectively, thereby optimizing the allocation of physician resources. Methods: 448 unstructured initial urology consultation patient notes following first positive prostate cancer biopsy were identified. The recommended and final treatments received were manually annotated to establish ground truth labels (Table 1). The dataset was split 80:20 for training and testing, preprocessed to remove plan sections and formatted into question-answer (QA) format. A domain-specific large language model (LLM) inspired by GPT and a specialized tokenizer (PCa- LLM) for prostate cancer terminology were developed. QA models were built using the PCa-LLM and compared with those using GPT-2 as the backbone to predict recommended and final treatments. Results: For the physician-recommended treatment plans, our LLM (PCa-LLM) showed superior performance with higher AUROC scores for curative vs. non-curative treatments (0.78 vs. 0.65), chemo-hormonal vs. other non-curative treatments (0.89 vs. 0.65), and surveillance vs. all other treatments (0.72 vs. 0.70), while both models achieved the same high AUROC of 0.99 for chemo-hormonal vs. all other treatments. For final treatments, PCa-LLM demonstrated better AUROC for curative vs. non-curative treatments (0.77 vs. 0.74) and chemo-hormonal vs. other non-curative treatments (0.71 vs. 0.66), while GPT2 outperformed PCa-LLM for surveillance vs. all other treatments (0.78 vs. 0.70). Both models achieved an AUROC of 0.99 for chemo-hormonal vs. all other treatments. Conclusions: PCa-LLM accurately predicted most treatment categories better than GPT2, with higher AUROC scores, and can be utilized to triage prostate cancer patients using initial consultation notes. Task Physician -Recommended Treatment Plan Final Treatment Received Curative Prostatectomy/Radiation 228 230 Non-Curative Focal Therapy 22 15 Active Surveillance 40 45 Chemo-hormonal 30 30 Model Predictions (AUROC) GPT2 PCa-LLM GPT2 PCa-LLM Curative vs. non-curative 0.65 0.78 0.74 0.77 Chemohormonal vs. other non-curative 0.65 0.89 0.66 0.71 Chemohormonal vs. all other 0.99 0.99 0.99 0.99 Surveillance vs. other non-curative 0.64 0.60 0.67 0.59 Surveillance vs. all other 0.70 0.72 0.78 0.70

109 Background: C and Lu are associated with overall survival benefit and approved for pts with mCRPC after prior treatment with docetaxel and an androgen receptor pathway inhibitor (ARPI). However, in real-world settings, administering Lu requires coordination across specialties and resources [PMID: 36637862], which could delay its administration. Herein, we sought to compare the time from medication order to administering the first dose of C versus Lu. Methods: This study used pt level data from the nationwide Flatiron Health electronic health record (EHR)-derived de-identified database. Inclusion criteria: availability of the date of the first order of single-agent C or Lu and the date of first administration of the drugs. The time between the first medication order to the administration of single-agent C or Lu was measured. Wilcoxon rank-sum test was used to compare the median days between medication order to administration between these 2 agents and a stratified analysis by practice type and insurance was performed. Results: Of 2,329 orders, 2,045 orders were for C between 1/1/2013 and 1/26/2024, while 284 were for Lu between 8/1/2013 and 12/6/2023. The time from medication order to administration was significantly longer with Lu (median 22 days, IQR 12-43) compared to C (median 7 days, IQR 3-14, p < 0.001). Time from medication order to administration of C and Lu by practice type and insurance is summarized (Table). At the time of the first order, pts in Lu cohort, compared to C cohort, were significantly older (median age 74 vs. 72, p < 0.001), more likely to be treated at an academic practice (27% vs. 15%, p < 0.001), and more likely to be covered by a commercial health plan (80% vs. 73%, p < 0.001). Conclusions: Our results show that it takes significantly longer to administer Lu compared to C across different practice types and most insurance plans. These results could inform healthcare policies aimed at improving access to Lu and guide treatment selection in the clinic. Time from medication order to administration of C and Lu by practice type and insurance. Category Subtype (number of orders for C/number of orders for Lu) CN = 2045 LuN = 284 p-value Practice type, mean (range) (days) Academic (315/77) 1.3 (0 – 59) 9.2 (0 – 55) < 0.001 Community (1730/207) 13 (0 – 243) 44 (1 – 282) < 0.001 Insurance, mean (range) (days) Commercial (1494/226) 11 (0 – 163) 32 (0 – 215) < 0.001 Medicare/other government programs (266/46) 10 (0 – 120) 46 (0 – 282) < 0.001 Medicaid (32/2) 9 (0 – 76) 28 (0 – 55) 0.8 Others (72/6) 12 (0 – 42) 21 (0 – 41) 0.2 Unknown (181/4) 14 (0 – 243) 76 (39 – 91) < 0.001

104 Background: Androgen deprivation therapy (ADT) intensification [ADT + androgen receptor-pathway inhibitor (ARPIs) +/- docetaxel] has been shown to improve survival outcomes in pts with mHSPC and is recommended as standard of care by all major guidelines. However, prior studies have shown underutilization of ADT intensification [PMID: 39191549]. There are limited updated and large-scale studies on the use of ADT intensification, beyond January 2023. In this study, we aimed to assess the treatment patterns of ADT intensification in real-world pts with mHSPC in the US, including data on those treated since January 2023. Methods: This retrospective study utilized the nationwide Flatiron Health Electronic Health Record (EHR) derived de-identified database. Eligibility: Pts diagnosed with mHSPC and availability of treatment information (ADT and/or line of therapy in mHSPC). The data cut-off date was 5/31/2024. Treatment patterns were summarized using frequency and percentages. All analysis was done using R version 4.2.3. Results: Among 24,105 pts with metastatic prostate cancer in the database, a total of 14,084 were eligible and included. The median age at metastatic diagnosis was 72 years (IQR 65 – 79), and 60% were White non-Hispanic. In the overall cohort (1/1/2013–5/15/2024), 56.8% received ADT monotherapy, and 37.7% received ADT intensification. Table summarizes the treatment patterns by year, highlightinga consistent increase in adoption of intensified ADT regimens, and a decline in use of ADT monotherapy. Notably, since January 2023, 76.8% pts with mHSPC received ADT intensification while only 17.3% of pts received ADT monotherapy. Conclusions: To our knowledge, this is the first and largest study to evaluate the current use of ADT intensification. These data indicate a notable improvement in the adoption of level 1 evidence (ADT intensification) in the treatment of pts with mHSPC. These encouraging data emphasize the benefits of continued efforts to promote guideline-concordant care and provide the current treatment landscape to design clinical trials. Treatment patterns by year in pts with mHSPC. Treatment 2013–2017n = 5,263(%) 2018n = 1,313 (%) 2019n = 1,407(%) 2020n = 1,353(%) 2021n = 1,483(%) 2022n = 1,556(%) 2023n = 1,459(%) 2024n = 250(%) ADT monotherapy 4,095 (77.8) 856 (65.2) 819 (58.2) 664 (49) 698 (47.1) 569 (36.6) 291 (20) 5 (2) ADT + ARPI 155 (2.9) 238 (18) 354 (25.1) 452 (33) 555 (37.4) 695 (44.7) 842 (58) 154 (61.6) ADT + Docetaxel 709 (13.5) 156 (12) 175 (12.4) 156 (12) 134 (9.0) 117 (7.5) 78 (5.3) 19 (7.6) Triplet(ADT + ARPI + Docetaxel) 6 (0.1) 2 (0.2) 1 (0.1) 2 (0.1) 9 (0.6) 82 (5.2) 166 (11.1) 53 (21.2) Other therapies 298 (5.7) 61 (4.6) 58 (4.1) 79 (5.8) 87 (5.9) 93 (6) 82 (5.6) 19 (7.6)

54 Background: The incidence of colorectal cancer is increasing rapidly, particularly amongst younger people. Despite a screening program being widely available, most patients are diagnosed with cancer through symptomatic presentation. Leveraging electronic medical records (EMRs) through clinical decision support platforms offers potential for identifying high-risk individuals before clinical suspicion develops. This study aims to explore the challenges associated with early colorectal cancer diagnosis and evaluate the effectiveness of the cancer case finding feature of a cancer management platform called C the Signs. Methods: A retrospective analysis was conducted using data from the Mayo Data Platform, comprising records of 894,275 patients, including 7,348 diagnosed with colorectal cancer. C the Signs was employed to identify patients at risk of colorectal cancer based on EMR data. Sensitivity and specificity analyses were performed to assess the platform's performance in identifying high-risk patients. Additionally, the study evaluated the temporal discrepancy in colorectal cancer diagnoses between those flagged by C the Signs and those detected by primary care physicians. Results: The C the Signs screening platform had a sensitivity of 93.8% and a specificity of 19.7% in identifying patients at risk of colorectal cancer. Notably, 29.4% of patients with colorectal cancer were identified as being at risk up to 5 years earlier by C the Signs compared to diagnoses made by primary care physicians, highlighting the platform's potential for early detection. Conclusions: Early identification of high-risk individuals holds promise for improving patient outcomes and reducing the burden of colorectal cancer. Whilst the specificity of the C the Signs platform for colorectal cancer is relatively low, the sensitivity is comparable to the most favorable rates reported for colonoscopy. In addition, if C the Signs was used in conjunction with a relatively cheap screening test such as Faecal Immunochemical Testing, this would significantly drive up the specificity whilst maintaining the sensitivity. Further studies are needed to assess its efficacy in conjunction with tests available in primary care.

Abstract Objective Data extraction from the published literature is the most laborious step in conducting living systematic reviews (LSRs). We aim to build a generalizable, automated data extraction workflow leveraging large language models (LLMs) that mimics the real-world 2-reviewer process. Materials and Methods A dataset of 10 trials (22 publications) from a published LSR was used, focusing on 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n = 5) and held-out test sets (n = 17). GPT-4-turbo and Claude-3-Opus were used for data extraction. Responses from the 2 LLMs were considered concordant if they were the same for a given variable. The discordant responses from each LLM were provided to the other LLM for cross-critique. Accuracy, ie, the total number of correct responses divided by the total number of responses, was computed to assess performance. Results In the prompt development set, 110 (96%) responses were concordant, achieving an accuracy of 0.99 against the gold standard. In the test set, 342 (87%) responses were concordant. The accuracy of the concordant responses was 0.94. The accuracy of the discordant responses was 0.41 for GPT-4-turbo and 0.50 for Claude-3-Opus. Of the 49 discordant responses, 25 (51%) became concordant after cross-critique, increasing accuracy to 0.76. Discussion Concordant responses by the LLMs are likely to be accurate. In instances of discordant responses, cross-critique can further increase the accuracy. Conclusion Large language models, when simulated in a collaborative, 2-reviewer workflow, can extract data with reasonable performance, enabling truly “living” systematic reviews.

Introduction Symptom burden in myeloproliferative neoplasms (MPN) is quantified using total symptom scores (TSS) calculated from the patient-reported scores for clinically meaningful symptoms. The impact of demographic, clinical, and laboratory features on these symptoms remains unclear and warrants further investigation. Methods Patients with polycythemia vera (PV), essential thrombocythemia (ET), and myelofibrosis (MF) were identified from the retrospective chart review. TSS, scores of individual symptoms (fatigue, early satiety, abdominal discomfort, inactivity, concentration difficulties, fever, night sweats, itching, bone pain, weight loss), demographic characteristics (race, ethnicity, age, and gender), clinical features (time since diagnosis, depression status, obesity status), laboratory results and season at the time of visit when the first TSS assessment was performed were recorded. Multicollinearity and homoscedasticity were assessed using variance inflation factors and visual inspection of the data, respectively. A multivariable regression analysis was performed using a robust variance estimator. A p-value &lt;0.05 indicated a statistically significant association with the symptom scores. Results The chart review identified 557 (PV: 201; ET: 191; MF: 165) patients. The median age of patients with PV, ET, and MF was 65 (range: 27-93), 65 (20-96), and 69 (27-96) years, respectively. Most patients were white (PV: 92%; ET: 90%; MF: 93%) and non-Hispanic or Latino (PV: 93%; ET: 96%; MF: 92%). Most patients with PV and ET were female (PV: 51%; ET: 73%), while most patients with MF were male (55%). Among the included patients 22% (PV: 21%; ET: 27%; MF: 18%) had the diagnosis of depression at the time of the clinical visit. TSS was highest in MF (mean (SD): 18.1±14.6), followed by PV (15.7±14.5) and ET (15.1±14.0) patients. Fatigue was the most reported symptom by the patients (PV: 81%; ET: 82%; MF: 84%), whereas weight loss was the least reported symptom (PV: 4.5%; ET: 6.3%; MF: 16%). With regards to the TSS multivariable regression analysis showed that age (beta coefficient: 0.2; p=0.01), depression (7.2; 0.008), and low hemoglobin (1.2; 0.003) in PV, depression (11.7; 0.001) in ET, and depression (9.5; 0.001), splenomegaly (6.9; 0.01), and WBC count (0.16; 0.048) in MF patients were significantly associated with TSS. In terms of individual symptoms in PV, depression was associated with fatigue (beta coefficient: 1.31; p-value: 0.01) and concentration difficulties (1.7; &lt;0.001), high WBC count was associated with early satiety (0.09; 0.03), and non-white race was associated with fever (1.4; 0.007). In ET, depression was associated with fatigue (1.6; &lt;0.001), abdominal discomfort (1.4; &lt;0.001), inactivity (2.5; &lt;0.001), concentration difficulties (2.4; &lt;0.001) and night sweats (1.4; &lt;0.001), anemia was associated with fatigue (1.2; 0.03), fewer months since diagnosis was associated with inactivity (0.01; 0.02) and young age was associated with bone pain (0.02; 0.03). In MF, depression was associated with fatigue (1.8; 0.001), concentration difficulties (1.3; 0.01), fever (1.1; 0.02), and night sweats (2.2; 0.03), low hemoglobin was associated with fatigue (0.46; &lt;0,001) and early satiety (0.31; 0.003), high WBC count was associated with fatigue (0.03; 0.04) and early satiety (0.04; 0.02), low platelet count was associated with abdominal discomfort (0.002; 0.03), splenomegaly was associated with inactivity (2.27; 0.003) and female gender was associated with itching (0.57; 0.01). Conclusions This study demonstrates that depression in PV and ET and depression, low hemoglobin, and a high WBC count in MF contribute to multiple symptoms seen in these patients. Identifying demographic groups at risk of a higher symptom burden and addressing the comorbidities that drive symptoms, in addition to disease-directed therapy, is crucial to improving the quality of life in these patients.

Introduction Prognostication in myelofibrosis (MF) currently relies on manual assessment of clinical, pathological, and laboratory variables, which is inefficient and error-prone. The potential of large language models (LLMs) and rule-based programming to automate data extraction and prognostication in these patients remains unexplored and merits investigation. Methods Patients with MF were identified from a retrospective chart review. Data regarding clinical (age, constitutional symptoms, transfusion requirements), laboratory (hemoglobin, circulating white blood cell, platelet, and blast counts), and pathological (grade of bone marrow fibrosis, chromosomal analyses, cytogenetics) features was extracted from the latest free-text clinical notes and pathology reports using the generative pretrained transformer-4 (GPT-4) model in a zero-shot setting with the prompting parameters set to generate deterministic responses. The patients were randomly sampled into development (15%) and test (85%) sets, with the documents of patients in the development set used for iterative prompt engineering. The finalized prompts were used for data extraction in the test set. Rule-based programming was used for post-processing of the extracted data and ascertaining prognosis via Dynamic International Prognostic Scoring System (DIPSS), DIPSS-plus (DIPSS+), and Mutation-enhanced International Prognostic Scoring System-70 (MIPSS-70). Automatically extracted data and prognostic categories were compared with the manually generated human annotations. Mean accuracy, precision, and recall with 95% confidence intervals (CI) were computed. Detailed error and sensitivity analyses were conducted to assess performance. Results The study included 100 patients (development: 15; test: 85) with 100 clinical notes (development: 15; test: 85) and 90 pathology reports (development: 13; test: 77). The mean age of these patients was 70 (SD: 10.6); 55 (55%) patients were men, and 93 (93%) patients were White. In the development set, the mean accuracy, precision, and recall for data extraction from clinical notes were 94% (95% CI: 90-98%), 92% (84-99%), and 99% (97-100%). For data extraction from pathology reports, the mean accuracy, precision, and recall were 96% (93-100%), 98% (92-100%), and 83% (63-100%). For prognostication, the mean accuracy, precision, and recall were 87% (79-94%), 80% (73-87%), and 100% using DIPSS (accuracy: 80%; precision: 73%; recall: 100%), DIPSS+ (accuracy: 93%; precision: 83%; recall: 100%) and MIPSS-70 (accuracy: 87%, precision: 83%; recall: 100%). In the test set, the mean accuracy, precision, and recall for data extraction from clinical notes were 96% (95% CI: 93-98%), 91% (87-96%), and 98% (97-100%). For data extraction from pathology reports, the mean accuracy, precision, and recall were 95% (92-98%), 93% (87-99%), and 90% (80-100%). For prognostication the mean accuracy, precision, and recall were 95% (94-96%), 89% (88-90%) and 99% (96-100%) using DIPSS (accuracy: 94%; precision: 89%; recall: 100%), DIPSS+ (accuracy: 95%; precision: 88%; recall: 95%) and MIPSS-70 (accuracy: 94%, precision: 90%; recall: 100%). Error analyses identified incorrect extractions by GPT-4 were related to the circulating blasts and constitutional symptoms variables from the clinical notes. Sensitivity analyses excluding the associated variables improved overall accuracy from 94% to 97% and 96% to 97% in the development and test sets, respectively. Similarly, incorrect extractions from the pathology reports were related to the CALR gene mutation variables. Sensitivity analyses excluding the associated variables improved the overall accuracy from 96% to 98% and 95% to 97% in the development and test sets, respectively. Conclusion LLMs and rule-based programming can accurately extract data from the electronic health records and prognosticate MF patients via DIPSS, DIPSS+, and MIPSS-70. The automated approach could significantly streamline clinical and research workflows, providing an efficient and less resource-intensive alternative to manual prognostication.

4591 Background: EV-302 and CheckMate 901 demonstrated significant survival benefit with enfortumab vedotin-pembrolizumab (EVPembro) and gemcitabine-cisplatin-nivolumab (GemCisNivo) as 1L therapy, respectively, in patients (pts) with la/mUC. However, whether these treatments are cost-effective or not remains unclear. Methods: A three-state Markov model (progression-free, progression and death) was developed. State utilities were derived from published literature. State transition probabilities were informed from point-probabilities and hazard ratios for OS and PFS obtained from the latest follow-up of eligible trials. Average sales price (2024 USD) for individual treatments (excluding ancillary charges) were obtained from the center of medicare and medicaid services (US payer’s perspective). Treatment strategies were modeled in accordance with the dose/schedule reported in the eligible trials. Pts who received GemCis or Gem-carboplatin (GemCarbo) and did not progress were modeled to receive Avelumab maintenance. Half-cycle corrected costs and utilities were accrued over a 10-year lifetime horizon and were discounted at 3%. Monte Carlo simulation was used to estimate quality adjusted life years (QALYs) and incremental cost-effectiveness ratios (ICERs). A willingness-to-pay threshold (WTP) of 100,000$ per QALY was used. Results: In cisplatin-eligible pts, GemCisNivo demonstrated a 0.8 QALY gain over GemCis at an ICER of 21,127 ($/QALY). GemCisNivo was also associated with an incremental net monetary benefit (INMB) of 63,099$ relative to GemCis. EVPembro demonstrated a 2.4 QALY gain over GemCis at an ICER of 915,283 and a 1.6 QALY gain over GemCisNivo at an ICER of 1,406,308. In cisplatin-ineligible pts, EVPembro demonstrated a 2.4 QALY gain over Gem-carboplatin (GemCarbo) at an ICER of 934,761. Price threshold analyses demonstrated that EVPembro is likely to be cost-effective at a cost of 12,344$ for cisplatin-eligible and 13,374$ per cycle for cisplatin-ineligible pts (original cost: 35,665.60$ per cycle). Sensitivity analysis using a total of 12 cycles for EVPembro showed cost-effectiveness at an ICER of 69,594 relative to GemCisNivo in cisplatin-eligible pts, and at an ICER of 53,099 relative to GemCarbo in cisplatin-ineligible pts. Conclusions: In terms of US payer’s perspective, GemCisNivo is most likely to be cost-effective in cisplatin-eligible pts at the WTP of 100,000 $/QALY. Limiting the number of cycles for EVPembro may render it as a cost-effective 1L treatment in la/mUC pts regardless of cisplatin-eligibility status.