Raina, Ammad

Abstract Objective Data extraction from the published literature is the most laborious step in conducting living systematic reviews (LSRs). We aim to build a generalizable, automated data extraction workflow leveraging large language models (LLMs) that mimics the real-world 2-reviewer process. Materials and Methods A dataset of 10 trials (22 publications) from a published LSR was used, focusing on 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n = 5) and held-out test sets (n = 17). GPT-4-turbo and Claude-3-Opus were used for data extraction. Responses from the 2 LLMs were considered concordant if they were the same for a given variable. The discordant responses from each LLM were provided to the other LLM for cross-critique. Accuracy, ie, the total number of correct responses divided by the total number of responses, was computed to assess performance. Results In the prompt development set, 110 (96%) responses were concordant, achieving an accuracy of 0.99 against the gold standard. In the test set, 342 (87%) responses were concordant. The accuracy of the concordant responses was 0.94. The accuracy of the discordant responses was 0.41 for GPT-4-turbo and 0.50 for Claude-3-Opus. Of the 49 discordant responses, 25 (51%) became concordant after cross-critique, increasing accuracy to 0.76. Discussion Concordant responses by the LLMs are likely to be accurate. In instances of discordant responses, cross-critique can further increase the accuracy. Conclusion Large language models, when simulated in a collaborative, 2-reviewer workflow, can extract data with reasonable performance, enabling truly “living” systematic reviews.

Introduction Prognostication in myelofibrosis (MF) currently relies on manual assessment of clinical, pathological, and laboratory variables, which is inefficient and error-prone. The potential of large language models (LLMs) and rule-based programming to automate data extraction and prognostication in these patients remains unexplored and merits investigation. Methods Patients with MF were identified from a retrospective chart review. Data regarding clinical (age, constitutional symptoms, transfusion requirements), laboratory (hemoglobin, circulating white blood cell, platelet, and blast counts), and pathological (grade of bone marrow fibrosis, chromosomal analyses, cytogenetics) features was extracted from the latest free-text clinical notes and pathology reports using the generative pretrained transformer-4 (GPT-4) model in a zero-shot setting with the prompting parameters set to generate deterministic responses. The patients were randomly sampled into development (15%) and test (85%) sets, with the documents of patients in the development set used for iterative prompt engineering. The finalized prompts were used for data extraction in the test set. Rule-based programming was used for post-processing of the extracted data and ascertaining prognosis via Dynamic International Prognostic Scoring System (DIPSS), DIPSS-plus (DIPSS+), and Mutation-enhanced International Prognostic Scoring System-70 (MIPSS-70). Automatically extracted data and prognostic categories were compared with the manually generated human annotations. Mean accuracy, precision, and recall with 95% confidence intervals (CI) were computed. Detailed error and sensitivity analyses were conducted to assess performance. Results The study included 100 patients (development: 15; test: 85) with 100 clinical notes (development: 15; test: 85) and 90 pathology reports (development: 13; test: 77). The mean age of these patients was 70 (SD: 10.6); 55 (55%) patients were men, and 93 (93%) patients were White. In the development set, the mean accuracy, precision, and recall for data extraction from clinical notes were 94% (95% CI: 90-98%), 92% (84-99%), and 99% (97-100%). For data extraction from pathology reports, the mean accuracy, precision, and recall were 96% (93-100%), 98% (92-100%), and 83% (63-100%). For prognostication, the mean accuracy, precision, and recall were 87% (79-94%), 80% (73-87%), and 100% using DIPSS (accuracy: 80%; precision: 73%; recall: 100%), DIPSS+ (accuracy: 93%; precision: 83%; recall: 100%) and MIPSS-70 (accuracy: 87%, precision: 83%; recall: 100%). In the test set, the mean accuracy, precision, and recall for data extraction from clinical notes were 96% (95% CI: 93-98%), 91% (87-96%), and 98% (97-100%). For data extraction from pathology reports, the mean accuracy, precision, and recall were 95% (92-98%), 93% (87-99%), and 90% (80-100%). For prognostication the mean accuracy, precision, and recall were 95% (94-96%), 89% (88-90%) and 99% (96-100%) using DIPSS (accuracy: 94%; precision: 89%; recall: 100%), DIPSS+ (accuracy: 95%; precision: 88%; recall: 95%) and MIPSS-70 (accuracy: 94%, precision: 90%; recall: 100%). Error analyses identified incorrect extractions by GPT-4 were related to the circulating blasts and constitutional symptoms variables from the clinical notes. Sensitivity analyses excluding the associated variables improved overall accuracy from 94% to 97% and 96% to 97% in the development and test sets, respectively. Similarly, incorrect extractions from the pathology reports were related to the CALR gene mutation variables. Sensitivity analyses excluding the associated variables improved the overall accuracy from 96% to 98% and 95% to 97% in the development and test sets, respectively. Conclusion LLMs and rule-based programming can accurately extract data from the electronic health records and prognosticate MF patients via DIPSS, DIPSS+, and MIPSS-70. The automated approach could significantly streamline clinical and research workflows, providing an efficient and less resource-intensive alternative to manual prognostication.

4591 Background: EV-302 and CheckMate 901 demonstrated significant survival benefit with enfortumab vedotin-pembrolizumab (EVPembro) and gemcitabine-cisplatin-nivolumab (GemCisNivo) as 1L therapy, respectively, in patients (pts) with la/mUC. However, whether these treatments are cost-effective or not remains unclear. Methods: A three-state Markov model (progression-free, progression and death) was developed. State utilities were derived from published literature. State transition probabilities were informed from point-probabilities and hazard ratios for OS and PFS obtained from the latest follow-up of eligible trials. Average sales price (2024 USD) for individual treatments (excluding ancillary charges) were obtained from the center of medicare and medicaid services (US payer’s perspective). Treatment strategies were modeled in accordance with the dose/schedule reported in the eligible trials. Pts who received GemCis or Gem-carboplatin (GemCarbo) and did not progress were modeled to receive Avelumab maintenance. Half-cycle corrected costs and utilities were accrued over a 10-year lifetime horizon and were discounted at 3%. Monte Carlo simulation was used to estimate quality adjusted life years (QALYs) and incremental cost-effectiveness ratios (ICERs). A willingness-to-pay threshold (WTP) of 100,000$ per QALY was used. Results: In cisplatin-eligible pts, GemCisNivo demonstrated a 0.8 QALY gain over GemCis at an ICER of 21,127 ($/QALY). GemCisNivo was also associated with an incremental net monetary benefit (INMB) of 63,099$ relative to GemCis. EVPembro demonstrated a 2.4 QALY gain over GemCis at an ICER of 915,283 and a 1.6 QALY gain over GemCisNivo at an ICER of 1,406,308. In cisplatin-ineligible pts, EVPembro demonstrated a 2.4 QALY gain over Gem-carboplatin (GemCarbo) at an ICER of 934,761. Price threshold analyses demonstrated that EVPembro is likely to be cost-effective at a cost of 12,344$ for cisplatin-eligible and 13,374$ per cycle for cisplatin-ineligible pts (original cost: 35,665.60$ per cycle). Sensitivity analysis using a total of 12 cycles for EVPembro showed cost-effectiveness at an ICER of 69,594 relative to GemCisNivo in cisplatin-eligible pts, and at an ICER of 53,099 relative to GemCarbo in cisplatin-ineligible pts. Conclusions: In terms of US payer’s perspective, GemCisNivo is most likely to be cost-effective in cisplatin-eligible pts at the WTP of 100,000 $/QALY. Limiting the number of cycles for EVPembro may render it as a cost-effective 1L treatment in la/mUC pts regardless of cisplatin-eligibility status.

e19534 Background: Immunomodulatory drugs, proteasome inhibitors plus dexamethasone are the standard triplet induction regimen for TENDMM. We analyzed the efficacy of adding either anti-CD38 monoclonal antibodies (mAbs) or anti-SLAMF7 antibody to the triplet therapy. Methods: Ovid MEDLINE, EMBASE and published abstracts were systematically searched from each database’s inception through 01-31-2024 to identify II/III trials assessing quadruplet therapy versus triplet therapy in TENDMM patients. Hazard ratios (HR) for progression-free survival (PFS) and overall survival (OS), odds ratios (OR) for overall response rates (ORR), measurable residual disease (MRD) negativity rate (with sensitivity of 10 -5) and grade 3 or higher treatment-related adverse events (G≥3 TRAE) were summarized using random effects meta-analysis via inverse variance approach. Additional subgroup analyses based on the cytogenetic risk were performed. P-value of interaction (Pint ) < 0.1 was pre-specified as statistically significant for the difference between the two groups. Results: 3806 studies were screened and seven trials were included with a total of 3546 patients. Four trials (2303 patients) assessed the addition of anti-CD-38 mAbs and three (1243 patients) assessed anti-SLAMF7 mAb, elotuzumab to the triplet regimen. Quadruplet therapy significantly improved PFS as compared to triplet (HR: 0.63; 95% CI: 0.43-0.92), and the PFS benefit was also observed in standard risk group (HR 0.5; 95% CI 0.2-0.9); however, results were insignificant for high-risk group (HR 0.8; 95% CI 0.62-1.93). No significant OS benefit was observed (HR: 0.79; 95% CI 0.49–1.26); however, data for three trials (CASSIOPEIA, DSMM XVII, PERSEUS) is immature, with longer-term follow-up ongoing. Quadruplet therapy showed significant results for very good partial response or better (OR: 1.7; 95% CI 1.2-2.36), stringent complete response (OR: 1.74; 95% CI 1.27-2.07) and complete response or better (OR: 1.91; 95% CI 1.19-3.06). MRD rate shows benefit with quadruplet therapy (OR: 2.59; 95% CI 2.22-3.01). Improved MRD rate was also observed for high-risk (OR 2.13; 95% CI 1.4-3.2) and standard-risk group (OR 2.58; 95% CI 1.68-3.94). Secondary analysis for efficacy of anti-CD38 mAbs showed benefit for PFS (HR: 0.44; 95% CI: 0.35-0.56), ORR (OR: 1.77;95% CI: 1.02-3.06), and MRD (OR: 2.98; 95% CI 1.8-4.8). G ≥3 TRAE did not significantly differ between the groups (OR: 1.05; 95% CI 0.97-1.12). However, quadruplet therapy showed significant risk of infection (OR: 1.2; 95% 1.05-1.35) and thrombocytopenia (OR: 1.41; 95% CI 1.24-1.59). Conclusions: Quadruplet therapy improved PFS, ORR, and MRD rates compared to triplet regimen and has demonstrated promising benefits to be considered the new standard of care for TENDMM patients.

e17059 Background: It is unclear if patients with different HRR gene-mutated mCRPC derive consistent benefit from treatment with PARPi and API combination. Approval labels for PARPi+API are inconsistent across regulatory bodies in the US (Food and Drug Administration, FDA) and Europe (European Medical Agency, EMA), adding to confusion among practicing clinicians. Therefore, we aimed to assess the differential efficacy of PARP+API therapy by different genes using up-to-date trial evidence. Methods: We maintain a living meta-analysis on mCRPC treatments that is updated when new data becomes available. Here, we report data from phase III randomized control trials (RCTs) assessing PARPi+API in 1L mCRPC. The outcomes of interest included radiographic progression-free survival (rPFS) and overall survival (OS). Hazard ratios (HR) for rPFS and OS by different genes were summarized using random effects meta-analysis via inverse variance approach. Results: We have screened 27376 references since the inception of this living review. Here, we report data from three trials (15 references) with a total of 1618 patients. Frequently reported HRR gene mutations in the included trials were BRCA2 (35%), ATM (21%), CDK12 (12.2%), CHEK2 (12.2%), BRCA1 (4.6%), and PALB2 (3.4%). In terms of rPFS, statistical benefit was observed in BRCA (0.28; 95% CI: 0.13-0.59) and CDK12 (0.58; 95% CI: 0.35-0.95) subgroups, whereas no statistically significant benefit was seen in PALB2 (0.53; 95% CI: 0.21-1.32), ATM (0.93; 95% CI, 0.57-1.53), and CHEK2 (0.92; 95% CI: 0.53-1.61) subgroups. For OS, benefit was observed in BRCA (0.47; 95%CI: 0.31-0.71) subgroup. While not statistically significant, a signal of meaningful OS benefit was observed in PALB2 (0.33; 95%CI: 0.10-1.16) but no OS benefit was observed in ATM (0.97; 95% CI: 0.57-1.67), CDK12 (0.80; 95% CI: 0.36-1.78), and CHEK2 (0.81; 95% CI: 0.37-1.75) subgroups. Conclusions: Current results show that mCRPC patients with BRCA and CDK12 alterations had delayed disease progression with PARPi+API as 1L therapy. BRCA mutations are also associated with improved OS, while patients with CHEK2 and ATM derived no significant benefit with PARPi+API combination. [Table: see text]

e16579 Background: Recent results from EV 302 and CheckMate 901 have transformed the treatment landscape of untreated locally advanced/metastatic urothelial cancer (Ia/mUC). Here, we present the most up-to-date evidence of comparative efficacy and safety of first line (1L) treatment options in Ia/mUC with emphasis on Enfortumab vedotin (EV)-pembrolizumab (P) and gemcitabine (Gem)-cisplatin (Cis)-nivolumab (N). Methods: This living systematic review used the living interactive evidence (LIvE) synthesis framework. Full-text and abstract publications of phase II/III randomized clinical trials comparing 1L immunotherapy combined with chemotherapy (IO+C) or immunotherapy alone (IO) regimens with platinum-based chemotherapy (Plat) in Ia/mUC were included. Outcomes of interest included overall survival (OS), progression-free survival (PFS) and treatment-related grade 3 or greater adverse events (≥G3 TRAE). Mixed treatment comparisons were made using network meta-analysis. P-scores were computed to assess relative treatment rankings. Results: A total of 5 trials (8 references) with 4,749 patients and 9 unique treatments were included. EV-P and Gem-Cis-N were ranked first and second in terms of OS and PFS improvement. EV-P improved OS compared to Gem-Cis-N (HR: 0.60; 0.45-0.81), Gem-Plat-atezolizumab(A) (0.55; 0.42-0.72), durvalumab(D)- tremelimumab(T) (0.55; 0.42-0.72), Gem-Plat-P (0.55; 0.42-0.72), P (0.51; 0.39-0.68), D (0.47; 0.36-0.63), A (0.48; 0.36-0.63) and Gem-Plat (0.47; 0.38-0.58). EV-P also improved PFS compared to Gem-Cis-N (0.63; 0.48-0.82), Gem-Plat-A (0.55; 0.43-0.69), D-T (0.42; 0.33-0.53), Gem-Plat-P (0.58; 0.45-0.74), P (0.34; 0.26-0.44), D (0.35; 0.27-0.44) and Gem-Plat (0.45; 0.38- 0.54). For ≥G3 TRAE, A was ranked first with the lowest adverse events compared to others. EV-P (rank 5) had significantly fewer events compared to Gem-Plat, Gem-Plat-A, Gem-Cis-N and Gem-Plat-P. Gem-Cis-N was ranked last with significantly greater events compared to A, P, D, D-T, Gem-Plat, EV-P, and Gem-Plat-A (Table with ranks [p-scores]). Conclusions: Current evidence suggests that patients with untreated la/mUC experience improved survival and slower disease progression with EV-P when compared to other contemporary options. Gem-Cis-N may be a reasonable alternative in patients who are cisplatin-eligible. Cost and access to these treatments are crucial and should be considered to facilitate shared decision making. [Table: see text]

412 Background: Previously we have reported comparative effectiveness data regarding survival of first line (1L) treatment options in mRCC. However, it is not known how the life-prolonging frontline combinations compare to patient-reported outcomes. Methods: Electronic databases including MEDLINE and EMBASE were searched from each database’s inception through July 1st 2023. Phase III randomized controlled trials (RCTs) assessing 1L immune-checkpoint inhibitor (ICI) combination therapies for mRCC and reporting HrQoL were included. Small sample size adjusted standardized mean differences (SMD; Hedges’ g) with 95% confidence intervals (CI) were computed for global, physical, functional, and emotional QoL. Positive SMD indicated an improvement. A network meta-analysis was conducted to assess the comparative HrQoL across different treatment options. P-scores were computed to assess relative treatment rankings. Results: Of 5770 citations identified, five trials (nine references) were included in this analysis. Patient-reported outcomes have not been reported in the JAVELIN Renal 101 or COSMIC-313 trials yet. Global QoL was significantly improved with cabozantinib-nivolumab (CaboNivo) as compared to Nivo-ipilimumab (SMD 2.74, 95% CI 2.54; 2.93), lenvatinib-pembrolizumab (LenPem; 2.79, 2.53; 3.06), sunitinib (2.87, 2.72; 3.03), Pem-axitinib (PemAxi; 3.09, 2.89; 3.30) and atezolizumab-bevacizumab (AteBev; 3.24, 3.04; 3.45). Global QoL improvement was also observed with NivoIpi compared to sunitinib (0.14, 0.02; 0.25), PemAxi (0.36, 0.18; 0.54) and AteBev (0.51, 0.33; 0.68). Physical QoL improvement were consistent with CaboNivo. Likewise, LenPem significantly improved QoL compared to sunitinib (0.26, 0.04; 0.47), PemAxi (0.38, 0.13; 0.64) and NivoIpi (0.51, 0.27; 0.76). Significant functional QoL benefits were only observed with CaboNivo (0.32, 0.17; 0.48) and NivoIpi (0.19, 0.08; 0.31) compared to sunitinib. No significant mixed treatment comparisons were observed for emotional QoL, however, currently available data suggests that LenPem (rank 1) may potentially improve QoL (Table). Conclusions: CaboNivo and NivoIpi as 1L therapy may offer an improved overall HrQoL compared to other contemporary treatment options in mRCC. [Table: see text]

230 Background: Elevated suicide risk is established in cancer patients compared with the general population. We aim to investigate the risk of SRD among individuals diagnosed with mGU cancers. Methods: Surveillance, Epidemiology, and End Results (SEER) database (2004-2020) was queried to obtain data on suicide death in adults (>20 years of age) diagnosed with mGU cancers including bladder cancer (BC), kidney cancer (KC), and prostate cancer (PC). Standardized mortality ratios (SMRs) with 95% confidence interval (CI) were computed overall and by latency period of <1, 1-5, and >5 years (y) after the initial diagnosis of mGU cancers. SMR >1 indicated increased while <1 indicated decreased SRD. Stratification was done by age (<50y, 50-70y, >70y), race (White, Black, Asian, Native American), ethnicity (Hispanic, non-Hispanic), chemotherapy, radiotherapy, surgery, and marital. Results: This analysis included 87,357 mGU patients. In overall population, the risk of suicide-related deaths was highest <1y after diagnosis (SMR: 3.29, 95% CI: 2.47-4.29) followed by 1-5y (2.61, 2.0-3.35) and >5y (1.27, 0.61-2.34) after diagnosis. With respect to race, SRD was significantly higher in Whites (3.31, 2.46-4.36) but not in Blacks (2.58, 0.31-9.33), and Asians (2.03 (0.05-11.3). In ethnic subgroups, non-Hispanics were observed to have increased SRD (3.71, 2.77-4.86) but not Hispanics (0.83, 0.1-3.0). Patients who received chemotherapy (3.74, 2.10-6.18) or those who did not receive radiotherapy (3.66, 2.68-4.89) were more likely to die from suicide. In terms of marital status, single patients had a higher SRD (4.39, 2.94-6.30) compared to those who were married (2.54, 1.65-3.76). Detailed results are outlined in the Table. Conclusions: Patients with GU malignancies are at increased risk of SRD. Targeted and timely interventions in highest risk subgroups such as Non-Hispanic patients with a recent cancer diagnosis with no partner support, may reduce the risk of SRD in patients diagnosed with mGU cancers. [Table: see text]

173 Background: The data for combining PARPi and ARPi in mCRPC have shown conflicting results. Here, we report a quantitative summary of benefits by clinically meaningful subgroups (HRR mutation status, BRCA mutation status) to guide the use of PARPi and ARPi combination in mCRPC setting. Methods: MEDLINE, EMBASE, and CENTRAL were systematically searched from each database’s inception through September 7, 2023. Phase III studies comparing PARPi + ARPi with ARPi + placebo in mCRPC were included. Radiographic progression-free survival (rPFS) and overall survival (OS) in patient subgroups based on HRR and BRCA mutation status were assessed. Hazard ratios for rPFS and OS with 95% CI were collected at the level of each trial, and a random-effects meta-analysis was conducted using an inverse variance approach. Subgroup differences were assessed and p-value of interaction <0.1 was considered statistically significant. Results: The literature search identified 4976 studies from which 3 trials (11 references) – TALAPRO2, PROpel and MAGNITUDE met the inclusion criteria. A total of 2254 participants were eligible for analysis. PARPi + ARPi prolonged rPFS in patients with HRR mutations (HR: 0.55, 95%CI: 0.38-0.78) as well as in patients without HRR mutations (HR: 0.74, 95%CI: 0.63-0.88). There is no evidence of treatment interaction by HRR mutation status (p = 0.14). In patients harboring HRR mutations, improved rPFS was observed in those with BRCA mutations (HR: 0.28, 95%CI: 0.13-0.62) as well as those without BRCA mutations (HR: 0.72, 95%CI: 0.59-0.87). However, there is evidence that patients with BRCA mutations derived greater rPFS benefit as compared to patients without BRCA mutations (p =0.02). PARPi + ARPi showed no significant improvement in overall survival either in patients with HRR mutations (HR: 0.79, 95%CI: 0.59-1.05) or in patients without HRR mutations (HR: 0.92, 95%CI: 0.74-1.14). Conclusions: Genomic testing for HRR genes is mandatory to guide clinical decision making in mCRPC patients. Patients harboring BRCA mutations are most likely to derive clinically meaningful benefit from combination PARPi + ARPi treatment. [Table: see text]

111 Background: Comparative evidence of quality of life (QoL) outcomes regarding life-prolonging treatments for the management of first-line treatment of metastatic castration sensitive prostate cancer (mCSPC) is not known. Methods: This systematic review uses the living interactive evidence synthesis framework (PMID: 36862387). Phase III randomized controlled trials assessing contemporary systemic treatments in mCSPC were included. Small sample size adjusted standardized mean differences (SMD; Hedges’ g) with 95% confidence intervals (CI) were computed for global QoL at 3, 6, 12, and 24 mo. Positive SMD indicated better health-related QoL, with zero indicating a null effect. Mixed treatment comparisons were made using a network meta-analysis. P-scores were computed; a higher score indicated better QoL. Results: Of 10 clinical trials initially included, eight trials (15 references) reported QoL. At 3mo global QoL was improved with abiraterone (AAP)+ADT compared to D+ADT (SMD: 0.26; 95% CI: 0.13-0.39), enzalutamide (E)+ADT (0.18; 0.03-0.33), and ADT (0.14; 0.02-0.27). At 6mo, AAP+ADT significantly improved global QoL compared to D+ADT (0.74; 0.62-0.87), E+ADT (1.74; 1.56-1.92), and ADT (1.79; 1.65-1.93). D+ADT also improved global QoL at 6mo compared to ADT (1.05; 0.93-1.16) but not when compared to E+ADT. At 12mo, AAP+ADT improved global QoL compared to E+ADT (0.20; 0.05-0.34) and ADT (0.19; 0.07-0.31) but not compared to D+ADT (0.05; -0.07-0.17). D+ADT also improved global QoL at 12mo compared to ADT (0.14; 0.03-0.24) (Table). Global QoL has not been reported for ARASENS trial at this time. These results are available on the Living interactive website. Conclusions: The totality of evidence suggests improved global QoL with androgen-pathway inhibitors (API) doublet therapy. D+ADT may offer better QoL at longer 6 and 12 months of follow-up. QoL data for triplet therapy is still emerging and will be incorporated in the living review as soon as it becomes available. [Table: see text]