Citations

3624 Background: Increasing chemotherapy completion and reducing chemotherapy adverse events (AE) are critical to improve survival after colon cancer diagnosis. Sex, as a biological variable, may impact chemotherapy treatment differently, and thus further investigation is needed to examine sex differences in chemotherapy completion and adverse events among patients with colon cancer. Methods: Among an NCI-sponsored trial conducted among patients with stage III colon cancer (CALGB/SWOG 80702), all patients received standard adjuvant chemotherapy FOLFOX (fluorouracil, leucovorin, and oxaliplatin). To signal chemotherapy completion, we utilized relative dose intensity (RDI) calculated as the ratio of the delivered dose intensity to planned dose intensity; and reduced RDI (RDI <85%) was considered as a clinically significant deviation from standard FOLFOX. From clinicians’ records of NCI’s Common Terminology Criteria for Adverse Events (CTCAE), we primarily focused on clinically significant AE such as neutrophils decrease, nausea, platelets decrease, hypertension, peripheral neuropathy, diarrhea, fatigue, gastritis, creatinine increase, gastric ulcer, myocardial ischemia, and cerebral ischemia; and severe AE was defined as the occurrence of any above AE with CTCAE grade ≥3. Using multivariable logistic regression that adjusted for body surface area (BSA) and other clinicopathological confounders, we estimated adjusted odds ratios (OR) for the associations of sex with reduced RDI and severe AE. Results: Of 2201 patients, mean (standard deviation [SD]) age was 60.9 (10.9) years, 1019 (46.3%) were female, 1750 (79.5%) were White, 172 (7.8%) were Hispanic, 964 (43.8%) experienced reduced RDI, and 1156 (52.5%) had severe adverse events. Compared to males, females were at significantly higher risks of experiencing reduced RDI (OR [95% CI]: 1.57 [1.27-1.93], P <0.001) and severe AE (OR [95% CI]: 1.73 [1.41-2.12], P <0.001). Conclusions: Our findings suggested females are more likely than men to experience reduced RDI and severe AE during colon cancer chemotherapy. Clinical Impact: In the era of precision medicine, sex (as a biological variable) should be considered in optimizing colon cancer chemotherapy to improve completion and reduce toxicities. Clinical trial information: NCT01150045 .

Abstract Infection is increasingly recognized as a significant cause of morbidity and mortality in patients with relapsed/refractory (R/R) large B-cell lymphoma (LBCL) receiving CD19 chimeric antigen receptor (CAR) T-cell therapy. The current study analyzed the natural history, risk factors, and outcomes of infection in 3350 patients with R/R LBCL receiving commercial CD19 CAR T-cell therapy (n = 2804 axicabtagene ciloleucel [axi-cel], n = 546 tisagenlecleucel) from December 2017 to June 2022. Infection developed in 834 patients (24.9%) within 100 days after infusion, resulting in an infection density of 0.43 per 100 patient days and a 100-day cumulative incidence of 22%. Bacterial, viral, and fungal infections were recorded in 527 (15.7%), 374 (11.2%), and 108 patients (3.2%), respectively, with corresponding infection densities of 0.23, 0.15, and 0.04 per 100 patient days. After a 24-month median follow-up, 1482 patients (44%) had died, with infection as the primary cause in 173 cases (12%). The 100-day infection-related mortality (IRM) was 1.6% (95% confidence interval, 1.2-2.0). Patients with a Karnofsky performance score of ≤80, infection history before CAR T-cell therapy, axi-cel therapy, severe cytokine release syndrome (grade ≥3), and severe immune effector cell–associated neurotoxicity syndrome (grade ≥3) had increased infection risk. Infections within 100 days were an independent risk factor for inferior overall survival beyond day 100 after CD19 CAR T-cell therapy. In conclusion, study results show a significant incidence of infection and IRM in patients with R/R LBCL treated with CD19 CAR T-cell therapy. Furthermore, results identify patients at a heightened risk of infection, offering insights to guide potential interventions aimed at mitigating infection and improving patient outcomes after CAR T-cell therapy.

6037 Background: The incidence of HPV-associated head and neck squamous cell carcinoma (HNSCC) continues to rise with over 90% of cases being driven by HPV16. Median overall survival (OS) with pembrolizumab in first-line recurrent/metastatic (R/M) HNSCC is 12.3 months in subjects with CPS ≥1, 10.8 months for CPS ≥1-19, and 14.9 months for CPS ≥20. There is an urgent need to improve survival rates in the growing population of HPV16-positive R/M HNSCC. PDS0101 (Versamune HPV) is an investigational T cell stimulating immunotherapy that unleashes a potent, durable attack against HPV16-positive cancers and is being studied in combination with pembrolizumab. Preliminary results were presented at ASCO 2023. (Price KAR, et al. ASCO 2023. Abstract 6012). Methods: VERSATILE-002 is a single-arm phase 2 study evaluating PDS0101 and pembrolizumab for first-line HPV16-positive R/M HNSCC with CPS ≥1. Subjects received pembrolizumab 200 mg IV Q3W with PDS0101 1 mL SC administered concurrently during Cycles 1, 2, 3, 4, and 12 and pembrolizumab alone for all other Cycles up to Cycle 35 (approx. 2 years). The primary study endpoint is confirmed objective response rate (ORR) per RECIST 1.1. Secondary endpoints include progression-free survival (PFS), OS, and safety. Results: The median follow-up is 18.4 months (range 0.2-42.7 months). The efficacy population consists of 53 subjects: 32 (60%) with CPS ≥1-19 and 21 (40%) with CPS ≥20. The median OS for subjects with CPS ≥1 is 30 months (95% CI 23.9, NE). For the CPS ≥1-19 subgroup, the median OS is 29.5 months (95% CI 15.3, NE). For the CPS ≥20 subgroup, the median OS is 39.3 months (95% CI 18.4, NE). Confirmed response rates by investigator assessment are shown in the Table. Twenty-three subjects are still on study: 3 on treatment and 20 in long-term follow-up. No new safety signals have emerged. The most common TRAEs are injection site reactions, fatigue, headache, and pruritus. Only 19% of subjects experienced Grade ≥3 TRAEs. No subject had a Grade 5 TRAE. Conclusions: These data represent one of the most extended follow-up periods to date of subjects receiving an HPV16-targeted therapy for HPV16-positive R/M HNSCC. The PDS0101 and pembrolizumab combination is well tolerated and has demonstrated deep and durable clinical responses. Median OS is promising in light of historic expectations, both overall and relative to PD-L1 subgroup, and remains durable with continued follow up. The results support further evaluation in a randomized phase 3 study with OS as the primary endpoint. Clinical trial information: NCT04260126 . Summary of results. CPS ≥1-19 (N=32) CPS ≥20 (N=21) CPS ≥1 (N=53) ORR, % 28.1 47.6 35.8 DCR, % 75.0 81.0 77.4 Median DOR, months (95 % CI) 21.8 (4.2, NE) NE (5.6, NE) 21.8 (11.5, NE) Median PFS, months (95% CI) 5.1 (2.4, 8.1) 14.1 (2.1, NE) 6.3 (3.5, 9.0) Median OS, months (95% CI) 29.5 (15.3, NE) 39.3 (18.4, NE) 30.0 (23.9, NE)

e13637 Background: Large language models (LLMs) like GPT-4 (OpenAI) and Claude Opus (Anthropic) showed high accuracy in medical multiple-choice exams, but data on their oncology-specific clinical reasoning and performance is limited. This study evaluates their accuracy and clinical reasoning on breast oncology multiple-choice questions (MCQs) from the American Society of Clinical Oncology (ASCO) question bank. Methods: Using OpenAI and Anthropic Application Programming Interface (APIs), questions were tested without additional prompts under consistent settings (GPT-4 and Claude Opus; Temperature = 0, Tokens = Max). Each question was tested three times to assess precision. Then, Chain-of-thought (COT) prompting was applied to promote LLMs stepwise reasoning to increase their accuracy. Accuracy before and after COT prompting was compared. Incorrect responses were reviewed by board-certified medical oncologists speclized in breast cancer, who scored reasoning clarity, bias, and clinical relevance. Qualitative feedback was descriptively analyzed. Results: A total of 273 breast oncology MCQs were evaluated across the two LLMs. GPT-4 achieved an initial accuracy of 81.3% (222/273; 95% CI: 76.3%-85.5%), compared to Claude Opus, which achieved an accuracy of 79.5% (217/273; 95% CI: 74.3%-83.9%). The Chi-squared test for difference between the models before chain-of-thought (COT) prompting yielded a p-value of 0.59, indicating no statistically significant difference in accuracy between GPT-4 and Claude Opus prior to COT prompting. After COT prompting, the performance of the two models diverged significantly. GPT-4 saw a net decline in accuracy, decreasing by a net of 1 correct answer, resulting in an overall accuracy of 80.95% (221/273). In contrast, Claude Opus experienced a notable improvement, with a net of 19 additional correct answers, leading to an accuracy of 86.4% (236/273). A statistical analysis using a Chi-squared test revealed a difference in accuracy between the two models of 5.5% (p = 0.08), demonstrating that the improvement in accuracy for Claude Opus after COT prompting was borderline statistically significant compared to GPT-4. Thematic analysis of oncologists' feedback revealed that the most common reasons for incorrect answers were reliance on outdated guidelines, misinterpretation of clinical trial data, and failure to consider multidisciplinary or patient-specific approaches in clinical decision-making. Conclusions: Although AI models can achieve high scores on multiple-choice exams, they still require human supervision. These models rely on potentially outdated training data and lack the ability to individualize patient care or apply data from clinical trials. especially in unique or unconventional/non textbook scenarios.

TPS1659 Background: Financial toxicity, the material and psychological burden of treatment cost, affects up to half of people with cancer and can affect adherence and survival. Financial toxicity is a health equity issue, disproportionately affecting Black patients. Patient education about cost and patient-provider cost discussions are recommended to mitigate financial toxicity but occur infrequently. Our goal is to mitigate financial toxicity through a tailorable education and communication intervention, the DISCO App. The DISCO App was shown to be feasible, acceptable, and preliminarily effective at prompting cost discussions and improving related outcomes in a pilot trial. The aim of the ongoing trial is to test the effectiveness of the DISCO App on short- and longer-term outcomes for Black and White patients with cancer. Methods: This study is a longitudinal RCT. Oncologists are eligible if they treat patients with solid tumors at the trial site. Patients of participating oncologists are eligible if they are ≥ 18 years of age; identify as Black or White; can read and write in English; have an email address; and were diagnosed with a solid tumor for which systemic therapy is a likely recommended treatment. Strata were created to balance arms by patient race, income, age, and sex. Upon consent, patients are randomized to one usual care arm (1) or one of two intervention arms (2 and 3). All patients are asked to allow one treatment discussion with their oncologist to be video recorded for analysis. Prior to the recording, intervention patients utilize the DISCO App on an iPad. The DISCO App includes a video about treatment costs, ways to manage costs, and the importance of discussing costs with oncologists. Once patients enter their socio-demographic information (e.g., employment, insurance) and any financial concerns, they receive a tailored list of questions to ask their oncologist. Arm 3 patients receive an intervention booster via email two months after the recording. Patients complete measures at baseline, right after the recording, and at 1, 3, 6 and 12 months after the recording. Measures assess outcomes including cost discussions, communication quality, cost knowledge, self-efficacy for cost management, referrals for support, short- and longer-term financial toxicity, and treatment adherence. The patient-oncologist interaction is the unit of analysis and we will use multi-linear models to compare outcomes by arm. We anticipate recruiting up to 15 oncologists and 240 patients. Data collection began in March 2021 and will continue until July 2025. Participants to date include 13 oncologists and 192 patients (116 Black, 76 White). Most patients completed the baseline assessment (n=164), the post-interaction assessment (n=137), and at least 1 follow-up assessment (n=132); 125 treatment discussions have been recorded. The IRB reviewed the trial in December 2024 and approved continuation. Clinical trial information: NCT04766190 .

7540 Background: The combination of a proteasome inhibitor (PI) with lenalidomide (R) and dex (d) has been a common initial therapy for newly diagnosed myeloma (NDMM). We designed a randomized phase 3 trial to examine if carfilzomib (K), a next generation PI, improved progression free survival (PFS) compared with bortezomib (V) when either is combined with Rd for initial treatment of NDMM. Initial analysis at a median follow up of 15.3 months (mos) showed comparable PFS for both triplets. We present the long-term results of the trial with ~70 months median follow up. Methods: Patients (Pts) with NDMM, were randomized 1:1 to receive VRd or KRd for 36 weeks followed by a 2 nd randomization (1:1) to indefinite versus 2 yrs of R maintenance. Pts without del17p, t(14;16), t(14;20), plasma cell leukemia or high-risk GEP70 profile, were enrolled. VRd arm included V 1.3 mg/m 2 on days(d) 1, 4, 8, and 11 (d1, 8 for cycles 9-12), R 25 mg d1-14, and d 20 mg d1, 2, 4, 5, 8, 9, 11, 12 of a 3-week (wk) cycle for 12 cycles, while pts in the KRd arm received K 36 mg/m 2 d1, 2, 8, 9, 15, 16 with R 25 mg daily on d1-21 and d 40 mg wkly, in 4 wk cycles for 9 cycles. Maintenance used 15 mg R d1-21 q4 wks. Results: The study accrued 1087 pts (VRd=542, KRd=545). Median age was 65y; baseline characteristics including intent to transplant were similar across the arms. Median induction duration (mos; IQR) was 7.2 (3.4-8.9) and 8.4 (5.1-9.1) for VRd and KRd, respectively; 59.8% in VRd and 45.3% in KRd did not proceed to Step 2. Median PFS (mos) was VRd=41.9 and KRd=44.6; HR = 0.89 (0.76-1.04). Toxicity data, PFS sensitivity analyses and OS probabilities are as in the table. Conclusions: In this randomized trial, with median follow up of nearly 6 years, KRd and VRd had comparable PFS and OS in an intent to treat analysis. While similar numbers proceeded to SCT in the 2 arms, more did so during induction in the VRd arm while more patients in the KRd arm went to SCT later. VRd remains a standard triplet induction regimen in standard and intermediate risk NDMM, and a suitable backbone for 4 drug combinations. Clinical trial information: NCT01863550 . N (%) VRd(n=527) KRd(n=526) SCT anytime 186 (34.3) 183 (33.6) SCT without Step 2 registration 146 112 Median Time to SCT (mos; range) 7.7 (3.5-83.9) 10.6 (3.7-70.6) Grade 3-4 Treatment-Related Toxicity 315 (59.8) 344 (65.4) Grade 5 Treatment-Related Toxicity 2 (0.4) 9 (1.7) Grade 5 All Events 11 (2.1) 20 (3.8) Survival outcomes HR Median (95% CI) Median (95% CI) PFS Primary: PD or death within 3 months of last evaluation as events 0.89(0.76-1.04) 41.9(35.7, 50.3) 44.6(38.2, 51.9) PFS Sensitivity: All deaths as events, 0.87(0.75-1.02) 39.5(34.9, 46.0) 42.8(37.4, 49.7) PFS Sensitivity: Censor at alternate Rx 0.83(0.69-0.99) 35.0(31.3, 42.6) 38.7(34.7, 49.0) PFS Sensitivity: Event at alternate Rx 0.84(0.73-0.97) 18.0(14.2, 23.0) 24.4(20.9, 27.9) Overall survival 0.92(0.75-1.12) 119(100, NE) 118(103-NE)

7507 Background: CARTITUDE-1 evaluated cilta-cel in pts with heavily pretreated RRMM who historically have an expected median progression-free survival (PFS) of <6 months (mo) and median overall survival (OS) of ~1 year (y). At 33.4 mo median follow-up, median PFS was 34.9 mo, and median OS was not reached (36-mo OS rate, 62.9%; Lin et al, ASCO 2023). We report OS, ≥5 y progression-free outcomes, and safety with a median study follow-up of 60.3 mo. Methods: Pts in CARTITUDE-1 received a single cilta-cel infusion. Correlative analyses were performed utilizing drug product, baseline, and postinfusion samples. Pts are followed for progression, survival, and safety in a 15-y follow-up study, CARTinue (NCT05201781), with pt evaluations per local standard of care (reported annually at a minimum). Results: Of 97 pts treated, 32 (33.0%) remain alive and progression free for ≥5 y after cilta-cel, without further MM treatment. For these 32 pts, prior to enrollment in CARTITUDE-1, median time from start of last line of therapy (LOT) to progression was 4.0 mo (range, 0.7–48.6). Among the 32, median age was 60 y (range, 43–78), median number of prior LOT was 6.5 (range, 3–14), 23.3% had high risk cytogenetics, 12.5% had extramedullary disease (EMD), 90.6% were triple-class refractory, and 46.9% were penta-drug refractory. Baseline characteristics of pts who were progression free for ≥5 y, including those with high-risk cytogenetics and EMD, were comparable to pts with progressive disease (PD) within 5 y. Compared with pts who had PD within 5 y, biomarkers significantly associated with ≥5 y progression free status included a higher fraction of naïve T cells in the drug product, lower neutrophil to T cell ratio, higher hemoglobin and platelets at baseline, and higher effector-to-target ratio (Cmax to sBCMA at baseline). At Cmax, these pts also had significantly higher CD4 central memory CAR+ T cell subsets and CAR+ T cells that were positive for the activation markers CD38, CD25, and PD-1. Data were collected on a subset of pts from a single center where local serial MRD assessments were performed. All 12 pts at this center who were progression free for ≥5 y were MRD negative at 10 –6 and imaging negative by PET/CT yearly for 5 y. Overall, at 60.3 mo median follow-up in CARTITUDE-1 (N=97), median OS was 60.6 mo (95% CI, 41.9–NE). With continued follow-up, 3 additional pts reported a second primary malignancy (1 of which was acute myeloid leukemia; onset, 2.8 y after infusion). No new cases of movement and neurocognitive disorders were reported. Conclusions: The median OS for pts enrolled in CARTITUDE-1 was 5 y, and 33% of pts remain progression free for ≥5 y following a single cilta-cel infusion. These data provide the first evidence that cilta-cel is potentially curative in pts with RRMM. Clinical trial information: NCT03548207 , NCT05201781 .

4166 Background: GN is a front-line therapy for aPDAC. A Phase I/II study demonstrated that GCN has a higher overall response rate and median overall survival (mOS). Our previous study found no difference in outcomes among patients with DNA damage repair gene mutations; however, stromal gene expression correlated with mOS in patients receiving GCN. Here, we further evaluate differences in the outcomes with GCN and GN by site of biopsy. Methods: PDAC samples (n = 4,463) were analyzed by NGS (NextSeq/NovaSeq) or RNA (NovaSeq) (Caris Life Sciences, Phx, AZ). Expression of stromal and related genes ( ACTA2, ADIRF, HAS2, IL-6, MMP-2, MMP-9, SPARC, STAT3, TBGB1, TGFB2, TGFBR3, IDO1, HLA-DRB4, VEGFB) from different biopsy sites [high expression (H) >50% of RNA transcripts per million] was correlated with outcomes to GCN or GN. mOS was obtained from insurance claims and calculated from first treatment to last contact. The hazard ratio (HR) was calculated by the Cox proportional hazards model, and p-values were calculated using the log-rank test. Results: 4325 patients [primary biopsy (PT), n = 1,878; non-liver metastatic biopsy (N-LM), n = 818; Liver biopsy (LM), n = 1,629] received GN while 138 patients (PT, n = 45; N-LM, n = 28; LM, n = 65) received GCN. GCN was associated with longer mOS than GN [∆: 5.2 months (m), HR: 0.76, 95% CI 0.63-0.92, p = 0.01]. GCN was associated with longer mOS compared to GN in LM (∆: 5.6 m, HR: 0.66, 95% CI 0.50-0.87, p = 0.003), but it was not significant in PT (∆: 4.6 m, p = 0.13) and N-LM (∆: 4.4 m, p = 0.35). Median MMP2 (38.5 vs. 163.1 vs. 162.2), VEGFB (14.4 vs. 16.9 vs. 16.2) and TGFBR3 (11.4 vs. 15 vs. 16.4) expression were lower in LM compared to PT and N-LM while TGFB1 (41.6 vs. 34.5 vs. 39.8) and IL6 (1.61 vs. 1.19 vs. 1.41) expression were highest in LM (p < 0.05). In LM, IL6 -H (∆: -11 m, p = 0.037) and TGFB1 -H (∆: -5.9 m, p = 0.10) were associated with worse post-GCN survival compared to GN (p = 0.82 and p = 0.57). Whereas, in N-LM, TGFBR3 -H trended towards longer post-GCN survival (∆: 12 m, p = 0.18), while ADIRF -H trended towards shorter post-GCN survival (∆: -12 m, p = 0.056) compared to GN (p = 0.42 and p = 0.10). While in PT, MMP2 -H (∆: 11.3m, HR: 0.46, p = 0.16), TGFB1 -H (∆: 11.3 m, HR: 0.33, p = 0.09), HLA-DRB4 -H (∆: 12.9 m, HR: 0.44, p = 0.11) and VEGFB -H (∆: 10.7, HR: 0.34, p = 0.09) trended towards longer post-GCN survival compared to GN ( MMP2 -H, ∆: 3.4 m, p = 0.03, TGFB1 -H, p = 0.661, HLA-DRB4 -H, p = 0.67, VEGFB-H , ∆: 1.3 m, p = 0.03). Conclusions: GCN is associated with improved mOS compared to GN, especially in LM.Stromal gene expression in the liver differs from that in N-LM and the pancreas. While high stromal gene expression trends towards worse post-GCN survival in LM, it is associated with improved survival in N-LM and PT. Further validation is needed to understand the impact of stromal gene expression in different tumor sites on survival outcomes and signature development.

4516 Background: Ipilimumab/nivolumab (ipi/nivo) is a standard of care first-line treatment (tx) for patients (pts) with mccRCC; however, ideal timing of subsequent immunotherapy txs are not well defined. We performed an analysis of patients initially treated with ipi/nivo and subsequent cohort assignments on PDIGREE (A031704). Methods: The PDIGREE trial treated pts with IMDC intermediate/poor risk mccRCC with first-line ipi/nivo (Step 1) at NCTN sites (categorized as academic (A), academic regional (R), and community (C)). Subsequent management was based on iRECIST responses at 12 weeks: pts with complete response (CR) received 1-year nivo maintenance; pts with progressive disease (PD) received cabozantinib (cabo) monotherapy, pts with non-CR/non-PD were randomized to nivo with or without cabo (for primary endpoint of overall survival [OS], target sample size was 1175). Enrollment was held when the randomized sample target was reached. Pts with unresolved toxicity at week 22 were managed off protocol. Step 1 pt demographics, assignments into Step 2, and adverse event (AE) data are presented. Descriptive statistics were used, and cohorts were compared using a chi-square test. These data were released with DSMB approval and do not inform the primary OS endpoint (1EP). Results: From May 2019 to May 2024, 1111 pts were enrolled and treated with ipi/nivo. Pt characteristics included median age 64.0 years (range 29.0-86.0); 819 (73.7%) males; White (85.1%), Hispanic (10.4%), Black (4.2%), and American Indian/Native Hawaiian (1.3%); 849 (76.8%) intermediate risk/257 (23.2%) poor risk; 458 (41.2%) at Academic, 113 (10.2%) at Regional, and 540 (48.6%) at Community centers; and 603 (54.3%) with de novo metastases. 364 pts (33%) stopped tx in Step 1: 160 (44%) for AEs, 46 (13%) for PD/clinical PD/suspected PD, 42 (12%) withdrawals, 39 (11%) alternative txs, 37 (10%) deaths on the study, 12 (3%) other complicating disease, 8 (2%) MD decision and 19 (5%) other reasons. Of the 37 deaths, 15 (1.4%) grade 5 SAEs were reported, 6 of which were due to PD. Of 747 (67%) pts registered to Step 2 at 3 months, 9 (1.2%) achieved CR, and 141 (18.9%) pts were assigned to the PD cohort. 597 pts (80%) were randomized for the 1EP, notable with fewer pts with poor risk [(21 vs 27%, p = 0.01] and bone metastases [24.5% vs 34.2%, p = 0.0007] compared to Step 1 pts who discontinued tx. Gr 3/4 tx-related AEs in 314/1093 (29%) evaluable pts included diarrhea/colitis (8%), transaminase elevation (3%), rash (2%), adrenal insufficiency (2%), fatigue (2%), and hypophysitis (1%). Conclusions: The PDIGREE trial enrolled a representative US-based population with mccRCC. Step 1 pt characteristics and associated outcomes with induction ipi/nivo reveal new insights into the tolerability and tx response of 1L ipi/nivo. Clinical trial information: NCT03793166 .

e20011 Background: The therapeutic landscape of lung cancer is rapidly evolving, presenting oncologists with the challenge of staying updated amidst an overwhelming influx of data. Clinical decision support (CDS) tools, including artificial intelligence (AI) and large language models (LLMs), may help bridge this gap. Evaluating the accuracy of LLMs in complex, real-world oncology scenarios is crucial to understanding their potential. Methods: Twenty-five de-identified lung cancer cases from the fellows’ clinic at Karmanos Cancer Institute, Detroit, MI, were analyzed. Two LLMs, GPT-4 (OpenAI) and Claude Opus (Anthropic), were assessed using advanced prompting techniques like persona-based and chain-of-thought prompting. Five board-certified lung cancer oncologists from NCI-designated centers evaluated LLM-generated responses based on accuracy, treatment recommendation comprehensiveness, and supportive care planning, using a 1–5 scale. Novel insights, the presence of fabricated information, and harmful recommendations were flagged as binary outcomes. Oncologists were blinded to the LLM source and actual treatment decisions. Results: Table 1 presents patient characteristics. GPT-4 achieved an average accuracy score of 4.2 (95% CI, 3.9–4.4), with 3.7 for comprehensiveness of medical/surgical treatment recommendations and 3.7 for supportive care planning. Six responses (32%) were flagged as potentially harmful, and two (8%) contained inaccuracies. Sixteen GPT-4 responses (64%) were rated trustworthy as a CDS tool. Claude Opus had an average accuracy score of 3.6 (95% CI, 3.1–4.1), scoring 3.6 for treatment recommendation comprehensiveness and 3.5 for supportive care planning. Nine responses (36%) were flagged for potential harm, and five (20%) included inaccuracies. Eleven Claude responses (44%) were deemed trustworthy. Significant differences were observed in accuracy (p=0.04) and trustworthiness (p=0.03) between models using McNemar's test. Other factors showed no statistical significance. Conclusions: GPT-4 outperformed Claude Opus in accuracy and trustworthiness, but both models demonstrated limitations, including harmful recommendations and inaccuracies. These findings highlight the need for improved LLM refinement before routine use as CDS tools in lung cancer treatment. Patient demographics and clinical characteristics. Category subcategory Number Median Age (range)- Yr 65 (26-78) Female 7 Male 18 Histology Adenocarcinoma 10 Squamous Cell Carcinoma (SCC) 7 Small Cell Carcinoma 6 Poorly Differentiated 2 Total 25 Stage NSCLC Stage 3 7 NSCLC Stage 4 13 Small Cell limited stage 3 Small Cell Extensive Stage 2