Citations

Background: Large language models (LLMs) are increasingly used in patient-facing clinical applications, creating a need for scalable methods to evaluate the accuracy, safety, and appropriateness of their responses. Although physician review remains the reference standard, it is resource-intensive and difficult to scale. Objective: To develop and clinically validate an LLM-as-a-judge framework for evaluating responses to patients’ periprocedural questions. Methods: We retrospectively evaluated 1285 patient–AIVA interactions from 112 patients across two Mayo Clinic sites and multiple surgical specialties. Physician reviewers classified interactions using a predefined true-positive (TP), false-negative (FN), true-negative (TN), and false-positive (FP) framework. The finalized LLM judge independently evaluated the same interactions. Agreement was assessed using four-class and category-specific agreement, Cohen’s kappa, and a secondary binary analysis of response correctness. Results: Overall, four-class agreement was 93.1% (95% CI, 91.7–94.5%), with Cohen’s κ = 0.852 (95% CI, 0.823–0.882). Category-specific agreement was 94.1% for TP, 90.7% for FN, 95.5% for TN, and 84.9% for FP classifications. In the binary correctness analysis, accuracy was 93.4% (95% CI, 92.0–94.7%), sensitivity 94.6%, specificity 90.2%, precision 96.2%, and F1-score 0.954. McNemar’s test demonstrated no significant asymmetry between paired classifications (p = 0.11). Conclusions: A clinically grounded LLM judge closely approximated physician evaluation of patient-facing AI responses. These findings support its potential as a scalable assistive monitoring tool while preserving physician oversight for uncertain or safety-sensitive interactions.

Abstract Background: Perineural invasion (PNI) is a key prognostic factor in oral squamous cell carcinoma (OSCC). Recent studies have proposed the importance of subcategorizing PNI based on location in relation to the tumor. However, current gaps in definition of PNI subcategories limit its integration into staging system and treatment planning. This study evaluates the prognostic value of the presence of PNI and its location, in correlation with other tumor characteristics, and examines their relationship with survival and recurrence. Design: A total of 159 patients with histologically confirmed OSCC from February 2022–May 2025 were included in the study. PNI was subclassified into intratumoral, peripheral and extratumoral. Association of PNI metrics with other prognostic factors (margin status, lymph node metastasis [LN mets], and lymphovascular invasion [LVI]) were assessed using a Fisher’s exact test (categorical variables). Comparisons of disease-free survival (DFS) and overall survival (OS) between the PNI subgroups were made with a logrank test. Cox proportional hazards regression models were used to find univariate predictors of death and the composite outcome of recurrence or death. A p value <0.05 was considered statistically significant. Statistical analyses were performed using R Statistical Software (version 4.4.1; R Foundation for Statistical Computing, Vienna, Austria). Results: 49/159 cases were PNI positive. Median age: 69 years and male predominance (58.5%). Floor of mouth was the most common site. Presence of PNI correlated significantly with positive margins and positive LVI (p = 0.013 and 0.048 respectively). However, only peripheral PNI showed significant correlation with LN mets (p = 0.004). On univariate analysis, the strongest predictors of recurrence or death were larger tumor size (p <0.001); presence of PNI (p = 0.014), especially intratumoral (p = 0.003); positive LN mets (p = 0.024); and non-white race (p = 0.023). Both DFS and OS were significantly worse in patients with PNI compared with those without (DFS: p = 0.012, OS: p = 0.018). Subclassification of PNI showed a significantly poorer DFS and OS in cases with intratumoral PNI (DFS: p = 0.002, OS: p = 0.012), but not with extratumoral or peripheral PNI.

Introduction Parkinson's disease (PD) and Dementia with Lewy Bodies (DLB) are progressive neurodegenerative disorders characterized by alpha-synuclein pathology. As current therapies largely target symptoms, there is a clear need for disease-modifying treatments. Emerging evidence links these disorders to glucose metabolism dysregulation, suggesting antidiabetic agents may hold therapeutic potential. Methods This was a prospective, randomized, double-blind pilot study involving 12 subjects with PD or DLB to evaluate the tolerability and preliminary effects of a dipeptidyl peptidase 4 (DPP4) inhibitor (sitagliptin 100 mg) vs. sodium glucose like transporter 2 (SGLT2) inhibitor (dapagliflozin 10 mg) over a 4-week period, compared to a placebo. We assessed changes in Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) scores, cognitive function, and autonomic biomarkers. Both sitagliptin and dapagliflozin were well-tolerated, with no adverse events. While no significant changes were observed in cognitive scores, a significant difference was found in the change from baseline for MDS-UPDRS part 1 ( P = 0.008), part 2 ( P = 0.020), and total scores ( P = 0.047) when comparing the three groups. Discussion The dapagliflozin group showed more favorable trends in MDS-UPDRS scores. Limitations include small study size and duration as well as all participants had PD except one subject in the DPP4 inhibitor group with DLB. This pilot study suggests that these inhibitors are safe in PD and DLB patients. The preliminary findings, particularly the trends in the dapagliflozin group, provide a basis for larger-scale, long-term studies to confirm these potential symptomatic benefits and explore disease-modifying effects. Clinical trial registration https://clinicaltrials.gov/study/NCT06263673 , identifier: NCT06263673.

Abstract Detection of germline pathogenic Variants (PVs) in BRCA1, BRCA2, and other genes that significantly elevate lifetime breast cancer (BC) risk are indications for early initiation of intensive screening. BRCA1/2 PVs guidelines recommend that magnetic resonance imaging (MRI) screening begin at age 25 and mammography screening at age 30. However, pregnancy, breastfeeding, and the postpartum period may complicate adherence due to patient concerns about MRI contrast, radiation exposure, logistical barriers, and limited guidance on timing. The incidence of BC peaks approximately 5 years postpartum, especially among women with a positive family history, and early-onset cancers are frequently aggressive, exacerbating the effects of delayed detection. We assessed screening and counseling practices during reproductive periods among women with or at risk for PVs to identify knowledge gaps and opportunities to improve surveillance. We conducted an anonymous survey from March 2023 through January 2025 distributed by Facing Our Risk of Cancer Empowered (FORCE) and social media platforms. Data were collected through a secure REDCap instrument and summarized with descriptive statistics and logistic regression. The Mayo Clinic Institutional Review Board deemed the study exempt. Among the 537 respondents (median age 43 years; 88.5% U.S. residents; 87.5% White), 66.2% were parous and 61.8% knew their family history of BC during pregnancy. Most women reported receiving counseling on screening breast examination, mammography, and MRI; however, 74-77% received no specific guidance about screening before, during, or after pregnancy and weaning. At the time of the survey, 89.9% had undergone BC screening, including 66.2% with annual mammography and 53.1% with annual MRI. Yet during pregnancy and breastfeeding, 69% reported not being screened. Parous women were more likely than nulliparous women to report screening (96.9% vs 85.6%,p&lt;0.0001) and regular mammography (76.4% vs 57.2%,p&lt;0.0001). Screening during pregnancy and breastfeeding was more common among the BRCA1/2 carriers than among carriers of other PVs. Breastfeeding affected screening behavior in 32% of the BRCA1/2 carriers versus 10.9% of other PV carriers, yet only 5.1% of PV carriers stopped breastfeeding to undergo imaging. After their last birth, many women postponed screenings for more than 3 years and only 20.5% received a mammogram within a year of their delivery. Overall, 28.8% of participants reported a history of cancer, of which 66.2% was BC. Approximately 20% of breast cancers reported were diagnosed within 3 years postpartum, with 8.6% occurring during lactation. Despite high rates of BC screening, most high-risk women received little counseling about imaging during pregnancy and breastfeeding, potentially contributing to delayed surveillance at a time of heightened BC risk. These findings highlight the need for targeted counseling, scheduling strategies to support MRI or other imaging during and after pregnancy, and development of novel early-detection tools such as blood or breastmilk biomarkers. Citation Format: Laura M. Pacheco-Spann, Sue Friedman, Diane Rose, William D. Foulkes, Zhihui Fang, Lauren E. Haydu, Mark E. Sherman. Breast Cancer Screening During Reproductive Years in Women with Germline Pathogenic Variants: Gaps in Counseling and Practice [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: The Rise in Early-Onset Cancers—Knowledge Gaps and Research Opportunities; 2025 Dec 10-13; Montreal, QC, Canada. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(23_Suppl):Abstract nr A029.