Citations

Context.— Machine learning applications in the pathology clinical domain are emerging rapidly. As decision support systems continue to mature, laboratories will increasingly need guidance to evaluate their performance in clinical practice. Currently there are no formal guidelines to assist pathology laboratories in verification and/or validation of such systems. These recommendations are being proposed for the evaluation of machine learning systems in the clinical practice of pathology. Objective.— To propose recommendations for performance evaluation of in vitro diagnostic tests on patient samples that incorporate machine learning as part of the preanalytical, analytical, or postanalytical phases of the laboratory workflow. Topics described include considerations for machine learning model evaluation including risk assessment, predeployment requirements, data sourcing and curation, verification and validation, change control management, human-computer interaction, practitioner training, and competency evaluation. Data Sources.— An expert panel performed a review of the literature, Clinical and Laboratory Standards Institute guidance, and laboratory and government regulatory frameworks. Conclusions.— Review of the literature and existing documents enabled the development of proposed recommendations. This white paper pertains to performance evaluation of machine learning systems intended to be implemented for clinical patient testing. Further studies with real-world clinical data are encouraged to support these proposed recommendations. Performance evaluation of machine learning models is critical to verification and/or validation of in vitro diagnostic tests using machine learning intended for clinical practice.

Background: Gestational diabetes mellitus (GDM) is a common complication of pregnancy that carries lasting risks. For women who have a history of GDM, the risk of developing type 2 diabetes mellitus (T2DM) increases by 50 to 60%. To monitor and prevent the development of T2DM, guidelines suggest screening those with a history of GDM at 6-weeks and 3-years post-delivery. For patients with abnormal lab results at the 6-week mark, it is advised to complete lab work sooner, at the 1-year mark. Data suggests that these guidelines are not being followed globally, and most do not receive adequate screening for T2DM. Methods: A retroactive chart review of 203 women between 18 and 55 years old with a past medical history significant for GDM was conducted to determine if patients with GDM completed screening for T2DM at 6-week, 1-year, and 3-year timestamps post-delivery. Patient demographics (age, race, ethnicity, language), clinical characteristics (body mass index), provider gender, and whether prediabetes or T2DM developed were recorded. Patients who did not complete the required follow-up received educational materials via the online portal system, and hemoglobin A1c (HbA1c) screening tests were ordered. Results: Ninety days post-intervention, 38 patients (50%) completed the HbA1c screening test, which exceeded our goal of a 5% improvement. Age and BMI 1-year postpartum were found to be significant factors in whether a patient completed HbA1c screening, with 16/18 (89%) completing screening with a healthy BMI 1-year postpartum (18.5 ≤ BMI < 25.0) but only 11/41 (27%) completing screening for patients under 40 years with an unhealthy BMI 1-year postpartum. Of the 38 patients who completed HbA1c screening, 9 (23.7%) had abnormal screening results. Conclusion: Follow-up surveillance for T2DM among women with a history of GDM is poor. Electronic health records can be used to identify and rectify gaps in care. Dissemination of standardized educational materials and electronic order entry for patients improved screening between the handoff from obstetrics to primary care.

BACKGROUND Several studies have highlighted the poor outcomes observed among patients with relapsed/refractory multiple myeloma (MM), particularly those with triple-class refractory (TCR; refractory to at least one PI, one IMiD, and one anti-CD38 mAb) disease. The aim of this study was to describe the longitudinal changes in quality of life (QOL) and other patient-reported outcomes (PROs) among TCR MM patients in real-world clinical practice who initiate a new line-of-therapy. METHODS A prospective, multi-site observational study of patients with TCR MM was conducted through the Mayo Clinic system. Patients initiating a new post-TCR therapy (defined as the index date) were followed for a 12-month observation period during which PRO surveys were administered for the first 6 months. For patients initiating a CAR-T therapy, a survey assessment was conducted at apheresis, lymphodepletion chemotherapy, and every month after T-cell infusion until Month 6. All other patients received a survey assessment before index and then every month thereafter until Month 6. PRO measures included the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire (QLQ-C30; including the Global Health Score [GHS]), the EORTC Multiple Myeloma Questionnaire (QLQ-MY20; including the Disease Symptoms [DS] domain score), the EORTC QLQ for Chemotherapy-Induced Peripheral Neuropathy (CIPN20), the EuroQoL-5 Dimensions (EQ-5D), the Patient Global Impression of Severity (PGIS), and the Patient Global Impression of Change (PGIC). Patient surveys were supplemented with retrospective chart reviews conducted 3 months and 12 months following the index date to obtain clinical information. Descriptive data were reported at each timepoint separately for patients who received CAR-T and non-CAR-T index therapies. The study is ongoing and this abstract reflects all patient visit data as of April 2023. RESULTS N=55 patients (60% male, 96% White, mean age 66 years) were included in this interim analysis. The median time interval between becoming TCR and the index date was 3.9 months (interquartile range [IQR]: 1.7, 15.5). Most patients had an ECOG score of 0 or 1 (53% and 38%, respectively) and were R-ISS I or II (36% and 54%, respectively); 53% of patients had high-risk cytogenetics and 20% had extramedullary disease. The median (IQR) number of prior treatment lines was 4 (4.0, 5.5). N=24 patients initiated a CAR-T therapy as their index therapy (equally split between ide-cel and cilta-cel); the remaining N=31 patients predominantly initiated a treatment regimen containing an IMiD (52%), a PI (55%), an anti-CD38 antibody (13%), or an alkylating agent (36%). Among CAR-T patients, QOL and disease-related symptoms worsened from baseline through Month 2-3 (eg, mean GHS was 65.9 at apheresis, 64.9 at lymphodepletion, and 53.3 at Month 2 [higher scores indicate better QOL]; mean DS domain score from the QLQ-MY20 was 24.1 at apheresis, 23.4 at lymphodepletion, and 33.7 at Month 2 [higher scores indicate more symptoms]; see Table 1). These decrements were followed by improvement back to baseline levels and, in some cases, improvement above baseline levels (eg, GHS was 67.9 and 65.0 at Months 4 and 5, respectively; DS was 18.3 and 17.8 at Months 4 and 5), though few patients had sufficient follow up to analyze QOL at later timepoints. Based on the PGIC, 77.8% of patients reported being “a little better” or “much better” as early as Month 1, though this number declined over time. Among non-CAR-T patients, QOL largely plateaued (GHS was 62.2 at baseline and 58.3 at Month 6), though disease symptoms improved over time (DS was 28.6 at baseline and 24.5 at Month 6; pain was 42.3 at baseline and 29.2 at Month 6). Based on the PGIC, 34.7% of patients reported being “a little better” or “much better” at Month 1, which improved to 50% by Month 6. CONCLUSIONS In this interim analysis of an ongoing prospective observational study, QOL and symptom measures generally worsened over the first few months for CAR-T patients in the real-world though improved to baseline levels and beyond at later time points. Patients did, however, directly report improvement in their disease state as early as Month 1 based on the PGIC. Non-CART patients generally showed a QOL plateau with some modest improvement in disease symptoms which was also reflected in their self-reported assessment of disease state improvement based on the PGIC.

Introduction Cold agglutinin disease (CAD) is an autoimmune hemolytic anemia that is rare and poorly understood. This means that clinicians can find understanding the immunopathogenesis of CAD and individualizing therapy challenging, and may struggle to maintain their knowledge of current and emerging therapies, so patients may not receive optimal care. Therefore, we designed an easily accessible, online, continuing medical education activity for hematologists and hematologist-oncologists, using quality improvement principles together with other change approaches, to improve the care clinicians provide to patients with CAD. Methods The online activity included 3 short videos presented by faculty who shared their expertise in managing patients with CAD. We set learning objectives based on key topics we identified as being challenging for clinicians. This allowed us to measure the impact of the education on knowledge, competence (based on Moore's outcomes framework Levels 3 and 4), intention to change practice and clinical confidence. We also measured participation and satisfaction (Moore's Levels 1 and 2), and changes by region (EU5, Latin America and USA) and clinician experience. Questionnaires were fielded by an independent third party (Nuaxia Limited [Richmond, UK]) at baseline (n=100) and within 3 months (n=100) and 6 months (n=50) after participation in the activity to assess knowledge, competence, satisfaction, intention to change practice and confidence. Additionally, these assessments allowed us to identify the need for further educational activities and the topics they should cover. Participation was measured using Google Analytics. Questionnaire responses were analyzed by touchIME statisticians and data from 3 and 6 months were pooled for combined analysis. The margin of error was calculated as ~3% based on a standard confidence interval of 95%. Results At 6 months, the combined analysis showed an increase from baseline in declarative and procedural knowledge, and competence of 12%, 25% and 20%, respectively (Figure 1). These improvements did not show any significant variation by geography (p=0.432) or by years in practice (p=0.923). Analysis by question topic showed that the significant improvements were achieved consistently (p=0.003) for immunopathogenesis, p&lt;0.001 for current and emerging therapies, and p&lt;0.001 for individualizing treatments; Figure 2). The total number of questions answered correctly also increased significantly (p&lt;0.001) from baseline (64.3%; SD 1.808) at the 6-month combined follow-up (82.0%; SD 1.808). The combined analysis showed clinician confidence in managing CAD improved from 44%, being moderately or extremely confident at baseline, to 72% at 6 months; a median of 57% of learners said they would make a change in their clinical practice. At 6 months post-launch, 6,754 participants had engaged with the education and the overall satisfaction was 88%. Conclusion This educational activity led to significant improvements in the knowledge and competence of hematologists and hematologist-oncologists in the understanding and management of CAD. Further, the majority of participants stated they were more confident and would change their clinical practice, which may improve the quality and effectiveness of patient care. The short, accessible format of this activity meant clinicians could participate whenever convenient and the use of expert-led videos made it engaging and applicable, which may have contributed to the success of the activity. Feedback from learners and our analysis indicate that further quality improvement initiatives on this topic would be beneficial and should cover integrating novel therapies into the clinic, as well as improving diagnostic accuracy in the non-speciality clinic.

Introduction: Daratumumab, lenalidomide, and dexamethasone (DRd) and bortezomib, lenalidomide, and dexamethasone (VRd) are the only two regimens that are considered as preferred per the National Comprehensive Care Network for transplant ineligible (TIE) patients with newly diagnosed multiple myeloma (NDMM). While there are no head-to-head studies comparing clinical outcomes of DRd and VRd in this population, an indirect comparison (PEGASUS study [Durie et al., 2020]) has shown that DRd was associated with significantly lower risk of disease progression or death compared with VRd and Rd. This study aims at supplementing this information by comparing real-world time-to-next-treatment (TTNT) or death between NDMM TIE patients treated with front-line (FL) DRd or VRd. Methods: A retrospective cohort study was conducted for patients with NDMM who initiated FL DRd or VRd in Acentrus (1/1/2018 - 5/31/2023), an electronic medical record (EMR) database from academic and non-teaching hospitals in the United States. Patients were included if they had ≥2 records with an ICD-10-CM diagnosis code for MM, ≥6 months of data availability prior to first record with MM diagnosis (without use of antineoplastic agent), initiated FL treatment (initiation of DRd or VRd; index date) within 12 months of first record of MM diagnosis, and ≥90 days of data availability post-index. Patients with pre-index diagnosis of amyloidosis or other cancers and those who participated in a clinical trial before or during FL treatment were excluded. To limit the analysis to the TIE population, patients who had a record of a stem-cell transplant (SCT) before or during FL treatment and those who were &lt;65 years old (used as proxy to SCT eligibility) were excluded. Patient characteristics were evaluated up to 12-months pre-index (baseline period). Patients were followed from the index date until earliest of date of initiation of a next line of treatment, death, or end of data availability. Medications received within 60 days from the date of the first MM antineoplastic agent were considered as FL therapy regimen. Next line of treatment was identified as initiation of a new antineoplastic agent (excluding corticosteroids) outside of FL therapy or re-treatment of FL after &gt;90-day gap. Inverse probability of treatment weighting (IPTW) was used to balance baseline characteristics between study cohorts, with variables with standardized differences &lt;10% considered balanced. Weighted Kaplan-Meier (KM) curves and a doubly-robust Cox proportional hazards model adjusting for baseline variables remaining imbalanced after IPTW were used compare TTNT or death between study cohorts. Results: Overall, 149 and 494 patients were identified in the DRd and VRd cohorts, respectively. After weighting (N DRd weighted = 302, N VRd weighted = 341), most baseline characteristics in both cohorts were similar, including mean age (DRd: 75.3, VRd: 74.5), female sex (DRd: 48.9%, VRd: 45.8%), white race (DRd: 55.8%, VRd: 58.7%), black race (DRd: 8.8%, VRd: 11.3%), Medicare insurance coverage (DRd: 62.7%, VRd: 65.9%), and mean Quan-Charlson comorbidity index (DRd: 3.8, VRd: 3.6; Table 1). Some imbalances remained for mean age, race, ethnicity, and index year, thus they were additionally adjusted for in doubly-robust Cox model. The median duration of follow-up was 20.2 months and 21.5 months among DRd and VRd patients, respectively. A total of 98 (32.4%) DRd patients and 175 (51.2%) VRd patients received a subsequent line of therapy or died, with the median TTNT or death being 37.8 and 18.7 months in the DRd and VRd cohorts, respectively (hazard ratio: 0.58, 95% CI: 0.35, 0.81; p&lt;0.001; Figure 1). KM estimates for patients remaining on FL therapy were significantly higher for DRd than VRd at 6 months (93.0% vs. 80.1%), 12 months (80.0% vs. 61.0%), 18 months (64.9% vs. 51.3%), and 24 months (60.2% vs. 42.9%). Conclusion: In this retrospective cohort study using EMR data, TIE NDMM patients who initiated DRd had a significantly longer time to next treatment or death than patients who initiated VRd. Results from this real-world study fills an unmet data gap by providing information on the comparative effectiveness evidence for DRd versus VRd among TIE NDMM patients. These findings support the use of DRd as an effective treatment option in the TIE NDMM patient population.

Introduction There is no prospective data to guide the utilization of peri-transplant radiation (PTRT) in patients with relapsed or refractory (R/R) classical Hodgkin lymphoma (cHL). In retrospective analyses, PTRT has been associated with local disease control, however evidence supporting improved progression free survival (PFS) is less well defined (Milgrom SA et al Cancer. 2017; 123(8):1363-1371). Furthermore, the added benefit of PTRT in the era of recent advancement in salvage strategies, including brentuximab vedotin (BV) and checkpoint inhibitors (CPI), is unclear. In a multicenter, retrospective analysis, we evaluated the impact of PTRT on PFS for patients with R/R cHL. Methods We identified consecutive pts ≥7 years old at time of diagnosis with R/R cHL treated with autologous stem cell transplant (ASCT) at 5 US academic institutions between 2010-2021. PTRT was defined as radiation administered within 3 months of ASCT. The primary objective was to compare the post-ASCT PFS, defined as the time from ASCT to subsequent relapse or death from any cause, between the two groups. Univariable and Multivariable Cox regression was performed to evaluate the impact of various relevant clinical factors on post-ASCT PFS. Results A total of 958 eligible pts were identified ( Table 1). The median age at cHL diagnosis was 31 (range, 7-72). At the time of first relapse/primary refractory disease, 406 (54%) pts had early-stage disease, 167 (24%) pts had bulky disease (≥5.0 cm), 308 (36%) pts had extra-nodal disease, 182 (22%) pts had B-symptoms (B-sx), and 338 (45%) had primary refractory disease. A total of 302 (32%) pts received &gt;1 line of salvage before ASCT. Salvage treatment at any time point before ASCT included chemotherapy only for 457 (48%) patients, BV(+/- chemotherapy) for 308 (32%) patients, and CPI (+/- BV or +/- chemotherapy) for 193 (20%) patients. In total, 239 (24.9%) pts received PTRT, 139 (58.1%) before ASCT and 100 (41.8%) after ASCT. Among pts with available dosing information (23.3%), the median radiation dose administered was 28 Gy. The most common doses administered were 30 Gy in 77 (31.7%) pts, 36 Gy in 62 (27.8%) pts, and 18 Gy in 43 (19.2%) pts. Radiated sites included mediastinum in 172 (80.7%) pts, neck/cervical lymph nodes in 68 (31.9%) pts, and supra/infra clavicular region and axilla, each in 31 (14.6%) pts. Compared to the 719 pts that did not receive PTRT, patients treated with PTRT were more likely to have early stage disease (73% vs 47%, p&lt;0.001), bulky disease (30% vs 22%, p = 0.048), primary refractory disease (56% vs 40%, p &lt;0.001), &gt;1 line of salvage treatment prior to ASCT (40% vs 29%, p&lt;0.001), and &lt;CR prior to ASCT (36% vs 27%, p = 0.007) (Table 1). Patients treated with PTRT were less likely to have RT at initial diagnosis (5.9% vs 22%, p&lt;0.001), extra-nodal disease (19% vs 42%, p&lt;0.001), B-sx (17% vs 25%, p=0.035), salvage treatment that included CPI (11% vs 23%, p&lt;0.001), and BV maintenance (20 vs 31%, p&lt;0.001). After a median follow up from ASCT of 4.5 years (range, 2.4-7.2), 2-year PFS was 76% (73%, 79%) and 5-year PFS was 70% (67%, 73%). In univariable analysis, receipt of PTRT showed non-significant reduced hazard for PFS (HR 0.79, 95% CI 0.59-1.05, p = 0.093) (Figure 1). In a multivariable model, PTRT showed significant association with PFS (HR 0.56, 95% CI 0.36-0.88, p=0.009) when adjusting for other clinically relevant factors at relapse such as stage, B-sx, bulk, primary refractory disease, BV(+/- chemotherapy) or CPI (+/- BV or +/- chemotherapy) in salvage, PET response prior to ASCT, and BV maintenance after ASCT. To determine if specific subgroups benefited from PTRT, we tested for interaction effects between PTRT and clinically relevant factors. Only bulk (HR 0.37, 95% CI 0.16-0.85, p=0.013) and B-sx (HR 0.45, 95% CI 0.20-1.02, p=0.041) showed significant interaction with PTRT.Among pts with bulky disease, 5-year PFS was significantly improved with receipt of PTRT (81% vs 62%, p=0.013). Conclusion In this large multi-center analysis of pts with R/R cHL undergoing ASCT, there was no significant difference in PFS regardless of receipt of PTRT. However, after adjusting for known clinically relevant factors, receipt of PTRT was significantly associated with PFS and was particularly beneficial for patients with bulky disease or B-sx at relapse. We found decreased utilization of PTRT in pts receiving CPI-based salvage; therefore the role for PTRT for pts receiving CPI-based salvage is unclear.