Saoud, Leila

Objectives Machine learning (ML)–based emulators improve the calibration of decision-analytical models, but their performance in complex microsimulation models is yet to be determined. Methods We demonstrated the use of an ML-based emulator with the Colorectal Cancer (CRC)-Adenoma Incidence and Mortality (CRC-AIM) model, which includes 23 unknown natural history input parameters to replicate the CRC epidemiology in the United States. We first generated 15,000 input combinations and ran the CRC-AIM model to evaluate CRC incidence, adenoma size distribution, and the percentage of small adenoma detected by colonoscopy. We then used this data set to train several ML algorithms, including deep neural network (DNN), random forest, and several gradient boosting variants (i.e., XGBoost, LightGBM, CatBoost) and compared their performance. We evaluated 10 million potential input combinations using the selected emulator and examined input combinations that best estimated observed calibration targets. Furthermore, we cross-validated outcomes generated by the CRC-AIM model with those made by CISNET models. The calibrated CRC-AIM model was externally validated using the United Kingdom Flexible Sigmoidoscopy Screening Trial (UKFSST). Results The DNN with proper preprocessing outperformed other tested ML algorithms and successfully predicted all 8 outcomes for different input combinations. It took 473 s for the trained DNN to predict outcomes for 10 million inputs, which would have required 190 CPU-years without our DNN. The overall calibration process took 104 CPU-days, which included building the data set, training, selecting, and hyperparameter tuning of the ML algorithms. While 7 input combinations had acceptable fit to the targets, a combination that best fits all outcomes was selected as the best vector. Almost all of the predictions made by the best vector laid within those from the CISNET models, demonstrating CRC-AIM’s cross-model validity. Similarly, CRC-AIM accurately predicted the hazard ratios of CRC incidence and mortality as reported by UKFSST, demonstrating its external validity. Examination of the impact of calibration targets suggested that the selection of the calibration target had a substantial impact on model outcomes in terms of life-year gains with screening. Conclusions Emulators such as a DNN that is meticulously selected and trained can substantially reduce the computational burden of calibrating complex microsimulation models. Highlights Calibrating a microsimulation model, a process to find unobservable parameters so that the model fits observed data, is computationally complex. We used a deep neural network model, a popular machine learning algorithm, to calibrate the Colorectal Cancer Adenoma Incidence and Mortality (CRC-AIM) model. We demonstrated that our approach provides an efficient and accurate method to significantly speed up calibration in microsimulation models. The calibration process successfully provided cross-model validation of CRC-AIM against 3 established CISNET models and also externally validated against a randomized controlled trial.

10545 Background: Approximately 20–30% of colorectal cancers (CRC) arise from the serrated polyp pathway. The multitarget stool DNA (mt-sDNA) test has greater sensitivity to detect sessile serrated polyps (SSPs) than a leading fecal immunochemical test (FIT). However, most modeling analyses do not account for the contribution of the SSP pathway to risk of CRC. We used the CRC-AIM model to assess the impact of the SSP pathway on predicted CRC outcomes with mt-sDNA or FIT screening. Methods: A simulated cohort of average-risk US patients underwent triennial mt-sDNA or annual FIT screening from ages 50–75. The percentage of CRCs arising from the SSP pathway were modeled at 0% (base case), 20%, and 30%, with stool screening adherence based on theoretical (100%) or previously reported (mt-sDNA 71%; FIT 43%) rates. Published SSP sensitivities for mt-sDNA and FIT were used. All other model inputs were identical to CISNET models. Sensitivity analyses used screening adherence rates of 40–70%. Key outcomes were life-years gained (LYG), CRC incidence and CRC mortality per 1000 patients. Results: Including SSPs in the model demonstrated a greater loss of LYG with FIT than mt-sDNA (Table). At 100% adherence, compared with base case, modeling 20% or 30% SSP pathway CRCs resulted in a decrease of 9–15 LYG with FIT and 2–4 LYG with mt-sDNA, a decrease in CRC incidence reduction of 3.9–6.1% with FIT and 0.7–1.1% with mt-sDNA, and a decrease in CRC mortality reduction of 2.6–4.0% with FIT and 0.4–0.8% with mt-sDNA. Using previously reported adherence, compared with base case, modeling 20% or 30% SSP pathway CRCs resulted in a decrease of 13–20 LYG with FIT and 2–5 LYG with mt-sDNA, a decrease in CRC incidence reduction of 4.4–6.9% with FIT and 0.6–1.1% with mt-sDNA, and a decrease in CRC mortality reduction of 3.5–5.4% with FIT and 0.4–0.9% with mt-sDNA. Assuming reported adherence and 30% SSP pathway CRCs, mt-sDNA had 48 more LYG, 14.6% greater CRC incidence reduction, and 12.4% greater CRC mortality reduction than FIT. Assuming 30% SSP pathway CRCs, outcomes favored mt-sDNA vs FIT even after modeling equivalent adherence rates ranging from 40–70%. Conclusions: After incorporating the SSP pathway into the model, outcomes with mt-sDNA neared those of FIT at 100% screening adherence rates and surpassed FIT at more realistic reported screening adherence rates.[Table: see text]

Abstract Colorectal cancer–screening models commonly assume 100% adherence, which is inconsistent with real-world experience. The influence of adherence to initial stool-based screening [fecal immunochemical test (FIT), multitarget stool DNA (mt-sDNA)] and follow-up colonoscopy (after a positive stool test) on colorectal cancer outcomes was modeled using the Colorectal Cancer and Adenoma Incidence and Mortality Microsimulation Model. Average-risk individuals without diagnosed colorectal cancer at age 40 undergoing annual FIT or triennial mt-sDNA screening from ages 50 to 75 were simulated. Primary analyses incorporated published mt-sDNA (71%) or FIT (43%) screening adherence, with follow-up colonoscopy adherence ranging from 40% to 100%. Secondary analyses simulated 100% adherence for stool-based screening and colonoscopy follow-up (S1), published adherence for stool-based screening with 100% adherence to colonoscopy follow-up (S2), and published adherence for both stool-based screening and colonoscopy follow-up after positive mt-sDNA (73%) or FIT (47%; S3). Outcomes were life-years gained (LYG) and colorectal cancer incidence and mortality reductions (per 1,000 individuals) versus no screening. Adherence to colonoscopy follow-up after FIT had to be 4%–13% higher than mt-sDNA to reach equivalent LYG. The theoretical S1 favored FIT versus mt-sDNA (LYG 316 vs. 297; colorectal cancer incidence reduction 68% vs. 64%; colorectal cancer mortality reduction 76% vs. 72%). The more realistic S2 and S3 favored mt-sDNA versus FIT (S2: LYG 284 vs. 245, colorectal cancer incidence reduction 61% vs. 50%, colorectal cancer mortality reduction 69% vs. 59%; S3: LYG 203 vs. 113, colorectal cancer incidence reduction 43% vs. 23%, colorectal cancer mortality reduction 49% vs. 27%, respectively). Incorporating realistic adherence rates for colorectal cancer screening influences modeled outcomes and should be considered when assessing comparative effectiveness. Prevention Relevance: Adherence rates for initial colorectal cancer screening by FIT or mt-sDNA and for colonoscopy follow-up of a positive initial test influence the comparative effectiveness of these screening strategies. Using adherence rates based on published data for stool-based testing and colonoscopy follow-up yielded superior outcomes with an mt-sDNA versus FIT-screening strategy.

Abstract Background Real‐world data for patients with positive colorectal cancer (CRC) screening stool‐tests demonstrate that adenoma detection rates are lower when endoscopists are blinded to the stool‐test results. This suggests adenoma sensitivity may be lower for screening colonoscopy than for follow‐up to a known positive stool‐based test. Previous CRC microsimulation models assume identical sensitivities between screening and follow‐up colonoscopies after positive stool‐tests. The Colorectal Cancer and Adenoma Incidence and Mortality Microsimulation Model (CRC‐AIM) was used to explore the impact on screening outcomes when assuming different adenoma sensitivity between screening and combined follow‐up/surveillance colonoscopies. Methods Modeled screening strategies included colonoscopy every 10 years, triennial multitarget stool DNA (mt‐sDNA), or annual fecal immunochemical test (FIT) from 50 to 75 years. Outcomes were reported per 1000 individuals without diagnosed CRC at age 40. Base‐case adenoma sensitivity values were identical for screening and follow‐up/surveillance colonoscopies. Ranges of adenoma sensitivity values for colonoscopy performance were developed using different slopes of odds ratio adjustments and were designated as small, medium, or large impact scenarios. Results As the differences in adenoma sensitivity for screening versus follow‐up/surveillance colonoscopies became greater, life‐years gained (LYG) and reductions in CRC‐related incidence and mortality versus no screening increased for mt‐sDNA and FIT and decreased for screening colonoscopy. The LYG relative to screening colonoscopy reached >90% with FIT in the base‐case scenario and with mt‐sDNA in a “medium impact” scenario. Conclusions Assuming identical adenoma sensitivities for screening and follow‐up/surveillance colonoscopies underestimate the potential benefits of stool‐based screening strategies.