Chen, Jing Voon

The Centers for Medicare & Medicaid Services (CMS) recommend covering blood-based tests meeting proposed minimum performance thresholds for colorectal cancer (CRC) screening. Outcomes were compared between currently available stool-based screening tests and a hypothetical blood-based test meeting CMS minimum thresholds. Using the Colorectal Cancer and Adenoma Incidence and Mortality Microsimulation Model (CRC-AIM), outcomes were simulated for average-risk individuals screened between ages 45 and 75 years with triennial multitarget stool DNA (mt-sDNA), annual fecal immunochemical test (FIT), and annual fecal occult blood test (FOBT). Per CMS guidance, blood-based CRC screening was modeled triennially, with 74% CRC sensitivity and 90% specificity. Although not specified by CMS, adenoma sensitivity was set between 10% and 20%. Published adenoma and CRC sensitivity and specificity were used for stool-based tests. Adherence was set at (1) 100%, (2) 30%–70%, in 10% increments, and (3) real-world rates for stool-based tests (mt-sDNA = 65.6%; FIT = 42.6%; FOBT = 34.4%). Assuming perfect adherence, a blood-based test produced ≥19 lower life-years gained (LYG) than stool-based strategies. At the best-case scenario for blood-based tests (100% adherence and 20% adenoma sensitivity), mt-sDNA at real-world adherence achieved more LYG (287.2 vs. 297.1, respectively) with 14% fewer colonoscopies. At 100% blood-based test adherence and real-world mt-sDNA and FIT adherence, the blood-based test would require advanced adenoma sensitivity of 30% to reach the LYG of mt-sDNA (297.1) and ∼15% sensitivity to reach the LYG of FIT (258.9). This model suggests that blood-based tests with CMS minimally acceptable CRC sensitivity and low advanced adenoma sensitivity will frequently yield inferior outcomes to stool-based testing across a wide range of adherence assumptions.

Objectives Machine learning (ML)–based emulators improve the calibration of decision-analytical models, but their performance in complex microsimulation models is yet to be determined. Methods We demonstrated the use of an ML-based emulator with the Colorectal Cancer (CRC)-Adenoma Incidence and Mortality (CRC-AIM) model, which includes 23 unknown natural history input parameters to replicate the CRC epidemiology in the United States. We first generated 15,000 input combinations and ran the CRC-AIM model to evaluate CRC incidence, adenoma size distribution, and the percentage of small adenoma detected by colonoscopy. We then used this data set to train several ML algorithms, including deep neural network (DNN), random forest, and several gradient boosting variants (i.e., XGBoost, LightGBM, CatBoost) and compared their performance. We evaluated 10 million potential input combinations using the selected emulator and examined input combinations that best estimated observed calibration targets. Furthermore, we cross-validated outcomes generated by the CRC-AIM model with those made by CISNET models. The calibrated CRC-AIM model was externally validated using the United Kingdom Flexible Sigmoidoscopy Screening Trial (UKFSST). Results The DNN with proper preprocessing outperformed other tested ML algorithms and successfully predicted all 8 outcomes for different input combinations. It took 473 s for the trained DNN to predict outcomes for 10 million inputs, which would have required 190 CPU-years without our DNN. The overall calibration process took 104 CPU-days, which included building the data set, training, selecting, and hyperparameter tuning of the ML algorithms. While 7 input combinations had acceptable fit to the targets, a combination that best fits all outcomes was selected as the best vector. Almost all of the predictions made by the best vector laid within those from the CISNET models, demonstrating CRC-AIM’s cross-model validity. Similarly, CRC-AIM accurately predicted the hazard ratios of CRC incidence and mortality as reported by UKFSST, demonstrating its external validity. Examination of the impact of calibration targets suggested that the selection of the calibration target had a substantial impact on model outcomes in terms of life-year gains with screening. Conclusions Emulators such as a DNN that is meticulously selected and trained can substantially reduce the computational burden of calibrating complex microsimulation models. Highlights Calibrating a microsimulation model, a process to find unobservable parameters so that the model fits observed data, is computationally complex. We used a deep neural network model, a popular machine learning algorithm, to calibrate the Colorectal Cancer Adenoma Incidence and Mortality (CRC-AIM) model. We demonstrated that our approach provides an efficient and accurate method to significantly speed up calibration in microsimulation models. The calibration process successfully provided cross-model validation of CRC-AIM against 3 established CISNET models and also externally validated against a randomized controlled trial.

Abstract Commercial insurance covers a follow-up colonoscopy after a positive colorectal cancer–screening test with no patient cost-sharing. Instituting a similar policy for Medicare beneficiaries may increase screening adherence and improve outcomes. The cost-effectiveness of stool-based colorectal cancer screening was compared across adherence scenarios that assumed Medicare coinsurance status quo (20% for follow-up colonoscopy) or waived coinsurance. The CRC-AIM model simulated previously unscreened eligible Medicare beneficiaries undergoing stool-based colorectal cancer screening at age 65 for 10 years. Medicare costs, colorectal cancer cases, colorectal cancer–related deaths, life-years gained (LYG), and quality-adjusted life-years (QALY) were estimated versus no screening. Scenario 1 (S1) assumed 20% coinsurance for follow-up colonoscopy. Scenario 2 (S2) assumed waived coinsurance without adherence changes. Scenarios 3–7 (S3–S7) assumed that waiving coinsurance increased real-world stool-based screening and/or follow-up colonoscopy adherence by 5% or 10%. Sensitivity analyses assumed 1%–4% increased adherence. Cost-effectiveness threshold was ≤$100,000/QALY. Waiving coinsurance without adherence changes (S2) did not affect outcomes versus S1. S3–S7 versus S1 over 10 years estimated up to 3.6 fewer colorectal cancer cases/1,000 individuals, up to 2.1 fewer colorectal cancer deaths, up to 20.7 more LYG, and had comparable total costs per-patient (≤$6,478 vs. $6,449, respectively) as reduced colorectal cancer medical costs offset increased screening and colonoscopy costs. In sensitivity analyses, any increase in adherence after waiving coinsurance was cost-effective and increased LYG. In simulated Medicare beneficiaries, waiving coinsurance for follow-up colonoscopy after a positive stool-based test improved outcomes and was cost-effective when assumed to modestly increase colorectal cancer screening and/or follow-up colonoscopy adherence. Prevention Relevance: Follow-up colonoscopy after a positive stool-based test is necessary to complete the colorectal cancer-screening process. This analysis demonstrated that in a simulated Medicare population, waiving coinsurance for a follow-up colonoscopy improved estimated outcomes and was cost-effective when it was assumed that waiving the coinsurance modestly increased screening adherence. See related Spotlight, p. 641

10529 Background: The Centers for Medicare & Medicaid Services (CMS) recommends covering blood-based biomarker tests with proposed minimum performance thresholds for colorectal cancer (CRC) screening test every 3 years for average-risk, asymptomatic Medicare beneficiaries ages 50–85 years. A blood-based test is a non-invasive screening method to detect CRC but is limited by low sensitivity to detect adenomas. Using the CRC-AIM microsimulation model, predicted life-years gained (LYG) and CRC incidence and mortality reduction were compared between a blood-based test meeting the CMS minimum thresholds and stool-based screening tests (fecal immunochemical test [FIT], fecal occult blood test [FOBT], and multitarget stool DNA [mt-sDNA]). Methods: Outcomes of blood- and stool-based tests were simulated for average-risk individuals free of diagnosed CRC at age 40 and screened between ages 45–75 years per USPSTF recommendations. Per CMS proposed criteria, CRC sensitivity and specificity for a blood-based test were set at 74% and 90%, respectively, and adenoma sensitivity was set at 10%. Published adenoma and CRC sensitivity and specificity were used for each stool test. For the primary analysis, adherence was assumed to be 100%. For secondary analysis, adherence was set at 30–70%, in 10% increments. Outcomes were per 1000 individuals. Results: At 100% adherence, LYG was 229 for a blood test and ≥305 for stool tests, corresponding to at least 25% lower LYG for a blood test relative to all stool tests (Table). Absolute CRC incidence and mortality reductions were at least 19% and 18% lower, respectively, for a blood test vs all stool tests. Secondary analysis indicates that at identical adherence rates that are less than 100%, a blood test had lower LYG vs all stool tests (Table). When the adherence of any test was 50%–70%, a blood test resulted in at least 20% lower LYG vs any stool test, and absolute reductions in CRC incidence and mortality were at least 9% and 10% lower, respectively, vs any stool test. Conclusions: This model suggests that if blood-based CRC screening tests do not sufficiently detect advanced adenomas, clinical outcomes will be inferior to stool-based testing due to lack of cancer prevention. Further discovery efforts to identify blood-based markers associated with both invasive and preinvasive neoplasia are needed to address this deficiency. [Table: see text]

e13624 Background: Commercial insurance plans are required to cover, with no patient cost sharing, a follow-up colonoscopy (COL) after a positive stool-based colorectal cancer (CRC) screening test. Medicare beneficiaries may still be responsible for 20% coinsurance for a follow-up COL, posing a financial barrier to screening completion. Removing the coinsurance may increase screening adherence and improve outcomes. The estimated cost-effectiveness of stool-based CRC screening was compared among adherence scenarios that assumed the status quo (20% coinsurance for follow-up COL) or waived follow-up COL coinsurance. Methods: The CRC-AIM microsimulation model simulated US Medicare beneficiaries undergoing stool-based CRC screening between ages 65-75 years. Outcomes of total costs, total quality adjusted life years (QALYs), life-years gained (LYG), CRC incidence reductions (IR), and CRC mortality reductions (MR) were calculated per 1000 individuals versus no screening using the weighted average of the outcome with multitarget stool DNA, fecal immunochemical test, and fecal occult blood test, scaled by their estimated use. Costs of follow-up COL were assumed to have 20% coinsurance in the status quo scenario. Four comparative scenarios assumed that waiving coinsurance increased published, real-world stool-based screening or follow-up COL adherence rates by 5% or 10%. Results: In scenarios where waiving coinsurance was assumed to increase screening and/or follow-up COL adherence, up to 21 more LYG and greater IR and MR were observed vs the status quo (Table). Total costs in waived coinsurance scenarios (≥$6,398) were comparable to the status quo ($6,449) as the cost savings in CRC care offset the increased COL costs. Total QALYs in waived coinsurance scenarios (≥9.3671) were greater than the status quo (9.3643) because of improved clinical outcomes. Stool-based testing in waived coinsurance scenarios was either dominant (more effective and less costly) or cost-effective at $50,000/QALY with minimal incremental costs vs the status quo. Conclusions: In a simulated Medicare population, clinical CRC outcomes improved and stool-based screening was cost-effective or cost-saving when waiving the 20% coinsurance was assumed to modestly (5%) increase adherence rates for total CRC screening and/or a COL following a positive test. [Table: see text]