Broderick, Brendan

ABSTRACT Introduction Community health workers (CHWs) help patients navigate community resources. CHW programs can improve health outcomes and reduce healthcare utilization, but identifying eligible patients is challenging. We developed an electronic health record (EHR)‐based algorithm trained to predict referrals to a CHW program. Our objective was to evaluate whether an algorithm trained on historical referrals identifies a subgroup of patients that differs from the subgroup identified through questionnaire‐based health‐related social needs (HRSN) screening alone. Methods This analysis used data from Mayo Clinic primary care patients in Southeast Minnesota. We developed a gradient boosted time‐to‐event model that incorporated demographics, diagnoses, HRSN questionnaire responses, and HRSNs derived from clinical notes using natural language processing. We compared the characteristics of algorithm‐identified patients with those identified through questionnaires and assessed the variable importance of model features. Results The algorithm had AUC: 0.87 in the training and 0.92 in the hold out set. Patients flagged by the algorithm tended to use interpreter services more commonly (12.6% vs. 1.3%), have a non‐English preferred language (14.1% vs. 2.3%), and have more instances of health literacy documented (5.36 vs. 1.29) compared to patients identified by the HRSN questionnaire. Only 47.0% of patients flagged by the algorithm had responded to the HRSN questionnaire within 2 years. Variable importance was highest for health literacy and financial strain from unstructured notes. Conclusions The algorithm is a chart‐review prioritization tool for referral to the CHW program for patients that may overcome challenges to identifying patients with HRSNs, especially when questionnaire data is incomplete.

INTRODUCTION: Colorectal cancer remains a leading cause of cancer associated death in the United States and colonoscopy the primary screening strategy for prevention. Rates of adenomatous and serrated neoplasia detection are inversely associated with postcolonoscopy colorectal cancer. This crucial quality metric depends on accurate ascertainment of colorectal neoplasia findings from both endoscopy and histopathology records. We aimed to assess the feasibility of a random forest machine learning model to rapidly and accurately categorize colorectal neoplasia from electronic health record data. METHODS: A retrospective cohort study compared neoplasia detection rates among individuals undergoing colonoscopy at a large academic institution to develop a rule-based algorithm to categorize colorectal neoplasia from endoscopy reports and pathology systematized nomenclature of medicine – clinical terms. This cohort provided a large training set to develop a natural language processing system using a random forest approach to automatically classify unstructured pathology findings into adenoma, serrated, or advanced neoplasms. This system was manually validated through an independent holdout set. RESULTS: The training set comprised 35,953 unstructured pathology reports with matched systematized nomenclature of medicine – clinical terms from 95,188 unstructured colonoscopy reports. The final model was assessed on an independent holdout set of 337 manually annotated procedures obtaining an area under the receiver operating characteristic curve of 0.997 (confidence interval [CI] 0.994–1), 0.99 (CI 0.98–1), and 0.99 (CI 0.98–0.99) for prediction of adenoma, serrated, and advanced lesions, respectively. DISCUSSION: The random forest-based hybrid natural language processing system for classification of colonoscopy results was both accurate and explainable. NLP combined with effective machine learning algorithms can provide a scalable strategy for colonoscopy quality monitoring.

Purpose Women with atypical hyperplasia (AH) on breast biopsy have an aggregate increased risk of breast cancer (BC), but existing risk prediction models do not provide accurate individualized estimates of risk in this subset of high-risk women. Here, we used the Mayo benign breast disease cohort to develop and validate a model of BC risk prediction that is specifically for women with AH, which we have designated as AH-BC. Patients and Methods Retrospective cohorts of women age 18 to 85 years with pathologically confirmed benign AH from Rochester, MN, and Nashville, TN, were used for model development and external validation, respectively. Clinical risk factors and histologic features of the tissue biopsy were selected using L1-penalized Cox proportional hazards regression. Identified features were included in a Fine and Gray regression model to estimate BC risk, with death as a competing risk. Model discrimination and calibration were assessed in the model-building set and an external validation set. Results The model-building set consisted of 699 women with AH, 142 of whom developed BC (median follow-up, 8.1 years), and the external validation set consisted of 461 women with 114 later BC events (median follow-up, 11.4 years). The final AH-BC model included three covariates: age at biopsy, age at biopsy squared, and number of foci of AH. At 10 years, the AH-BC model demonstrated good discrimination (0.63 [95% CI, 0.57 to 0.70]) and calibration (0.87 [95% CI, 0.66 to 1.24]). In the external validation set, the model showed acceptable discrimination (0.59 [95% CI, 0.51 to 0.67]) and calibration (0.91 [95% CI, 0.65 to 1.42]). Conclusion We have created a new model with which to refine BC risk prediction for women with AH. The AH-BC model demonstrates good discrimination and calibration, and it validates in an external data set.