Extracting social support and social isolation information from clinical psychiatry notes: comparing a rule-based natural language processing system and a large language model
Abstract Objectives Social support (SS) and social isolation (SI) are social determinants of health (SDOH) associated with psychiatric outcomes. In electronic health records (EHRs), individual-level SS/SI is typically documented in narrative clinical notes rather than as structured coded data. Natural language processing (NLP) algorithms can automate the otherwise labor-intensive process of extraction of such information. Materials and Methods Psychiatric encounter notes from Mount Sinai Health System (MSHS, n = 300) and Weill Cornell Medicine (WCM, n = 225) were annotated to create a gold-standard corpus. A rule-based system (RBS) involving lexicons and a large language model (LLM) using FLAN-T5-XL were developed to identify mentions of SS and SI and their subcategories (eg, social network, instrumental support, and loneliness). Results For extracting SS/SI, the RBS obtained higher macroaveraged F1-scores than the LLM at both MSHS (0.89 versus 0.65) and WCM (0.85 versus 0.82). For extracting the subcategories, the RBS also outperformed the LLM at both MSHS (0.90 versus 0.62) and WCM (0.82 versus 0.81). Discussion and Conclusion Unexpectedly, the RBS outperformed the LLMs across all metrics. An intensive review demonstrates that this finding is due to the divergent approach taken by the RBS and LLM. The RBS was designed and refined to follow the same specific rules as the gold-standard annotations. Conversely, the LLM was more inclusive with categorization and conformed to common English-language understanding. Both approaches offer advantages, although additional replication studies are warranted.
National Trends in Emergency Department Visits for Suicide Attempts and Intentional Self-Harm
Importance There has been increasing concern about the burden of mental health problems among youth, especially since the COVID-19 pandemic. Trends in mental health–related emergency department (ED) visits are an important indicator of unmet outpatient mental health needs. Objective To estimate annual trends in mental health–related ED visits among US children, adolescents, and young adults between 2011 and 2020. Design, Setting, and Participants Data from 2011 to 2020 in the National Hospital Ambulatory Medical Care Survey, an annual cross-sectional national probability sample survey of EDs, was used to examine mental health–related visits for youths aged 6 to 24 years (unweighted = 49 515). Main Outcomes and Measures Mental health–related ED visits included visits associated with psychiatric or substance use disorders and were identified by International Classification of Diseases-Ninth Revision, Clinical Modification ( ICD-9-CM ; 2011-2015) and ICD-10-CM (2016-2020) discharge diagnosis codes or by reason-for-visit (RFV) codes. We estimated the annual proportion of mental health–related pediatric ED visits from 2011 to 2020. Subgroup analyses were performed by demographics and broad psychiatric diagnoses. Multivariable-adjusted logistic regression analyses estimated factors independently associated with mental health–related ED visits controlling for period effects. Results From 2011 to 2020, the weighted number of pediatric mental health–related visits increased from 4.8 million (7.7% of all pediatric ED visits) to 7.5 million (13.1% of all ED visits) with an average annual percent change of 8.0% (95% CI, 6.1%-10.1%; P < .001). Significant linearly increasing trends were seen among children, adolescents, and young adults, with the greatest increase among adolescents and across sex and race and ethnicity. While all types of mental health–related visits significantly increased, suicide-related visits demonstrated the greatest increase from 0.9% to 4.2% of all pediatric ED visits (average annual percent change, 23.1% [95% CI, 19.0%-27.5%]; P < .001). Conclusions and Relevance Over the last 10 years, the proportion of pediatric ED visits for mental health reasons has approximately doubled, including a 5-fold increase in suicide-related visits. These findings underscore an urgent need to improve crisis and emergency mental health service capacity for young people, especially for children experiencing suicidal symptoms.
Extracting social determinants of health from electronic health records using natural language processing: a systematic review
Abstract Objective Social determinants of health (SDoH) are nonclinical dispositions that impact patient health risks and clinical outcomes. Leveraging SDoH in clinical decision-making can potentially improve diagnosis, treatment planning, and patient outcomes. Despite increased interest in capturing SDoH in electronic health records (EHRs), such information is typically locked in unstructured clinical notes. Natural language processing (NLP) is the key technology to extract SDoH information from clinical text and expand its utility in patient care and research. This article presents a systematic review of the state-of-the-art NLP approaches and tools that focus on identifying and extracting SDoH data from unstructured clinical text in EHRs. Materials and Methods A broad literature search was conducted in February 2021 using 3 scholarly databases (ACL Anthology, PubMed, and Scopus) following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A total of 6402 publications were initially identified, and after applying the study inclusion criteria, 82 publications were selected for the final review. Results Smoking status (n = 27), substance use (n = 21), homelessness (n = 20), and alcohol use (n = 15) are the most frequently studied SDoH categories. Homelessness (n = 7) and other less-studied SDoH (eg, education, financial problems, social isolation and support, family problems) are mostly identified using rule-based approaches. In contrast, machine learning approaches are popular for identifying smoking status (n = 13), substance use (n = 9), and alcohol use (n = 9). Conclusion NLP offers significant potential to extract SDoH data from narrative clinical notes, which in turn can aid in the development of screening tools, risk prediction models, and clinical decision support systems.