Wang, Zhen

Abstract Background The ability of large language models (LLMs) to work collaboratively and screen studies in a systematic review (SR) is under-explored. Hence, we aimed to evaluate the effectiveness of LLMs in automating the process of screening in systematic reviews. Methods This is an observational study which included labeled data (title and abstracts) for five SRs. Originally, two reviewers screened the citations independently for eligibility. A third reviewer cross-checked each citation for quality assurance. GPT-4, Claude-3-Sonnet, and Gemini-Pro-1.0 were used using zero-shot chain-of-thought prompting. Collaborative approaches included (i): conflict resolution using benefit of the doubt, (ii) majority voting using an independent third LLM and (iii) conflict resolution using an informed third LLM. Performance was assessed using accuracy, precision for exclusion, and recall for inclusion. Work saved over samples (WSS) was computed to estimate the reduction in manual human effort. Results A total of 11300 articles were included in this study. The individual models, GPT-4, Claude-3-Sonnet, and Gemini-Pro-1.0 exhibited a high precision for exclusion, achieving 99.7%, 99.7%, and 99.2% and high recall for inclusion achieving 95.5%, 96.6% and 85.7%, respectively. However, the collaborative approach utilizing the two best-performing models (GPT-4 and Claude-3S) achieved an average precision of 99.9% and a recall of 98.5% (across all collaborative approaches). Furthermore, the proposed collaborative approach resulted in an average WSS of 63.5%, compared to the average WSS of 45.2% for individual models. Conversational LLM interactions showed a consistent pattern of results. Limitations This study was limited due to reliance on proprietary models, and evaluation on oncology datasets. Conclusion Evidence shows that collaborative LLMs enable efficient, high-performing screening in systematic reviews, supporting continuous evidence updates. Primary funding source NIH (U24CA265879-01-1) and Carolyn-Ann-Kennedy-Bacon Fund.

Background: Spontaneous coronary artery dissection (SCAD) is a rare and often underdiagnosed cause of acute coronary syndrome (ACS), predominantly affecting younger women without traditional cardiovascular risk factors. The management of SCAD remains a subject of debate, likely secondary to inconclusive evidence. This study aims to compare the clinical outcomes of SCAD patients treated with optimal medical therapy (OMT) versus those who underwent percutaneous coronary intervention (PCI) using a national population-based cohort. Methods: We conducted a retrospective analysis using the National Inpatient Sample (NIS) database from 2016 to 2020. The study included patients identified with SCAD using the ICD-10-CM (the International Classification of Diseases, Tenth Revision, Clinical Modification) code I25.42. We excluded individuals who did not receive PCI or coronary angiography, those who underwent coronary artery bypass grafting, and patients with incomplete records. The primary outcome was in-hospital mortality, while secondary outcomes included acute kidney injury, cardiac arrest, cardiogenic shock, use of temporary mechanical circulatory support, cost of hospitalization, and length of stay. National estimates were obtained using discharge weights, and statistical comparisons were performed using chi-square tests and linear regression. Multivariate logistic regression was employed to identify predictors of mortality and other outcomes. Results: A total of 31,105 SCAD patients were included in the study, with 10,480 receiving OMT and 20,625 undergoing PCI. Patients in the PCI group were older (mean age 64 vs. 54 years) and had higher comorbidities compared to those in the OMT group. The proportion of SCAD patients receiving PCI declined from 72% in 2016 to 60% in 2020. In multivariable analysis, PCI was associated with increased in-hospital mortality (odds ratio (OR) 1.89, 95% confidence interval (CI) 1.24–2.90, p = 0.0003), cardiogenic shock (OR 2.29, 95% CI 1.71–3.07, p < 0.0001), use of a left ventricular assist device (LVAD) (OR 3.97, 95% CI 2.42–6.53, p < 0.0001), and an intra-aortic balloon pump (IABP) (OR 2.24, 95% CI 1.63–3.09, p < 0.0001). Trends also suggested an association between PCI and cardiac arrest, extracorporeal membrane oxygenation (ECMO), and acute kidney injury (AKI). The PCI group had significantly higher hospitalization costs and longer lengths of stay compared to the OMT group (both p < 0.001). Conclusions: In this large, national cohort study, SCAD patients who underwent PCI had significantly higher risks of adverse in-hospital outcomes, including mortality, compared to those treated with OMT. These findings underscore the importance of careful patient selection and the potential advantages of conservative management in SCAD, particularly in patients without severe or unstable presentations. Further research is needed to develop evidence-based guidelines for the optimal management of SCAD.

Introduction: Heart failure (HF) is a leading cause of hospitalization, morbidity and mortality. Deep learning (DL) techniques appear to show promising results in risk stratification and prognosis in several conditions in medicine. However, few methods using DL exist to help quantitatively estimate prognosis of HF. We hypothesized that deep learning (DL) techniques could prognosis of HF using simple variables. We propose application of a custom-built deep-neural-network model to identify mortality in HF patients. Methods: Custom-built deep-neural-networks were assessed using survey data from 42,147 participants from the National Health and Nutrition Examination Survey 1999-2016 (NHANES). Variables were selected using clinical judgment and stepwise backward regressions to develop prediction models. We partitioned the data into training and testing sets and repetitive experiments. We then evaluated model performance based on discrimination and calibration including the area under the receiver-operator characteristics curve (C-statistics), balanced accuracy, probability calibration with sigmoid, and the Brier score, respectively. As sensitivity analyses, we examined results limited to cases with complete clinical information available. We validated models’ performance using Mount Sinai database. Results: Of 42,147 participants with 4,060 variables, 1,491 (3.5%) had HF and HF mortality was 51.8%. In validation cohort, of 26,333 HF patients, the mortality in HF patients was 405 (1.5%). Final model using only 20 variables (age, race, gender, BMI, smoking, alcohol consumption, HTN, COPD, SBP, DBP, HR, HDL, LDL, CRP, A1C, BUN, creatinine, hemoglobin, sodium level, on statin) was tested. A state-of-the-art deep learning models achieved high accuracy for predicting mortality in HF patients with an AUC of 0.96 (95% CI: 0.95-0.99) in the first cohort and AUC of 0.93 (95% CI: 0.91-0.96) in validation cohort. Conclusions: A deep neural network model has shown to have high predictive accuracy and discriminative and calibrative power for prediction of HF mortality. Further research can delineate the clinical implications of DL in predicting HF mortality.