Abstract Aims To develop a deep learning model that: (i) utilizes transthoracic echocardiography (TTE) clips to detect left ventricular (LV) enlargement without being provided information regarding a patient’s sex and body size; and (ii) can be accurately applied to clips acquired using either standard comprehensive TTE or handheld cardiac ultrasound (HCU). Methods and results Using retrospective TTE data (training: 8722 patients; internal validation: 468 patients), we developed a deep learning model that estimates a patient’s end-diastolic LV volume (indexed to body surface area and normalized across the sexes), and then thresholds this estimate to perform the following classifications: (1) normally sized LV vs. ≥ mild LV enlargement; (2) normal/mildly enlarged LV vs. ≥ moderate LV enlargement. For retrospective datasets, the model showed strong performance in TTE across three geographically distinct locations (Minnesota and Wisconsin: 1082 patients, AUC = 0.925 and 0.953 for classifications 1 and 2, respectively; Arizona: 1475 patients, AUC = 0.935 and 0.969; and Florida: 1481 patients, AUC = 0.934 and 0.970). Additionally, performance was strong for both TTE and HCU clips collected from a prospective cohort of 410 patients who underwent HCU immediately following TTE (TTE: AUC = 0.925 and 0.971; HCU: AUC = 0.874 and 0.902, for classifications 1 and 2, respectively). Conclusion An automated deep learning model applied to TTE or HCU images accurately categorizes LV volumes. These results lay a foundation for future work aimed at optimizing clinical outcomes for heart failure patients by enabling early detection of LV enlargement across various point-of-care settings.