Wahsh, EzzAddin Al

Abstract Objectives Traditional laboratory regulatory frameworks provide robust guidance for conventional clinical testing but lack requirements for data documentation for artificial intelligence and machine learning (AI/ML) solutions. This gap creates significant risks since AI solutions directly inherit patterns, biases, and limitations from their training data. Here, we describe the development and implementation of a comprehensive data documentation checklist with accompanying data lineage for use within an AI lifecycle framework for clinical laboratories. Methods Building on the previously established Clinical AI Readiness Evaluator (CARE) framework, we developed a comprehensive data checklist and data lineage methodology through a multiphase process, including (1) comprehensive review of existing AI/ML data documentation frameworks, (2) focused meetings with 3 institutional AI operations teams, and (3) 3 rounds of iterative refinement by our multidisciplinary team. The checklist’s effectiveness was then assessed using 3 diverse AI/ML projects moving through the CARE framework. Results The CARE Data Checklist and Data Lineage provide a structured approach to documenting critical aspects of datasets used in AI/ML projects, including data composition, demographics, collection methods, labeling processes, usage constraints, maintenance requirements, and a data readiness assessment. The checklist addresses unique data-centric challenges of AI/ML applications, facilitating transparency, reproducibility, and regulatory compliance. Conclusions The CARE Data Checklist and Data Lineage serve as both a technical guide and a communication tool bridging gaps between technical and clinical stakeholders. By working on these documents early in the AI lifecycle, laboratories can anticipate and address data-related challenges, ultimately saving time, optimizing resources, and improving the reliability of AI-augmented laboratory solutions.

AbstractInterruptive alerts can negatively impact clinical workflows and contribute to alert fatigue, provider frustration, and burnout. Given that interruptive alert overriding is a heterogeneous and recurring phenomenon, occurring across different organizational contexts with varying characteristics and circumstances, we hypothesize a pragmatic approach with multimodal interventions to address malfunctioning alert populations and maintain those contributing to better patient care.This study aimed to develop a systematic approach to screen, identify, and correct malfunctioning interruptive alerts within a tertiary healthcare system.We performed screening by assessing the alert population, exploring available resources, and defining alert population inclusion and exclusion criteria. We identified interruptive alerts and then conducted an exploratory analysis. We shared insights from discussions with our expert panel to validate our findings and find gaps in current alert monitoring. We then performed focus groups and interviews as part of a root cause analysis. To address the findings of these investigations, we prioritized which alerts to improve, evaluated solutions, and recommended steps to improve our governance structure.We developed an approach to assess around 1,500 unique alerts in a tertiary center from January to June 2023. We introduced two approaches to visually analyze alert populations: alert-focused analysis and people- and systems-focused analysis. We utilized an expert panel to further enhance the power and speed of alert evaluation and then investigated one emerging alert with focus groups, identifying root causes for its malfunction. This alert demonstrated how enterprise practice changes, coupled with design and cultural issues, can trigger significant alert malfunctions.A multi-modal intervention approach is needed to evaluate interruptive alerts and act quickly on findings. Utilizing both analytical and nonanalytical methods can work in synergy to facilitate this framework. Such approaches may reduce time and be valuable tools for optimally allocating resources to tackle institutional alert challenges.