Tavolara, Thomas E

Abstract Background Interpretation of multi-analyte laboratory profiles, such as lupus anticoagulant (LAC) testing, requires integration of numerous test results and clinical information. However, this process is time-consuming, requires domain knowledge, and has limited scalability. Given that rule-based systems are rigid and difficult to maintain and large language models pose risks of “hallucinations,” we developed and evaluated a novel retrieval-based approach to automate LAC profile interpretation to improve efficiency while maintaining fidelity and reliability for clinical use. Methods Our method processes laboratory values through ensemble random forest (RF) models to retrieve the most similar expert interpretation from a curated database of 347 previously interpreted LAC profiles. The system's performance was assessed through technical accuracy metrics, a work flow efficiency timing study, and expert review of interpretation accuracy for held-out cases. Results Technical validation demonstrated high accuracy in retrieving correct interpretative elements (median: 93% accuracy, interquartile range [IQR]: 67% to 100%). Retrieval accuracy was highly correlated with the prevalence of a given element (ρ = 0.8307). In a timing study of 93 consecutive LAC profiles, ensemble-assisted interpretation achieved a median time of 22 s per report (IQR: 16 s to 31 s), representing a 78.6% total reduction compared to manual interpretation (median: 118 s, IQR: 96 s to 172 s; t = 18.61, P < 0.001). Furthermore, 92.5% of AI-assisted interpretations were completed within 60 s, exceeding the target of 85%. Expert review confirmed 100% concordance for LAC presence/absence. Conclusions The proposed approach significantly improves efficiency of complex laboratory interpretation for LAC profiles, while maintaining consistency and high fidelity. Future studies will explore the method’s inherent extensibility to the interpretation of other multi-analyte laboratory profiles.

Abstract Objectives Traditional laboratory regulatory frameworks provide robust guidance for conventional clinical testing but lack requirements for data documentation for artificial intelligence and machine learning (AI/ML) solutions. This gap creates significant risks since AI solutions directly inherit patterns, biases, and limitations from their training data. Here, we describe the development and implementation of a comprehensive data documentation checklist with accompanying data lineage for use within an AI lifecycle framework for clinical laboratories. Methods Building on the previously established Clinical AI Readiness Evaluator (CARE) framework, we developed a comprehensive data checklist and data lineage methodology through a multiphase process, including (1) comprehensive review of existing AI/ML data documentation frameworks, (2) focused meetings with 3 institutional AI operations teams, and (3) 3 rounds of iterative refinement by our multidisciplinary team. The checklist’s effectiveness was then assessed using 3 diverse AI/ML projects moving through the CARE framework. Results The CARE Data Checklist and Data Lineage provide a structured approach to documenting critical aspects of datasets used in AI/ML projects, including data composition, demographics, collection methods, labeling processes, usage constraints, maintenance requirements, and a data readiness assessment. The checklist addresses unique data-centric challenges of AI/ML applications, facilitating transparency, reproducibility, and regulatory compliance. Conclusions The CARE Data Checklist and Data Lineage serve as both a technical guide and a communication tool bridging gaps between technical and clinical stakeholders. By working on these documents early in the AI lifecycle, laboratories can anticipate and address data-related challenges, ultimately saving time, optimizing resources, and improving the reliability of AI-augmented laboratory solutions.

AbstractBreast cancer affects millions globally, necessitating precise biomarker testing for effective treatment. HER2 testing is crucial for guiding therapy, particularly with novel antibody‐drug conjugates (ADCs) like trastuzumab deruxtecan, which shows promise for breast cancers with low HER2 expression. Current HER2 testing methods, including immunohistochemistry (IHC) and in situ hybridization (ISH), have limitations. IHC, a semi‐quantitative assay, is prone to interobserver variability. While ISH provides higher precision than IHC, it remains more resource‐intensive in terms of cost and workflow. However, turnaround time is typically faster than that of other advanced molecular methods such as next‐generation sequencing. We adapted the clustering‐constrained‐attention multiple‐instance deep learning model to improve IHC testing and reduce dependence on reflex fluorescence ISH (FISH) tests. Using 5,731 HER2 IHC images, including 592 cases with FISH testing, we trained two models: one for predicting HER2 scores from IHC images and another for predicting FISH scores from equivocal cases. The HER2 IHC score prediction model achieved 91% ± 0.01 overall accuracy and a receiver operating characteristic (ROC) area under the curve (AUC) of 0.98 ± 0.01. The FISH score prediction model had an ROC AUC of 0.84 ± 0.07, with sensitivity at 0.37 ± 0.13 and specificity at 0.96 ± 0.03. External validation on cases from 203 institutions showed similar performance. The HER2 IHC model maintained a 91% ± 0.01 accuracy and an ROC AUC of 0.98 ± 0.01, while the FISH model had an ROC AUC of 0.75 ± 0.03, with sensitivity at 0.28 ± 0.04 and specificity at 0.93 ± 0.01. Our model advances HER2 scoring by reducing subjectivity and variability in current scoring methods. Despite lower accuracy and sensitivity in the FISH prediction model, it may be a beneficial option for settings where reflex FISH testing is unavailable or prohibitive. With high specificity, our model can serve as an effective screening tool, enhancing breast cancer diagnosis and treatment selection.