Han, Wenchao

Abstract Objectives Traditional laboratory regulatory frameworks provide robust guidance for conventional clinical testing but lack requirements for data documentation for artificial intelligence and machine learning (AI/ML) solutions. This gap creates significant risks since AI solutions directly inherit patterns, biases, and limitations from their training data. Here, we describe the development and implementation of a comprehensive data documentation checklist with accompanying data lineage for use within an AI lifecycle framework for clinical laboratories. Methods Building on the previously established Clinical AI Readiness Evaluator (CARE) framework, we developed a comprehensive data checklist and data lineage methodology through a multiphase process, including (1) comprehensive review of existing AI/ML data documentation frameworks, (2) focused meetings with 3 institutional AI operations teams, and (3) 3 rounds of iterative refinement by our multidisciplinary team. The checklist’s effectiveness was then assessed using 3 diverse AI/ML projects moving through the CARE framework. Results The CARE Data Checklist and Data Lineage provide a structured approach to documenting critical aspects of datasets used in AI/ML projects, including data composition, demographics, collection methods, labeling processes, usage constraints, maintenance requirements, and a data readiness assessment. The checklist addresses unique data-centric challenges of AI/ML applications, facilitating transparency, reproducibility, and regulatory compliance. Conclusions The CARE Data Checklist and Data Lineage serve as both a technical guide and a communication tool bridging gaps between technical and clinical stakeholders. By working on these documents early in the AI lifecycle, laboratories can anticipate and address data-related challenges, ultimately saving time, optimizing resources, and improving the reliability of AI-augmented laboratory solutions.

Abstract Deep learning models enable the prediction of clinical endpoints from whole-slide images (WSIs), but many such models function as “black boxes”, lacking transparency about whether and which histomorphological patterns drive their predictions, hindering interpretability and clinical adoption. Here we propose a human-in-the-loop explanation framework, MorphoXAI, which provides both local and global interpretability for deep learning models by incorporating human-expert interpretations. At the global level, it reveals the histomorphological patterns on which the model consistently relies to distinguish between classes of WSIs, as well as the patterns associated with confusion between classes. At the local level, it indicates which of these patterns are used in the prediction of an individual WSI and which regions within the slide correspond to such patterns. We validated our method on a deep learning model trained for ovarian tumor histologic subtype prediction. The results show that our framework generates explanations that accurately reflect the histomorphology underlying the model’s predictions at both global and local levels. For interpretability and clinical utility in diagnostic contexts, human evaluation results showed that our explanations were easy to interpret, rich in diagnostic features, and directly helpful for diagnostic decision-making, thereby enhancing pathologist-AI collaboration. Our work highlights that unifying global and local explanations and grounding them in expert-interpreted morphology enhances the interpretability and verifiability of deep learning models, thereby facilitating the transparent deployment of such models in clinical practice.

AbstractBreast cancer affects millions globally, necessitating precise biomarker testing for effective treatment. HER2 testing is crucial for guiding therapy, particularly with novel antibody‐drug conjugates (ADCs) like trastuzumab deruxtecan, which shows promise for breast cancers with low HER2 expression. Current HER2 testing methods, including immunohistochemistry (IHC) and in situ hybridization (ISH), have limitations. IHC, a semi‐quantitative assay, is prone to interobserver variability. While ISH provides higher precision than IHC, it remains more resource‐intensive in terms of cost and workflow. However, turnaround time is typically faster than that of other advanced molecular methods such as next‐generation sequencing. We adapted the clustering‐constrained‐attention multiple‐instance deep learning model to improve IHC testing and reduce dependence on reflex fluorescence ISH (FISH) tests. Using 5,731 HER2 IHC images, including 592 cases with FISH testing, we trained two models: one for predicting HER2 scores from IHC images and another for predicting FISH scores from equivocal cases. The HER2 IHC score prediction model achieved 91% ± 0.01 overall accuracy and a receiver operating characteristic (ROC) area under the curve (AUC) of 0.98 ± 0.01. The FISH score prediction model had an ROC AUC of 0.84 ± 0.07, with sensitivity at 0.37 ± 0.13 and specificity at 0.96 ± 0.03. External validation on cases from 203 institutions showed similar performance. The HER2 IHC model maintained a 91% ± 0.01 accuracy and an ROC AUC of 0.98 ± 0.01, while the FISH model had an ROC AUC of 0.75 ± 0.03, with sensitivity at 0.28 ± 0.04 and specificity at 0.93 ± 0.01. Our model advances HER2 scoring by reducing subjectivity and variability in current scoring methods. Despite lower accuracy and sensitivity in the FISH prediction model, it may be a beneficial option for settings where reflex FISH testing is unavailable or prohibitive. With high specificity, our model can serve as an effective screening tool, enhancing breast cancer diagnosis and treatment selection.