Hart, Steven N.

Abstract Objectives Traditional laboratory regulatory frameworks provide robust guidance for conventional clinical testing but lack requirements for data documentation for artificial intelligence and machine learning (AI/ML) solutions. This gap creates significant risks since AI solutions directly inherit patterns, biases, and limitations from their training data. Here, we describe the development and implementation of a comprehensive data documentation checklist with accompanying data lineage for use within an AI lifecycle framework for clinical laboratories. Methods Building on the previously established Clinical AI Readiness Evaluator (CARE) framework, we developed a comprehensive data checklist and data lineage methodology through a multiphase process, including (1) comprehensive review of existing AI/ML data documentation frameworks, (2) focused meetings with 3 institutional AI operations teams, and (3) 3 rounds of iterative refinement by our multidisciplinary team. The checklist’s effectiveness was then assessed using 3 diverse AI/ML projects moving through the CARE framework. Results The CARE Data Checklist and Data Lineage provide a structured approach to documenting critical aspects of datasets used in AI/ML projects, including data composition, demographics, collection methods, labeling processes, usage constraints, maintenance requirements, and a data readiness assessment. The checklist addresses unique data-centric challenges of AI/ML applications, facilitating transparency, reproducibility, and regulatory compliance. Conclusions The CARE Data Checklist and Data Lineage serve as both a technical guide and a communication tool bridging gaps between technical and clinical stakeholders. By working on these documents early in the AI lifecycle, laboratories can anticipate and address data-related challenges, ultimately saving time, optimizing resources, and improving the reliability of AI-augmented laboratory solutions.

Abstract Objectives To develop and validate an agent-based Large Language Model (LLM) system for extracting structured data from breast cancer synoptic pathology reports and assess the performance gap between synthetic and real-world validation. Materials and Methods We developed a modular artificial intelligence (AI) agent-based framework employing sequential specialized LLMs for parsing pathology reports and extracting structured data. We normalized College of American Pathologists (CAP) cancer protocols into 8 sections, 86 subsections, and 229 discrete fields. Seven leading LLMs (gemini-2.5-pro, llama3.3-70b, phi4-14b, deepseek-r1 14B/70B, gemma3-27b, gemini-2.0-flash-lite) were validated using dual evaluation: synthetic validation (864 controlled test cases) and real-world ground truth (6651 annotated fields from 90 pathology reports). Results Synthetic validation demonstrated strong performance (accuracy: 93.8%-99.0%). Real-world evaluation revealed field extraction recall ranging from 61.8% to 87.7%, demonstrating a substantial “reality gap” with performance drops of 11-32 percentage points. The gemini-2.5-pro model achieved the highest real-world recall (87.7%). Model size did not predict performance: the 14B-parameter deepseek-r1 (77.6%) outperformed its 70B-parameter counterpart (70.4%). Discussion The substantial performance degradation from synthetic to real-world data underscores the complexity of authentic clinical documentation. Smaller models can achieve competitive or superior recall, reducing computational costs. With even the best models missing 12%-38% of annotated fields, mandatory human verification is essential for clinical deployment. Conclusion While LLM-based extraction systems show promise for pathology data extraction, synthetic validation alone provides false confidence. Rigorous real-world ground truth evaluation with expert annotation is essential before clinical deployment. These systems are best positioned as screening tools with mandatory human oversight rather than autonomous decision-making systems.

Abstract Purpose: Germline pathogenic variants (PV) in ATM increase the risk of pancreatic ductal adenocarcinoma (PDAC), but the underlying tumor biology of PDAC associated with germline PV in ATM has not been adequately explored. Experimental Design: Whole-genome, whole-exome, and RNA sequencing were performed on PDAC tumors from 25 germline ATM PV carriers diagnosed at Mayo Clinic between 2007 and 2017. Somatic and copy-number alterations, mutational signatures, transcriptomic subtypes, and the immune landscape were evaluated. Results: High-quality whole-exome and whole-genome sequencing were obtained from 21 and 15 tumors, respectively. Biallelic inactivation of ATM was observed in 87%, KRAS PV in 90%, CDKN2A homozygous loss in 60%, and TP53 alterations in <10% of these tumors. A predominant clock-like mutational signature was present in all samples. Whole-transcriptome analysis identified that the aberrantly differentiated endocrine exocrine subtype accounted for 18% of PDAC and was consistently associated with >5-year overall survival. In addition, a 28-gene expression–based signature associated with overall survival was identified and further validated in The Cancer Genome Atlas cohort. Immune landscape analysis through CODEX identified enriched CD4 T-helper cell/tumor interactions and reduced B7H3-high cell/tumor interactions in ATM PV carriers compared with noncarriers. Conclusions: The observed absence of TP53 PV and enrichment for CDKN2A alterations in ATM tumors, along with differences in the mutational signatures, transcriptomic subtypes and immune landscape, improve our understanding of the mechanistic pathways involved in PDAC development in germline ATM PV carriers and help identify potential targeted therapeutic strategies.

Abstract Objective Despite exponential growth in artificial intelligence (AI) research for laboratory medicine and pathology, a significant gap exists between model development and clinical AI implementation. This article introduces a structured framework, the Clinical AI Readiness Evaluator (CARE), to bridge this gap and support the responsible adoption of AI in clinical laboratory settings. Methods Building upon the Machine Learning Technology Readiness Levels framework, we developed CARE specifically for the clinical laboratory environment by incorporating health care–specific requirements, regulatory considerations, and workflow integration needs. This framework was iteratively refined through practical application across diverse AI use cases within laboratory medicine and pathology. Results The CARE framework provides a systematic approach to AI development and implementation through 8 component workstreams: clinical use case, data, data pipeline, code, clinical user experience, clinical technology infrastructure, clinical orchestration, and regulatory compliance. Unlike generic AI frameworks, CARE distinguishes itself by emphasizing both health care and laboratory workflow integration, regulatory requirements, ethical considerations, and comprehensive validation for clinical contexts. The framework accommodates both internally developed models and commercial AI solutions, providing clear guidance through technology readiness levels and structured review processes. Conclusions The CARE framework addresses the unique challenges of implementing AI in laboratory medicine and pathology by providing a comprehensive roadmap from initial concepts through clinical deployment and maintenance. This article, the first in a series of 4, establishes the foundational AI lifecycle framework, while subsequent articles will explore data documentation, ethical AI considerations, and governance structures. By adopting this structured approach, laboratories can responsibly harness AI’s potential to enhance diagnostic accuracy and operational efficiencies and, ultimately, improve patient care.

Abstract Background The loss-of-function (LOF) classification of most missense variants in tumor suppressor breast cancer genes BRCA1, BRCA2, PALB2 , and RAD51C remains unclassified and confounds clinical actionability. Classifying these variants is challenging due to their rarity, leading clinicians to rely on in silico predictive methods. Protein stability changes are associated with function, making stability predictors valuable. Stability predictions upon missense variant perturbations require high-resolution protein structures. However, the availability of these high-resolution structures is lacking. This study explores using generative AI to predict high-resolution protein structures, which can then be analyzed with in silico protein stability prediction methods to assess LOF activity in ordered regions of the protein. This study also determines the appropriate in silico protein stability and dedicated in silico missense prediction methods in dbNSFP v4.7 database to predict LOF activity in ordered regions of these four genes. Functional classifications from homology recombination DNA repair (HDR) assays and variant classifications from the ClinVar database provide a reliable dataset for evaluating the performance of these in silico prediction methods. Results Complex AlphaFold2 structures of the BRCA1-C terminal (BRCT) domain and the DNA-binding (DB) domain of BRCA2, analyzed using protein stability tool FoldX predicts LOF activity from missense variants significantly better than experimentally-derived structures in ordered regions. The BRCT domain achieved an Area Under the Curve (AUC)= 0.861 (95 % CI:0.858–0.863) and AUC= 0.842 (95 % CI:0.840–0.845), while the DB domain achieved an AUC= 0.836 (95 % CI:0.8322–0.841), compared to AUC= 0.847 (95 % CI:0.844–0.850) and AUC= 0.835 (95 % CI:0.832–0.837) from the BRCT domain, and AUC= 0.830 (95 % CI:0.821–0.8320) from the DB domain from experimentally-derived structures. Protein stability does not predict LOF activity from missense variants better than dedicated in silico missense predictors. Overall, we find that AlphaMissense ranks highly, with an average AUC= 0.890 (95 % CI 0.886–0.895) from ordered regions across these four cancer genes, compared to all other in silico missense predictors present in the dbNSFP database. Conclusions The study reveals that generative AI protein predicted structures can outperform experimentally-derived structures in evaluating LOF activity from predicted protein stability in ordered regions of genes BRCA1, BRCA2, PALB2 and RAD51C. The study also highlights the predictive performance of AlphaMissense as the premier in silico missense prediction method to predict LOF activity from missense variants in these four tumor suppressor breast cancer genes. The code for this study can be downloaded for free on GitHub ( https://github.com/rohandavidg/CarePred )

Context.— Generative artificial intelligence (AI) technologies are rapidly transforming numerous fields, including pathology, and hold significant potential to revolutionize educational approaches. Objective.— To explore the application of generative AI, particularly large language models and multimodal tools, for enhancing pathology education. We describe their potential to create personalized learning experiences, streamline content development, expand access to educational resources, and support both learners and educators throughout the training and practice continuum. Data Sources.— We draw on insights from existing literature on AI in education and the collective expertise of the coauthors within this rapidly evolving field. Case studies highlight practical applications of large language models, demonstrating both the potential benefits and unique challenges associated with implementing these technologies in pathology education. Conclusions.— Generative AI presents a powerful tool kit for enriching pathology education, offering opportunities for greater engagement, accessibility, and personalization. Careful consideration of ethical implications, potential risks, and appropriate mitigation strategies is essential for the responsible and effective integration of these technologies. Future success lies in fostering collaborative development between AI experts and medical educators, prioritizing ongoing human oversight and transparency to ensure that generative AI augments, rather than supplants, the vital role of educators in pathology training and practice.

Abstract Breast cancer is one of the most common cancers in women. With early diagnosis, some breast cancers are highly curable. Classifying normal versus tumor breast tissues from microscopy images is an ideal case to use for deep learning and could help diagnose breast cancer with higher reproducibility. Since data preprocessing and hyperparameter tuning have impacts on breast cancer classification accuracies, training a classifier with appropriate data preprocessing and optimized hyperparameters could improve breast cancer classification accuracy. Using 12 combinations of model architectures, data preprocessing, and hyperparameter configurations, the validation accuracy was calculated using the BreAst Cancer Histology (BACH) dataset. The DenseNet201, a non-specialized model architecture, with transfer learning approach achieved 98.61% validation accuracy compared to only 64.00% for the digital pathology (DP)-specialized model architecture. The combination of image data preprocessing and hyperparameters have a profound impact on the performance of deep neural networks for image classification. To identify a well-performing neural network to classify tumor versus normal breast histology, researchers should not only focus on developing new models specifically for DP, since hyperparameter tuning for existing deep neural networks in the computer vision field could also achieve a better prediction accuracy.

Context.— Computational pathology combines clinical pathology with computational analysis, aiming to enhance diagnostic capabilities and improve clinical productivity. However, communication barriers between pathologists and developers often hinder the full realization of this potential. Objective.— To propose a standardized framework that improves mutual understanding of clinical objectives and computational methodologies. The goal is to enhance the development and application of computer-aided diagnostic (CAD) tools. Design.— This article suggests pivotal roles for pathologists and computer scientists in the CAD development process. It calls for increased understanding of computational terminologies, processes, and limitations among pathologists. Similarly, it argues that computer scientists should better comprehend the true use cases of the developed algorithms to avoid clinically meaningless metrics. Results.— CAD tools improve pathology practice significantly. Some tools have even received US Food and Drug Administration approval. However, improved understanding of machine learning models among pathologists is essential to prevent misuse and misinterpretation. There is also a need for a more accurate representation of the algorithms’ performance compared to that of pathologists. Conclusions.— A comprehensive understanding of computational and clinical paradigms is crucial for overcoming the translational gap in computational pathology. This mutual comprehension will improve patient care through more accurate and efficient disease diagnosis.

Context.— Artificial intelligence is a transforming technology for anatomic pathology. Involvement within the workforce will foster support for algorithm development and implementation. Objective.— To develop a supportive ecosystem that enables pathologists with variable expertise in artificial intelligence to create algorithms in a development environment with seamless transition to a production environment. Design.— Platform requirements included (1) approachable and intuitive user interface, (2) diverse algorithmic modeling options, (3) support capability for internal and external collaborations, (4) a seamless mechanism for discovery to clinical deployment transition, (5) the ability to meet minimum institutional requirements for information technology (IT) review, and (6) the ability to be scalable over time. The ecosystem required platform education, data science guidance, project management structure, and ongoing leadership. Results.— The development team considered internal development and vended solutions. Because of the extended timeline and resource requirements for internal development, a decision was made to use a vended solution. Vendor proposals were solicited and reviewed by pathologists, IT, and security groups. A vendor was selected and pipelines for development and production were established. Proposals for development were solicited from the pathology department. Eighty-four investigators were selected for the initial cohort, receiving training and access to dedicated subject matter experts. A total of 30 of 31 projects progressed through the model development process of annotating, training, and validation. Based on these projects, 15 abstracts were submitted to national meetings. Conclusions.— Democratizing artificial intelligence by creating an ecosystem to support pathologists with varying levels of expertise can break down entry barriers, reduce overall cost of algorithm development, improve algorithm quality, and enhance the speed of adoption.

Abstract Recent advances in the field of immuno‐oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high‐throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi‐marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.