Recommendation Statement for the Validation, Implementation, and Clinical Application of Artificial Intelligence Within a Clinical Laboratory from the Digital Pathology Association
Background: The rapid adoption of digital pathology and artificial intelligence (AI) in oncology creates multiple opportunities for precision diagnostics but also creates an urgent need for evidence-based standards to ensure safe and effective implementation. Goal: This article, developed by the Digital Pathology Association, presents recommendations for digital pathology as well as the validation and clinical utility of AI-enabled digital pathology tools in clinical practice. This guidance addresses analytical and clinical validation, algorithm reliability, and criteria for establishing clinical utility attributed to test use. Key recommendations emphasize separate validation of scanning processes and AI algorithms, concordance studies to support interscanner generalizability, rigorous assessment of accuracy and reliability in real-world settings, and clear description of algorithmic use limitations. These recommendations further provide frameworks for when AI may replace or augment existing diagnostic approaches, such as biomarker scoring, cancer diagnosis, and prognostic risk assessment. Conclusions: By considering payer, regulatory, and clinical perspectives, the recommendations promote transparency, trust, and reproducibility in digital pathology while encouraging value-based care delivery. We support responsible innovation in computational pathology, ensuring that AI applications achieve not only technical performance goals but also deliver measurable clinical benefit to patients.
Utilization of an artificial intelligence–enhanced, web‐based application to review bile duct brushing cytologic specimens: A pilot study
Abstract Background The authors previously developed an artificial intelligence (AI) to assist cytologists in the evaluation of digital whole‐slide images (WSIs) generated from bile duct brushing specimens. The aim of this trial was to assess the efficiency and accuracy of cytologists using a novel application with this AI tool. Methods Consecutive bile duct brushing WSIs from indeterminate strictures were obtained. A multidisciplinary panel reviewed all relevant information and provided a central interpretation for each WSI as being “positive,” “negative,” or “indeterminate.” The WSIs were then uploaded to the AI application. The AI scored each WSI as positive or negative for malignancy (i.e., computer‐aided diagnosis [CADx]). For each WSI, the AI prioritized cytologic tiles by the likelihood that malignant material was present in the tile. Via the AI, blinded cytologists reviewed all WSIs and provided interpretations (i.e., computer‐aided detection [CADe]). The diagnostic accuracies of the WSI evaluation via CADx, CADe, and the original clinical cytologic interpretation (official cytologic interpretation [OCI]) were compared. Results Of the 84 WSIs, 15 were positive, 42 were negative, and 27 were indeterminate after central review. The WSIs generated on average 141,950 tiles each. Cytologists using the AI evaluated 10.5 tiles per WSI before making an interpretation. Additionally, cytologists required an average of 84.1 s of total WSI evaluation. WSI interpretation accuracies for CADx (0.754; 95% CI, 0.622–0.859), CADe (0.807; 95% CI, 0.750–0.856), and OCI (0.807; 95% CI, 0.671–0.900) were similar. Conclusions This trial demonstrates that an AI application allows cytologists to perform a triaged review of WSIs while maintaining accuracy.
Recommendations for Performance Evaluation of Machine Learning in Pathology: A Concept Paper From the College of American Pathologists
Context.— Machine learning applications in the pathology clinical domain are emerging rapidly. As decision support systems continue to mature, laboratories will increasingly need guidance to evaluate their performance in clinical practice. Currently there are no formal guidelines to assist pathology laboratories in verification and/or validation of such systems. These recommendations are being proposed for the evaluation of machine learning systems in the clinical practice of pathology. Objective.— To propose recommendations for performance evaluation of in vitro diagnostic tests on patient samples that incorporate machine learning as part of the preanalytical, analytical, or postanalytical phases of the laboratory workflow. Topics described include considerations for machine learning model evaluation including risk assessment, predeployment requirements, data sourcing and curation, verification and validation, change control management, human-computer interaction, practitioner training, and competency evaluation. Data Sources.— An expert panel performed a review of the literature, Clinical and Laboratory Standards Institute guidance, and laboratory and government regulatory frameworks. Conclusions.— Review of the literature and existing documents enabled the development of proposed recommendations. This white paper pertains to performance evaluation of machine learning systems intended to be implemented for clinical patient testing. Further studies with real-world clinical data are encouraged to support these proposed recommendations. Performance evaluation of machine learning models is critical to verification and/or validation of in vitro diagnostic tests using machine learning intended for clinical practice.
Cytomorphology and sonographic features of ectopic thymic tissue diagnosed in paediatric FNA biopsies
IntroductionEctopic thymic tissue can arise as an asymptomatic neck mass, which may be detected on imaging studies. The aim of this study was to determine the incidence of ectopic thymic tissue in paediatric FNAs and to the correlate clinical, radiological and cytomorphological findings.MethodsFNAs in children with neck and mediastinal lesions performed between January 2012 and July 2016 were reviewed for cases of ectopic thymus. These were then evaluated and correlated with the cytology findings.ResultsOf 739 FNAs, 13 (1.8%) cases from 11 patients showed ectopic thymic tissue. The targeted lesions were in the thyroid (n = 7), submandibular region (n = 1), superior mediastinum (n = 1) and paratracheal region (n = 1). The most common indication was for microcalcifications concerning for papillary thyroid carcinoma on ultrasound (n = 6). Imaging findings included fusiform lesions with linear and punctuate bright echoes. The cytology evaluation showed small lymphocytes with discohesive epithelioid cells in most cases, and proteinaceous fluid in the cystic case. There were rare macrophages and Hassall's corpuscles. Flow cytometry and/or immunostains were performed in all cases, supporting thymic origin.ConclusionEctopic thymic tissue is rarely present as a neck mass or thyroid nodule on FNA biopsy. The ultrasound imaging findings reveal a well‐defined fusiform lesion with punctate bright echoes that could be misinterpreted as papillary thyroid carcinoma. The aspirates show a small lymphoid population, immunophenotypically compatible with thymic T‐cells, in addition to scattered epithelial cells. Therefore, knowledge of the typical ultrasonographic and cytopathological features can help make a definitive diagnosis and avoid more invasive procedures in paediatric patients.