Reiser, Ingrid

Abstract Medical imaging is undergoing a transformation driven by the advent of new, highly effective, machine learning techniques paired with increases in computational capabilities (Cheng et al. 2021; Gilson et al. 2023; Almeida et al. 2024; Krishna et al. 2024). These advanced algorithms have the potential to improve disease detection, diagnosis, prognosis, and treatment outcomes. However, the complexity of machine learning models, the large amounts of curated and annotated data required by some methods, and the potential for bias and error make it challenging for individuals to safely and effectively leverage these methods (Lin et al. 2024; Guo et al. 2024; Xu et al. 2024; Linguraru et al. 2024; Wood et al. 2019). To address these challenges, the American Association of Physicists in Medicine (AAPM), American College of Radiology (ACR), Radiological Society of North America (RSNA), and Society for Imaging Informatics in Medicine (SIIM) have worked together to develop a syllabus detailing a recommended set of competencies for medical imaging professionals interacting with these systems. This guide is aimed at four different personas: users of AI systems, purchasers of AI systems, individuals who provide clinical expertise during the development of AI systems (“clinical collaborators”), and developers of AI systems.1 This is a syllabus, not a curriculum, and is intentional in this scope. Recognizing that individuals may benefit from different presentations of the same material, this work enumerates a series of relevant competencies but does not prescribe, nor offer, a method of instruction (Schuur, Rezazade Mehrizi, and Ranschaert 2021; Garin et al. 2023). By addressing the task‐specific demands of each role, this guide will enable medical imaging professionals to utilize machine learning systems more safely and effectively, ultimately improving patient care and outcomes.

AbstractReject rate analysis is considered an integral part of a diagnostic radiography quality control (QC) program. A rejected image is a patient radiograph that was not presented to a radiologist for diagnosis and that contributes unnecessary radiation dose to the patient. Reject rates that are either too high or too low may suggest systemic department shortcomings in QC mechanisms. Due to the lack of standardization, reject data often cannot be easily compared between radiography systems from different vendors. The purpose of this report is to provide guidance to help standardize data elements that are required for comprehensive reject analysis and to propose data reporting and workflows to enable an effective and comprehensive reject rate monitoring program. Essential data elements, a proposed schema for classifying reject reasons, and workflow implementation options are recommended in this task group report.

PurposeThe task‐based assessment of image quality using model observers is increasingly used for the assessment of different imaging modalities. However, the performance computation of model observers needs standardization as well as a well‐established trust in its implementation methodology and uncertainty estimation. The purpose of this work was to determine the degree of equivalence of the channelized Hotelling observer performance and uncertainty estimation using an intercomparison exercise.Materials and MethodsImage samples to estimate model observer performance for detection tasks were generated from two‐dimensional CT image slices of a uniform water phantom. A common set of images was sent to participating laboratories to perform and document the following tasks: (a) estimate the detectability index of a well‐defined CHO and its uncertainty in three conditions involving different sized targets all at the same dose, and (b) apply this CHO to an image set where ground truth was unknown to participants (lower image dose). In addition, and on an optional basis, we asked the participating laboratories to (c) estimate the performance of real human observers from a psychophysical experiment of their choice. Each of the 13 participating laboratories was confidentially assigned a participant number and image sets could be downloaded through a secure server. Results were distributed with each participant recognizable by its number and then each laboratory was able to modify their results with justification as model observer calculation are not yet a routine and potentially error prone.ResultsDetectability index increased with signal size for all participants and was very consistent for 6 mm sized target while showing higher variability for 8 and 10 mm sized target. There was one order of magnitude between the lowest and the largest uncertainty estimation.ConclusionsThis intercomparison helped define the state of the art of model observer performance computation and with thirteen participants, reflects openness and trust within the medical imaging community. The performance of a CHO with explicitly defined channels and a relatively large number of test images was consistently estimated by all participants. In contrast, the paper demonstrates that there is no agreement on estimating the variance of detectability in the training and testing setting.