Watkins, William T.

AbstractPurposeTo reduce the likelihood of errors in organ delineations used for radiotherapy treatment planning, a knowledge‐based quality control (KBQC) system, which discriminates between valid and anomalous delineations is developed.Method and MaterialsThe KBQC is comprised of a group‐wise inference system and anomaly detection modules trained using historical priors from 296 locally advanced lung and prostate cancer patient computational tomographies (CTs). The inference system discriminates different organs based on shape, relational, and intensity features. For a given delineated image set, the inference system solves a combinatorial optimization problem that results in an organ group whose relational features follow those of the training set considering the posterior probabilities obtained from support vector machine (SVM), discriminant subspace ensemble (DSE), and artificial neural network (ANN) classifiers. These classifiers are trained on nonrelational features with a 10‐fold cross‐validation scheme. The anomaly detection module is a bank of ANN autoencoders, each corresponding with an organ, trained on nonrelational features. A heuristic rule detects anomalous organs that exceed predefined organ‐specific tolerances for the feature reconstruction error and the classifier's posterior probabilities. Independent data sets with anomalous delineations were used to test the overall performance of the KBQC system. The anomalous delineations were manually manipulated, computer‐generated, or propagated based on a transformation obtained by imperfect registrations. Both peer‐review–based scoring system and shape similarity coefficient (DSC) were used to label regions of interest (ROIs) as normal or anomalous in two independent test cohorts.ResultsThe accuracy of the classifiers was 99.8%, and the minimum per‐class F1‐scores were 0.99, 0.99, and 0.98 for SVM, DSE, and ANN, respectively. The group‐wise inference system reduced the miss‐classification likelihood for the test data set with anomalous delineations compared to each individual classifier and a fused classifier that used the average posterior probability of all classifiers. For 15 independent locally advanced lung patients, the system detected 79% of the anomalous ROIs. For 1320 auto‐segmented abdominopelvic organs, the anomaly detection system identified anomalous delineations, which also had low Dice similarity coefficient values with respect to manually delineated organs in the training data set.ConclusionThe KBQC system detected anomalous delineations with superior accuracy compared to classification methods that judge only based on posterior probabilities.

PurposeTo develop a quality assurance (QA) tool that identifies inaccurate organ at risk (OAR) delineations.MethodsThe QA tool computed volumetric features from prior OAR delineation data from 73 thoracic patients to construct a reference database. All volumetric features of the OAR delineation are computed in three‐dimensional space. Volumetric features of a new OAR are compared with respect to those in the reference database to discern delineation outliers. A multicriteria outlier detection system warns users of specific delineation outliers based on combinations of deviant features. Fifteen independent experimental sets including automatic, propagated, and clinically approved manual delineation sets were used for verification. The verification OARs included manipulations to mimic common errors. Three experts reviewed the experimental sets to identify and classify errors, first without; and then 1 week after with the QA tool.ResultsIn the cohort of manual delineations with manual manipulations, the QA tool detected 94% of the mimicked errors. Overall, it detected 37% of the minor and 85% of the major errors. The QA tool improved reviewer error detection sensitivity from 61% to 68% for minor errors (P = 0.17), and from 78% to 87% for major errors (P = 0.02).ConclusionsThe QA tool assists users to detect potential delineation errors. QA tool integration into clinical procedures may reduce the frequency of inaccurate OAR delineation, and potentially improve safety and quality of radiation treatment planning.