Purpose Automating quality assurance (QA) for contours generated by automatic algorithms is critical in radiotherapy treatment planning. Manual QA is tedious, time-consuming, and prone to subjective experiences. Automatic segmentation reduces physician workload and improves consistency. However, an effective QA process for these automatic contours remains an unmet need in clinical practice. Materials and Methods The patient data used in this study was derived from the AAPM Thoracic Auto-Segmentation Challenge dataset, including left and right lungs, heart, esophagus, and spinal cord. Two groups of organ-at-risk (OAR) were generated. A ResNet-152 network was used as a feature extractor, and a one-class support vector machine (OC-SVM) was employed to classify contours as ‘high’ or ‘low’ quality. To evaluate the generalizability, we generated low-quality contours using translation and resizing techniques and assessed correlations between detection limits and metrics such as volume, Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), and mean surface distance (MSD). Results The proposed OC-SVM model outperformed binary classifiers n metrics such as balanced accuracy and area under the receiver operating characteristic curve (AUC) . It demonstrated superior performance in detecting various types of contour errors while maintaining high interpretability. Strong correlations were observed between detection limits and contour metrics. Conclusion Our proposed model integrates an attention mechanism with a one-class classification framework to automate QA for OAR delineations. This approach effectively detects diverse types of contour errors with high accuracy, significantly reducing the burden on physicians during radiotherapy planning.