Memon, Fatima

Abstract BACKGROUND Volumetric analysis of brain metastatic lesions is of great relevance in neuro-oncology. Since manual annotation of a lesion is a laborious and time-consuming task, utilizing artificial intelligence (AI) tools for automated segmentation of brain metastases has shown promising potential, but published algorithms are not incorporated into clinical practice because of poor generalizability across institutions. Large publicly available and well-annotated datasets from across the world are critical for further clinical translation. MATERIAL AND METHODS The Brain Tumor Segmentation - Metastases (BraTS-METS) 2025 Lighthouse Challenge provides a competitive environment where scientists from various disciplines can benchmark their AI-based algorithms for automatic segmentation of pre- and post-treatment brain metastases on a well-curated publicly available brain metastasis dataset comprising 1,778 multiparametric brain MRI cases. Each case consists of T1-weighed, T1 post-contrast and FLAIR sequences. The cases were retrospectively obtained from 8 institutions in the USA and overseas, and underwent 3D nnU-Net pre-segmentation, followed by manual refinements performed by medical students and neuroradiologists, and a final approval by two independent board-certified neuroradiologists. A testing dataset was prepared through four rounds of annotation: two completed ‘from scratch’ and two following 3D nnU-Net pre-segmentation, all conducted by two independent neuroradiologists, with the segmentation process screen recorded for future analysis. This allowed us to establish inter- and intra-rater variability in brain metastases segmentation and create a reference standard dataset that will be publicly available after the conclusion of the challenge. The model performance of the participating teams in the challenge will be assessed via the following metrics: Dice Similarity Coefficient (DSC), Normalized Surface Distance (NSD), sensitivity, and precision of the model. RESULTS A multi-annotator testing dataset of 75 MRI cases of brain metastases and 300 segmentations (150 created ‘from scratch’ and 150 from 3D nnU-Net pre-segmentations of the cases) was developed as a new reference standard. The mean intra- and inter-rater DSC for the segmentations ‘from scratch’ were 0.68 and 0.59, respectively. The mean intra- and inter-rater DSC for the segmentations built upon 3D nnU-Net pre-segmentations were 0.88 and 0.77, respectively. CONCLUSION The BraTS-METS 2025 Lighthouse Challenge is a key initiative designed to advance the use of AI to improve characterization of brain metastases by providing algorithms for automatic segmentation of all lesions and peritumoral edema that are measured against a novel reference standard based on rigorous multi-annotator brain MRI segmentations.

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well- annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied environments and providing the tools for future development of personalized volumetric reports to patients undergoing BM treatment.

Abstract Background Although response in pediatric low-grade glioma (pLGG) includes volumetric assessment, more simplified 2D-based methods are often used in clinical trials. The study’s purpose was to compare volumetric to 2D methods. Methods An expert neuroradiologist performed solid and whole tumor (including cyst and edema) volumetric measurements on MR images using a PACS-based manual segmentation tool in 43 pLGG participants (213 total follow-up images) from the Pacific Pediatric Neuro-Oncology Consortium (PNOC-001) trial. Classification based on changes in volumetric and 2D measurements of solid tumor were compared to neuroradiologist visual response assessment using the Brain Tumor Reporting and Data System (BT-RADS) criteria for a subset of 65 images using receiver operating characteristic (ROC) analysis. Longitudinal modeling of solid tumor volume was used to predict BT-RADS classification in 54 of the 65 images. Results There was a significant difference in ROC area under the curve between 3D solid tumor volume and 2D area (0.96 vs 0.78, P = .005) and between 3D solid and 3D whole volume (0.96 vs 0.84, P = .006) when classifying BT-RADS progressive disease (PD). Thresholds of 15–25% increase in 3D solid tumor volume had an 80% sensitivity in classifying BT-RADS PD included in their 95% confidence intervals. The longitudinal model of solid volume response had a sensitivity of 82% and a positive predictive value of 67% for detecting BT-RADS PD. Conclusions Volumetric analysis of solid tumor was significantly better than 2D measurements in classifying tumor progression as determined by BT-RADS criteria and will enable more comprehensive clinical management.

Abstract Radiographic response assessment in neuro-oncology is critical in clinical practice and trials. Conventional criteria, such as the MacDonald and response assessment in neuro-oncology (RANO) criteria, rely on bidimensional (2D) measurements of a single tumor cross-section. Although RANO criteria are established for response assessment in clinical trials, there is a critical need to address the complexity of brain tumor treatment response with multiple new approaches being proposed. These include volumetric analysis of tumor compartments, structured MRI reporting systems like the Brain Tumor Reporting and Data System, and standardized approaches to advanced imaging techniques to distinguish tumor response from treatment effects. In this review, we discuss the strengths and limitations of different neuro-oncology response criteria and summarize current research findings on the role of novel response methods in neuro-oncology clinical trials and practice.