Young, Robert J.

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well- annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied environments and providing the tools for future development of personalized volumetric reports to patients undergoing BM treatment.

Abstract BACKGROUND Accurate molecular diagnosis of infiltrating brainstem tumors, including diffuse midline gliomas and IDH mutant astrocytomas, is essential for prognostication and optimal therapy. Surgical biopsy of the brainstem carries risk of permanent neurological deficits and may yield insufficient or non-diagnostic tissue. We hypothesized that minimally invasive “liquid biopsy” analyzing circulating tumor cell-free DNA (cfDNA) in cerebrospinal fluid (CSF) could represent an alternative molecular diagnostic modality. METHODS We performed MSK-IMPACT, a New York State Department of Health authorized hybridization capture-based next-generation panel DNA sequencing clinical assay on 44 CSF samples from 37 unique patients with brainstem tumors. Ages at time of CSF collection ranged from 1–47 years (median: 29 years). For samples in which diagnostic alterations could not be detected, we performed a series of secondary analyses for known hotspot mutations such as H3F3A, IDH1/2, and BRAF, including manual review for mutant reads below the official clinical reporting threshold, as well as droplet digital PCR (ddPCR). RESULTS Among 44 samples included in this study, 10/44 (23%) samples had mutations detected by MSK-IMPACT using standard clinical calling criteria and 14 of the remaining 34 samples (41%) had supporting evidence of a diagnostically relevant mutation based on manual review. Testing by ddPCR was performed on 11 samples (based on DNA availability), uncovering 4 additional mutations. Overall positive diagnostic yield was 64% (28/44). In biopsied patients, CSF confirmed disease-defining mutations in 80% (16/20) of CSF samples, including H3F3A K28M (9/10; 90%) and IDH1/2 (5/6; 83%). Among 24 unbiopsied patients, a disease-defining mutation was identified in 60% (15/25) of CSF samples. CONCLUSIONS Analysis of CSF cfDNA in patients with brainstem tumors has a high diagnostic yield and obviates the need for tissue biopsy in a majority of patients. A tiered testing approach using next-generation sequencing and ddPCR assays helps maximize diagnostic information and sensitivity.