Storrs, Judd M

AbstractMRI has become an indispensable tool for brain volumetric studies, with the hippocampus an important region of interest. Automation of the MRI segmentation process has helped advance the field by facilitating the volumetric analysis of larger cohorts and more studies. FreeSurfer has emerged as the de facto standard tool for these analyses, but studies validating its output are all based on older versions. To characterize FreeSurfer's validity, we compare several versions of FreeSurfer software with traditional hand‐tracing. Using MRI images of 262 males and 402 females aged 38 to 84, we directly compare estimates of hippocampal volume from multiple versions of FreeSurfer, its hippocampal subfield routines, and our manual tracing protocol. We then use those estimates to assess asymmetry and atrophy, comparing performance of different estimators with each other and with brain atrophy measures. FreeSurfer consistently reports larger volumes than manual tracing. This difference is smaller in larger hippocampi or older people, with these biases weaker in version 6.0.0 than prior versions. All methods tested agree qualitatively on rightward asymmetry and increasing atrophy in older people. FreeSurfer saves time and money, and approximates the same atrophy measures as manual tracing, but it introduces biases that could require statistical adjustments in some studies.

432 Background: In clinical trials and clinical practice, tumor response assessment with computed tomography (CT) defines critical end points in patients with metastatic disease treated with systemic agents. Methods to reduce errors and improve efficiency in tumor response assessment could improve patient care. Methods: Eleven readers from 10 different institutions independently categorized tumor response according to three different therapeutic response criteria using paired baseline and initial post-therapy CT studies from 20 randomly selected patients with metastatic renal cell carcinoma treated with sunitinib as part of a completed phase III multi-institutional study. Images were evaluated with a manual tumor response evaluation method (standard-of-care) and with computer-assisted response evaluation (CARE) that included stepwise guidance, interactive error-identification and correction methods, automated tumor metric extraction, calculations, response categorization, and data/image archival. A cross-over design, patient randomization, and two-week washout period were used to reduce recall bias. Comparative effectiveness metrics included error rate and mean patient evaluation time. Results: The standard-of-care method was on average associated with one or more errors in 30.5% (6.1/20) of patients while CARE had a 0.0% (0.0/20) error rate (p<0.001). The most common errors were related to data transfer and arithmetic calculation. In patients with errors, the median number of error types was 1 (range 1-3). Mean patient evaluation time with CARE was twice as fast as the standard-of-care method (6.4 vs. 13.1 minutes, p<0.001). Conclusions: Computer-assisted tumor response evaluation reduced errors and time of evaluation, indicating better overall effectiveness than manual tumor response evaluation methods that are the current standard-of-care.