Prakaashana, Carl M.

Abstract Background Use of software to “de‐face” research brain images has grown as advances in automated face recognition have made it increasingly possible to re‐identify participants. Imaging studies of Alzheimer's Disease and related diseases have included more participants from historically underrepresented groups, but de‐facing software have primarily been validated using images from mostly non‐Hispanic White participants, and its performance has not been directly compared across ethnoracial groups. Method From The Health and Aging Brain Study: Health Disparities (HABS‐HD) study, we constructed a sample of 305 T1‐weighted MRI from HABS‐HD baseline visits, matched across three ethnoracial groups (non‐Hispanic White (NHW, n = 104), non‐Hispanic African American (AA, n = 104), Mexican American (MA, n = 97)) according to age‐decade, sex, cognitive status (impaired/unimpaired), and MRI scanner model. We ran mri_reface 0.3.5 on each image, and we ran FreeSurfer 7.4.1 on each unmodified image and on each de‐faced output. We then compared regional gray matter (GM) volume and cortical thickness measurements by subtracting each de‐faced image measurement from the corresponding unmodified image measurement. We compared these differences between de‐faced and unmodified images across the three matched groups using GLM analyses adjusted for age, sex, and cognitive status, and Bonferroni corrected. Result When aggregating across all brain regions, differences in cortical thickness measures due to de‐facing were significantly different between the NHW and MA groups, and between AA and MA, but not between NHW and AA (Figure 1). Differences between these group means were <0.005mm. For GM volume, differences were significant between AA and MA (magnitude < 0.2%), but not between NHW and AA or NHW and MA (Figure 2). Examining each region individually (Figure 3), cortical thickness differences were significant for only 4 regions (all with magnitudes <0.03mm), and GM volume differences were significant for only 6 regions (all with magnitudes <2%). Conclusion Effects of mri_reface on FreeSurfer brain measurements were significantly different across ethnoracial groups. Our future work will optimize mri_reface to eliminate these differences. However, the magnitudes of these differences are sufficiently small that they are unlikely to have a practical effect on analyses of data using current versions.

Abstract Background Use of software to “de‐face” research brain images has grown as advances in automated face recognition have made it increasingly possible to re‐identify participants. Imaging studies of Alzheimer’s Disease and related diseases have included more participants from historically underrepresented groups, but de‐facing software have primarily been validated using images from mostly non‐Hispanic White participants, and its performance has not been directly compared across ethnoracial groups. Method From The Health and Aging Brain Study: Health Disparities (HABS‐HD) study, we constructed a sample of 305 T1‐weighted MRI from HABS‐HD baseline visits, matched across three ethnoracial groups (non‐Hispanic White (NHW, n = 104), non‐Hispanic African American (AA, n = 104), Mexican American (MA, n = 97)) according to age‐decade, sex, cognitive status (impaired/unimpaired), and MRI scanner model. We ran mri_reface 0.3.5 on each image, and we ran FreeSurfer 7.4.1 on each unmodified image and on each de‐faced output. We then compared regional gray matter (GM) volume and cortical thickness measurements by subtracting each de‐faced image measurement from the corresponding unmodified image measurement. We compared these differences between de‐faced and unmodified images across the three matched groups using GLM analyses adjusted for age, sex, and cognitive status, and Bonferroni corrected. Result When aggregating across all brain regions, differences in cortical thickness measures due to de‐facing were significantly different between the NHW and MA groups, and between AA and MA, but not between NHW and AA (Figure 1). Differences between these group means were <0.005mm. For GM volume, differences were significant between AA and MA (magnitude < 0.2%), but not between NHW and AA or NHW and MA (Figure 2). Examining each region individually (Figure 3), cortical thickness differences were significant for only 4 regions (all with magnitudes <0.03mm), and GM volume differences were significant for only 6 regions (all with magnitudes <2%). Conclusion Effects of mri_reface on FreeSurfer brain measurements were significantly different across ethnoracial groups. Our future work will optimize mri_reface to eliminate these differences. However, the magnitudes of these differences are sufficiently small that they are unlikely to have a practical effect on analyses of data using current versions.

AbstractBackgroundRecent advances in automatic face recognition have increased the risk that de‐identified research imaging data could be re‐identified from face imagery in brain scans.MethodAn ADNI committee of independent imaging experts evaluated 11 published techniques for face‐deidentification (“de‐facing”) and selected four algorithms (FSL‐UK Biobank, HCP/XNAT, mri_reface, and BIC) for formal testing using 183 longitudinal scans of 61 racially and ethnically diverse ADNI participants, evaluated by their facial feature removal on 3D rendered surfaces (confirming sufficient privacy protection) and by comparing measurements from ADNI routine image analyses on unmodified vs. de‐faced images (confirming negligible side effects on analyses).ResultEffects of the de‐facing methods on FreeSurfer at UCSF (Figure 1‐top), ASHS at University of Pennsylvania (Figure 1 bottom‐left), and longitudinal atrophy at Mayo Clinic (Figure 1 bottom‐right) were comparable with each other and sufficiently small vs. unmodified scans to recommend de‐facing in ADNI‐4. The FSL and mri_reface methods were more thorough in removing facial features, partly because these also removed ears (Figure 2‐top). mri_reface was recommended for ADNI‐4 primarily because fewer (zero) images had modified brain (Figure 2‐bottom), and ADNI leadership approved de‐facing with mri_reface. ADNI performed a second validation of its latest version on another diverse sample of 100 ADNI‐3 participants. These analyses included: boundary shift interval at University College London, where average differences in measurements were all <0.05%; cortical white matter hyperintensities at UC Davis, where they found “minimum impact” (R>0.97); and others.ConclusionADNI‐4 de‐faces all indicated image types (Table 1) before subsequent pre‐processing, analyses, and public release. Sites upload images to the central repository (LONI), where they are queued to the MRI Core at Mayo Clinic for de‐facing. Image types requiring de‐facing are de‐faced using mri_reface. Trained analysts inspect de‐faced images to confirm complete removal of the face and complete non‐modification of the brain, and any flagged images are re‐processed manually until issues are resolved. De‐faced images are converted back to DICOM and re‐uploaded for further distribution. Image types not requiring de‐facing are indicated to proceed directly to subsequent steps. In summary, ADNI‐4 has adopted mri_reface because it successfully de‐faces without substantially affecting brain measurements.

AbstractBackgroundResearch subjects can potentially be re‐identified from de‐identified MRI, CT, and PET brain scans with up to 98% accuracy using Microsoft Azure’s cloud‐based commercial facial recognition software. This showed the need to “de‐face” publicly shared research brain scans. Subsequently, Microsoft has begun restricting its face recognition services, intending to prevent misuse. This study tests a variety of popular open‐source computer vision and facial recognition software packages to measure the re‐identification risk with only unrestricted free software.MethodThis study used brain MRI from Siemens 3T scanners and face photographs from 182 participants (ages 30‐90+) in the Mayo Clinic Study of Aging. Each participant had photos taken with an iPad, which were cropped loosely around the head and converted to black and white. A total of 81 renders (2D reconstruction images; Figure 1) were made from each 3D T2‐FLAIR MRI, simulating various camera and lighting positions. These face renders approximate black and white photos, allowing use with software packages developed for matching faces/features between photographs.Open‐source Python packages (OpenCV, DeepFace, face_recognition, scikit‐learn) and their included pre‐trained models (designed for photographs) were used to match participant photographs to the labelled library of MRI‐based face renders. A correct match was recorded when the top‐scoring render or set of renders automatically chosen by the algorithm was from the MRI of the correct participant.ResultFeature matching with ORB and SWIFT (both from OpenCV) produced no correct matches, despite a prior publication reporting moderate success. The best match rates (Figure 2) were 58‐59% accuracy using face_recognition or DeepFace packages with support vector machines (SVMs). Without SVMs, these packages performed modestly worse (51%‐55%).ConclusionWhile the highest match rates obtained by widely available open‐source packages (roughly 60%) are much lower than those of a limited‐access commercial product (Microsoft Azure, with 98%), they show that actors with only access to no‐cost readily available software could still re‐identify a research subject from a brain scan most of the time. This further highlights the need for image refacing software, such as mri_reface, to protect the privacy and identifies of research volunteers.

AbstractBackgroundSeveral programs are available for automatically “de‐facing” brain images. These are designed to prevent the potential use of face imagery to re‐identify research participants. Previous works have compared available software and mri_reface is frequently among the top methods. The “Minimally Invasive DeFacing of MRI images” (MiDeFace) program was more recently added to FreeSurfer, but minimal validation of MiDeFace has been published.MethodWe ran mri_reface 0.3.3 and MiDeFace 7.3.2 on two datasets: A) 207 participants with face photographs and 3D T1‐weighted and 3D T2‐weighted‐FLAIR MRI from the Mayo Clinic Study of Aging (all unimpaired, Siemens); and B) 300 participants from ADNI3 with 3D T1‐weighted MRI (50 unimpaired + 50 with clinical AD, from each of 3 vendors). For dataset A, we tested the rate at which Microsoft Azure face recognition matched each participant’s photographs with the correct MRI, comparing unmodified images, those from mri_reface, and those from MiDeFace. For dataset B, we compared measured regional brain volume and cortical thickness from original vs. de‐faced images with SPM12, FreeSurfer 7.4.1, and the FSL‐UK Biobank pipeline. For both datasets, images were visually rated to assess removal of the face and retention of brain tissue.ResultAutomated face recognition match rates were lower with mri_reface (17/207 8.2%) than with MiDeFace (24/207 11.6%), indicating stronger identity protection with mri_reface, but both were greatly reduced vs. unmodified images (203/207 98.1%). Example images with both programs are shown in Figure 1. From visual ratings (Table 1), rates of complete success (modification of the entire face while retaining the entire brain) were much higher with mri_reface, but the great majority of failures from MiDeFace were subtle and occurred in relatively less recognizable face regions. The effects of both programs on downstream imaging measurements (Figure 2) were small and very comparable, but MiDeFace was favored mildly for SPM12 and for FSL, while mri_reface was more favored for FreeSurfer.ConclusionAltogether, mri_reface more reliably removed the complete face, leading to smaller rates of potential re‐identification (8.2% vs 11.6%). Effects on downstream image measurements were small and comparable with both methods.

Abstract Background The Mayo Clinic Study of Aging (MCSA) is a longitudinal, population‐based study of residents of Olmsted County, Minnesota, USA. MCSA is releasing de‐identified clinical and imaging data on GAAIN.org to benefit the research community. Method We included longitudinal clinical data from all participants of the Mayo Clinic Study of Aging 30‐90 years of age (average 70.0y) through the first 15 years of the study. Clinical data includes age, sex, self‐reported race/ethnicity, APoE4 allele status, cognitive impairment status (normal, MCI, dementia, or other), height, weight, blood pressure, multi‐domain neuropsychological scores, neuroimaging summary measurements, medications, cardiovascular risk factors, social activity, sleep, mood/anxiety, functional measures, and more. For our initial release of imaging data, we selected the first imaging timepoint acquired on 3T GE MRI scanners and amyloid (PiB) PET during the same visit, from all available imaging participants 50‐90 years of age (average 70.7y). Currently, we included 3D T1‐weighted (MPRAGE) MRI (1x1x1.2mm 3 ), T2‐weighted FLAIR MRI (0.85x0.85x3.6mm 3 ), diffusion MRI (2.7mm resolution with 41 directions at b=1000 and 5 b=0), and numeric cortical global SUVR and Centiloid values from PiB PET scans at 40‐60 minutes post‐injection. All images are released in DICOM format and thoroughly de‐identified using CTP while ensuring preservation of necessary fields for analyses. T1‐weighted and T2‐FLAIR MRI were also de‐faced (identifiable face imagery was removed and replaced with an average) using mri_reface version 0.3.3 to further protect participants’ privacy, and all de‐faced images were visually inspected by a trained data scientist for quality assurance. Result The first phase of our public data release includes clinical data from 5925 unique participants from the Mayo Clinic Study of Aging (MCSA), each ranging from 1‐12 visits at roughly 15‐month intervals, and three MRI sequences (T1‐w, T2‐w‐FLAIR, and diffusion) from 1802 unique participants. Demographic summaries are given in Table 1. Conclusion This large, richly characterized, de‐identified clinical and imaging dataset is available to the research community at https://www.gaaindata.org/partner/MCSA , and it is suitable for a wide range of research uses. Additional data will be released over time, including de‐faced amyloid PET images.

AbstractBackgroundResearch subjects can potentially be re‐identified from de‐identified MRI, CT, and PET brain scans with up to 98% accuracy using Microsoft Azure’s cloud‐based commercial facial recognition software. This showed the need to “de‐face” publicly shared research brain scans. Subsequently, Microsoft has begun restricting its face recognition services, intending to prevent misuse. This study tests a variety of popular open‐source computer vision and facial recognition software packages to measure the re‐identification risk with only unrestricted free software.MethodThis study used brain MRI from Siemens 3T scanners and face photographs from 182 participants (ages 30‐90+) in the Mayo Clinic Study of Aging. Each participant had photos taken with an iPad, which were cropped loosely around the head and converted to black and white. A total of 81 renders (2D reconstruction images; Figure 1) were made from each 3D T2‐FLAIR MRI, simulating various camera and lighting positions. These face renders approximate black and white photos, allowing use with software packages developed for matching faces/features between photographs. Open‐source Python packages (OpenCV, DeepFace, face_recognition, scikit‐learn) and their included pre‐trained models (designed for photographs) were used to match participant photographs to the labelled library of MRI‐based face renders. A correct match was recorded when the top‐scoring render or set of renders automatically chosen by the algorithm was from the MRI of the correct participant.ResultFeature matching with ORB and SWIFT (both from OpenCV) produced no correct matches, despite a prior publication reporting moderate success. The best match rates (Figure 2) were 58‐59% accuracy using face_recognition or DeepFace packages with support vector machines (SVMs). Without SVMs, these packages performed modestly worse (51%‐55%).ConclusionWhile the highest match rates obtained by widely available open‐source packages (roughly 60%) are much lower than those of a limited‐access commercial product (Microsoft Azure, with 98%), they show that actors with only access to no‐cost readily available software could still re‐identify a research subject from a brain scan most of the time. This further highlights the need for image refacing software, such as mri_reface, to protect the privacy and identifies of research volunteers.

AbstractBackgroundSeveral programs are available for automatically “de‐facing” brain images. These are designed to prevent the potential use of face imagery to re‐identify research participants. Previous works have compared available software and mri_reface is frequently among the top methods. The “Minimally Invasive DeFacing of MRI images” (MiDeFace) program was more recently added to FreeSurfer, but minimal validation of MiDeFace has been published.MethodWe ran mri_reface 0.3.3 and MiDeFace 7.3.2 on two datasets: A) 207 participants with face photographs and 3D T1‐weighted and 3D T2‐weighted‐FLAIR MRI from the Mayo Clinic Study of Aging (all unimpaired, Siemens); and B) 300 participants from ADNI3 with 3D T1‐weighted MRI (50 unimpaired + 50 with clinical AD, from each of 3 vendors). For dataset A, we tested the rate at which Microsoft Azure face recognition matched each participant’s photographs with the correct MRI, comparing unmodified images, those from mri_reface, and those from MiDeFace. For dataset B, we compared measured regional brain volume and cortical thickness from original vs. de‐faced images with SPM12, FreeSurfer 7.4.1, and the FSL‐UK Biobank pipeline. For both datasets, images were visually rated to assess removal of the face and retention of brain tissue.ResultAutomated face recognition match rates were lower with mri_reface (17/207 8.2%) than with MiDeFace (24/207 11.6%), indicating stronger identity protection with mri_reface, but both were greatly reduced vs. unmodified images (203/207 98.1%). Example images with both programs are shown in Figure 1. From visual ratings (Table 1), rates of complete success (modification of the entire face while retaining the entire brain) were much higher with mri_reface, but the great majority of failures from MiDeFace were subtle and occurred in relatively less recognizable face regions. The effects of both programs on downstream imaging measurements (Figure 2) were small and very comparable, but MiDeFace was favored mildly for SPM12 and for FSL, while mri_reface was more favored for FreeSurfer.ConclusionAltogether, mri_reface more reliably removed the complete face, leading to smaller rates of potential re‐identification (8.2% vs 11.6%). Effects on downstream image measurements were small and comparable with both methods.

AbstractBackgroundThe Mayo Clinic Study of Aging (MCSA) is a longitudinal, population‐based study of residents of Olmsted County, Minnesota, USA. MCSA is releasing de‐identified clinical and imaging data on GAAIN.org to benefit the research community.MethodWe included longitudinal clinical data from all participants of the Mayo Clinic Study of Aging 30‐90 years of age (average 70.0y) through the first 15 years of the study. Clinical data includes age, sex, self‐reported race/ethnicity, APoE4 allele status, cognitive impairment status (normal, MCI, dementia, or other), height, weight, blood pressure, multi‐domain neuropsychological scores, neuroimaging summary measurements, medications, cardiovascular risk factors, social activity, sleep, mood/anxiety, functional measures, and more. For our initial release of imaging data, we selected the first imaging timepoint acquired on 3T GE MRI scanners and amyloid (PiB) PET during the same visit, from all available imaging participants 50‐90 years of age (average 70.7y). Currently, we included 3D T1‐weighted (MPRAGE) MRI (1x1x1.2mm3), T2‐weighted FLAIR MRI (0.85x0.85x3.6mm3), diffusion MRI (2.7mm resolution with 41 directions at b=1000 and 5 b=0), and numeric cortical global SUVR and Centiloid values from PiB PET scans at 40‐60 minutes post‐injection. All images are released in DICOM format and thoroughly de‐identified using CTP while ensuring preservation of necessary fields for analyses. T1‐weighted and T2‐FLAIR MRI were also de‐faced (identifiable face imagery was removed and replaced with an average) using mri_reface version 0.3.3 to further protect participants’ privacy, and all de‐faced images were visually inspected by a trained data scientist for quality assurance.ResultThe first phase of our public data release includes clinical data from 5925 unique participants from the Mayo Clinic Study of Aging (MCSA), each ranging from 1‐12 visits at roughly 15‐month intervals, and three MRI sequences (T1‐w, T2‐w‐FLAIR, and diffusion) from 1802 unique participants. Demographic summaries are given in Table 1.ConclusionThis large, richly characterized, de‐identified clinical and imaging dataset is available to the research community at https://www.gaaindata.org/partner/MCSA, and it is suitable for a wide range of research uses. Additional data will be released over time, including de‐faced amyloid PET images.

AbstractBackgroundRecent advances in automatic face recognition have increased the risk that de‐identified research imaging data could be re‐identified from face imagery in brain scans.MethodAn ADNI committee of independent imaging experts evaluated 11 published techniques for face‐deidentification (“de‐facing”) and selected four algorithms (FSL‐UK Biobank, HCP/XNAT, mri_reface, and BIC) for formal testing using 183 longitudinal scans of 61 racially and ethnically diverse ADNI participants, evaluated by their facial feature removal on 3D rendered surfaces (confirming sufficient privacy protection) and by comparing measurements from ADNI routine image analyses on unmodified vs. de‐faced images (confirming negligible side effects on analyses).ResultEffects of the de‐facing methods on FreeSurfer at UCSF (Figure 1‐top), ASHS at University of Pennsylvania (Figure 1 bottom‐left), and longitudinal atrophy at Mayo Clinic (Figure 1 bottom‐right) were comparable with each other and sufficiently small vs. unmodified scans to recommend de‐facing in ADNI‐4. The FSL and mri_reface methods were more thorough in removing facial features, partly because these also removed ears (Figure 2‐top). mri_reface was recommended for ADNI‐4 primarily because fewer (zero) images had modified brain (Figure 2‐bottom), and ADNI leadership approved de‐facing with mri_reface. ADNI performed a second validation of its latest version on another diverse sample of 100 ADNI‐3 participants. These analyses included: boundary shift interval at University College London, where average differences in measurements were all <0.05%; cortical white matter hyperintensities at UC Davis, where they found “minimum impact” (R>0.97); and others.ConclusionADNI‐4 de‐faces all indicated image types (Table 1) before subsequent pre‐processing, analyses, and public release. Sites upload images to the central repository (LONI), where they are queued to the MRI Core at Mayo Clinic for de‐facing. Image types requiring de‐facing are de‐faced using mri_reface. Trained analysts inspect de‐faced images to confirm complete removal of the face and complete non‐modification of the brain, and any flagged images are re‐processed manually until issues are resolved. De‐faced images are converted back to DICOM and re‐uploaded for further distribution. Image types not requiring de‐facing are indicated to proceed directly to subsequent steps. In summary, ADNI‐4 has adopted mri_reface because it successfully de‐faces without substantially affecting brain measurements.