Savvides, Marios

AbstractBackgroundResearch subjects can potentially be re‐identified from de‐identified MRI, CT, and PET brain scans with up to 98% accuracy using Microsoft Azure’s cloud‐based commercial facial recognition software. This showed the need to “de‐face” publicly shared research brain scans. Subsequently, Microsoft has begun restricting its face recognition services, intending to prevent misuse. This study tests a variety of popular open‐source computer vision and facial recognition software packages to measure the re‐identification risk with only unrestricted free software.MethodThis study used brain MRI from Siemens 3T scanners and face photographs from 182 participants (ages 30‐90+) in the Mayo Clinic Study of Aging. Each participant had photos taken with an iPad, which were cropped loosely around the head and converted to black and white. A total of 81 renders (2D reconstruction images; Figure 1) were made from each 3D T2‐FLAIR MRI, simulating various camera and lighting positions. These face renders approximate black and white photos, allowing use with software packages developed for matching faces/features between photographs.Open‐source Python packages (OpenCV, DeepFace, face_recognition, scikit‐learn) and their included pre‐trained models (designed for photographs) were used to match participant photographs to the labelled library of MRI‐based face renders. A correct match was recorded when the top‐scoring render or set of renders automatically chosen by the algorithm was from the MRI of the correct participant.ResultFeature matching with ORB and SWIFT (both from OpenCV) produced no correct matches, despite a prior publication reporting moderate success. The best match rates (Figure 2) were 58‐59% accuracy using face_recognition or DeepFace packages with support vector machines (SVMs). Without SVMs, these packages performed modestly worse (51%‐55%).ConclusionWhile the highest match rates obtained by widely available open‐source packages (roughly 60%) are much lower than those of a limited‐access commercial product (Microsoft Azure, with 98%), they show that actors with only access to no‐cost readily available software could still re‐identify a research subject from a brain scan most of the time. This further highlights the need for image refacing software, such as mri_reface, to protect the privacy and identifies of research volunteers.

AbstractBackgroundResearch subjects can potentially be re‐identified from de‐identified MRI, CT, and PET brain scans with up to 98% accuracy using Microsoft Azure’s cloud‐based commercial facial recognition software. This showed the need to “de‐face” publicly shared research brain scans. Subsequently, Microsoft has begun restricting its face recognition services, intending to prevent misuse. This study tests a variety of popular open‐source computer vision and facial recognition software packages to measure the re‐identification risk with only unrestricted free software.MethodThis study used brain MRI from Siemens 3T scanners and face photographs from 182 participants (ages 30‐90+) in the Mayo Clinic Study of Aging. Each participant had photos taken with an iPad, which were cropped loosely around the head and converted to black and white. A total of 81 renders (2D reconstruction images; Figure 1) were made from each 3D T2‐FLAIR MRI, simulating various camera and lighting positions. These face renders approximate black and white photos, allowing use with software packages developed for matching faces/features between photographs. Open‐source Python packages (OpenCV, DeepFace, face_recognition, scikit‐learn) and their included pre‐trained models (designed for photographs) were used to match participant photographs to the labelled library of MRI‐based face renders. A correct match was recorded when the top‐scoring render or set of renders automatically chosen by the algorithm was from the MRI of the correct participant.ResultFeature matching with ORB and SWIFT (both from OpenCV) produced no correct matches, despite a prior publication reporting moderate success. The best match rates (Figure 2) were 58‐59% accuracy using face_recognition or DeepFace packages with support vector machines (SVMs). Without SVMs, these packages performed modestly worse (51%‐55%).ConclusionWhile the highest match rates obtained by widely available open‐source packages (roughly 60%) are much lower than those of a limited‐access commercial product (Microsoft Azure, with 98%), they show that actors with only access to no‐cost readily available software could still re‐identify a research subject from a brain scan most of the time. This further highlights the need for image refacing software, such as mri_reface, to protect the privacy and identifies of research volunteers.

AbstractBackgroundStudies of aging and dementia often acquire brain MRI and PET images, then de‐identify and share these widely with other researchers. It is known that research participants can potentially be re‐identified from brain MRI and CT using face recognition, but this has not been tested with PET imaging.MethodWe generated face reconstruction images of 182 volunteers using their existing amyloid (PiB), tau (FTP), and FDG PET scans from clinical GE PET/CT scanners, and we measured how often commercial automatic face recognition (Microsoft Azure) correctly matched these reconstructions with photographs of their faces. We additionally compared these match rates with face reconstructions from CT (from PET/CT) and MRI. When testing face recognition from PET scans, CT was used only during PET reconstruction, not for face recognition. We also validated our updated mri_reface software, originally developed to de‐identify (“de‐face”) faces in MRI, with new support for PET images. We measured face recognition rates before and after applying mri_reface to PET, CT, and MRI. In addition, we measured effects of de‐facing on biomarker measurements (global and regional SUVR) using 244 ADNI Florbetapir (FBP) amyloid PET scans with two automated pipelines: FreeSurfer 6.0 (PETSurfer) and an in‐house SPM12‐based pipeline.ResultRates of successful face recognition with PET ranged from 32‐41% (Table 1) across tracers. These were lower than MRI (98%) and CT (78%) but still concerning for participant privacy. Using mri_reface reduced face recognition to 0‐3.5% for PET, 5% for CT, and 8% for MRI. mri_reface had little effect on global amyloid SUVR measurements from ADNI PET images (Figure 1). ICC values between original and de‐faced images were 1.00 with biases and median absolute differences < 0.5%, across both pipelines. Effects on individual regions all had ICC values > 0.98 with biases and median absolute differences < 2%.ConclusionFace reconstructions from PET images have sufficient quality for potential re‐identification using publicly available face recognition technology. Research studies should consider using face de‐identification (de‐facing) software on PET images in addition to CT and structural MRI. Our free, automatic mri_reface software greatly reduced the potential for face recognition without substantially affecting SUVR measurements.

AbstractBackgroundStudies of aging and dementia often acquire brain MRI and PET images, then de‐identify and share these widely with other researchers. It is known that research participants can potentially be re‐identified from brain MRI and CT using face recognition, but this has not been tested with PET imaging.MethodWe generated face reconstruction images of 182 volunteers using their existing amyloid (PiB), tau (FTP), and FDG PET scans from clinical GE PET/CT scanners, and we measured how often commercial automatic face recognition (Microsoft Azure) correctly matched these reconstructions with photographs of their faces. We additionally compared these match rates with face reconstructions from CT (from PET/CT) and MRI. When testing face recognition from PET scans, CT was used only during PET reconstruction, not for face recognition. We also validated our updated mri_reface software, originally developed to de‐identify (“de‐face”) faces in MRI, with new support for PET images. We measured face recognition rates before and after applying mri_reface to PET, CT, and MRI. In addition, we measured effects of de‐facing on biomarker measurements (global and regional SUVR) using 244 ADNI Florbetapir (FBP) amyloid PET scans with two automated pipelines: FreeSurfer 6.0 (PETSurfer) and an in‐house SPM12‐based pipeline.ResultRates of successful face recognition with PET ranged from 32‐41% (Table 1) across tracers. These were lower than MRI (98%) and CT (78%) but still concerning for participant privacy. Using mri_reface reduced face recognition to 0‐3.5% for PET, 5% for CT, and 8% for MRI. mri_reface had little effect on global amyloid SUVR measurements from ADNI PET images (Figure 1). ICC values between original and de‐faced images were 1.00 with biases and median absolute differences < 0.5%, across both pipelines. Effects on individual regions all had ICC values > 0.98 with biases and median absolute differences < 2%.ConclusionFace reconstructions from PET images have sufficient quality for potential re‐identification using publicly available face recognition technology. Research studies should consider using face de‐identification (de‐facing) software on PET images in addition to CT and structural MRI. Our free, automatic mri_reface software greatly reduced the potential for face recognition without substantially affecting SUVR measurements.