Soppe, Carol L.

Abstract Background PET visual reads for multi‐center studies are conducted centrally by one or a few experts. In CLARiTI, a broader network of raters will perform the reads, necessitating methods to ensure their accuracy and reliability. Method All ADRC sites ( N = 37) will prospectively acquire amyloid PET scans for 800 cognitively unimpaired and 1200 cognitively impaired participants. Each site can use one of four amyloid PET tracers ([ 18 F]florbetaben, [ 18 F]florbetapir, [ 11 C]PIB, [ 18 F]NAV4694). PET scans will be visually interpreted by ten PET neuroimaging experts located across eight ADRCs, using their preferred hardware, software, and file format (DICOM, NIFTI). Calibration requires readers to perform blinded, independent visual interpretation of 180 amyloid PET scans (30 unique and 15 duplicate scans per tracer), previously read and selected by an expert (Table 1). An even mix of visually non‐elevated and elevated scans were selected, with quantitation spanning ‐26‐123 Centiloids (Figure 1), and 15% were visually borderline (similar proportion to ADNI4), with quantitation spanning 4‐48 Centiloids (mean 21). Kruskal‐Wallis H test showed no significant difference in the distribution of Centiloid values for scans in each tracer group, Χ 2 (3)=2.133, p = 0.545. Result Calibration scans were read by a second expert not serving in CLARiTI. There was 97% concordance and almost perfect agreement between readers (κ=0.932, p <0.001; Figure 2). Discordantly read scans were visually borderline elevated ( N = 2), borderline non‐elevated ( N = 1), and negative ( N = 1), with quantitation of 8‐21 Centiloids (mean 14). Intra‐rater reads were 98% concordant and had almost perfect agreement (κ=0.966, p <0.001). These findings hold promise for scaling‐up visual reads in CLARiTI, and we expect ≥80% concordance and substantial to almost perfect agreement (κ≥0.7) for inter/intra‐rater reliability. Conclusion Calibration is essential for accurate/reliable amyloid PET visual reads in large multi‐center studies with multiple raters. Through the calibration process, we will develop methods to improve the adjudication of borderline/challenging scans and formalize methods for visual reads of amyloid PET tracers that are investigational (i.e., [ 11 C]PIB, [ 18 F]NAV4694). Involving multiple raters across different sites, who use diverse hardware/software/file format combinations, will mirror the natural heterogeneity present in real‐world clinical practice. This calibration set can also serve as a standardized training tool for other studies incorporating amyloid PET visual reads.

Abstract Background PET visual reads for multi‐center studies are conducted centrally by one or a few experts. In CLARiTI, a broader network of raters will perform the reads, necessitating methods to ensure their accuracy and reliability. Method All ADRC sites ( N = 37) will prospectively acquire amyloid PET scans for 800 cognitively unimpaired and 1200 cognitively impaired participants. Each site can use one of four amyloid PET tracers ([ 18 F]florbetaben, [ 18 F]florbetapir, [ 11 C]PIB, [ 18 F]NAV4694). PET scans will be visually interpreted by ten PET neuroimaging experts located across eight ADRCs, using their preferred hardware, software, and file format (DICOM, NIFTI). Calibration requires readers to perform blinded, independent visual interpretation of 180 amyloid PET scans (30 unique and 15 duplicate scans per tracer), previously read and selected by an expert (Table 1). An even mix of visually non‐elevated and elevated scans were selected, with quantitation spanning ‐26‐123 Centiloids (Figure 1), and 15% were visually borderline (similar proportion to ADNI4), with quantitation spanning 4‐48 Centiloids (mean 21). Kruskal‐Wallis H test showed no significant difference in the distribution of Centiloid values for scans in each tracer group, Χ 2 (3)=2.133, p = 0.545. Result Calibration scans were read by a second expert not serving in CLARiTI. There was 97% concordance and almost perfect agreement between readers (κ=0.932, p <0.001; Figure 2). Discordantly read scans were visually borderline elevated ( N = 2), borderline non‐elevated ( N = 1), and negative ( N = 1), with quantitation of 8‐21 Centiloids (mean 14). Intra‐rater reads were 98% concordant and had almost perfect agreement (κ=0.966, p <0.001). These findings hold promise for scaling‐up visual reads in CLARiTI, and we expect ≥80% concordance and substantial to almost perfect agreement (κ≥0.7) for inter/intra‐rater reliability. Conclusion Calibration is essential for accurate/reliable amyloid PET visual reads in large multi‐center studies with multiple raters. Through the calibration process, we will develop methods to improve the adjudication of borderline/challenging scans and formalize methods for visual reads of amyloid PET tracers that are investigational (i.e., [ 11 C]PIB, [ 18 F]NAV4694). Involving multiple raters across different sites, who use diverse hardware/software/file format combinations, will mirror the natural heterogeneity present in real‐world clinical practice. This calibration set can also serve as a standardized training tool for other studies incorporating amyloid PET visual reads.

Abstract INTRODUCTION In multicenter Alzheimer's disease studies, amyloid positron emission tomography (PET) visual reads are typically performed centrally by a few experts. Incorporating a broader reader network enhances scalability and generalizability. METHODS Ten neuroimaging experts from eight Alzheimer's Disease Research Centers (ADRCs) visually read 180 amyloid PET scans (30 scans and 15 duplicate scans for each of four tracers, imaged across a wide variety of scanners), using preferred reading software without anatomical imaging or quantitation. Scans were classified as elevated or non‐elevated per tracer‐specific criteria. Inter‐ and intra‐rater agreement was assessed. RESULTS Inter‐rater agreement was substantial (Fleiss’ κ = 0.78), with full consensus on 69% of scans. Inter‐rater reliability was substantial to perfect across tracers (Fleiss’ κ = 0.70–0.87). Intra‐rater agreement was substantial to perfect (Cohen's κ = 0.79‐1). Scans with intermediate (10–40 Centiloid) quantitation had lower reader agreement. DISCUSSION A multicenter expert network achieved substantial agreement classifying amyloid PET scans. These scans provide a standard for reader training and reliability assurance in future studies. Highlights Calibration methods ensure reliable amyloid positron emission tomography (PET) visual reads across multiple raters. Substantial agreement is possible across readers using their preferred tools. Agreement is also substantial regardless of the amyloid PET tracer used. Scans with intermediate (10–40 Centiloid) quantitation have lower reader agreement. The calibration set will become a training tool for amyloid PET visual read studies.