Assessment of the Inter-observer Variability of the Radiomic Tools: BRODERS (Benign vs aggressive nODule Evaluation using Radiomic Stratification) and SILA (Score Indicative of Lung cancer Aggression) to Classify Lung Nodules
Abstract Introduction: While CT lung cancer screening reduces lung cancer-specific mortality, there are remaining challenges. Radiomic tools promiss to address these challenges, however, they are subject to interobserver variability if semi-automated segmentation techniques are used. Herein we report interobserver variability for two validated radiomic tools, BRODERS (Benign versus aggRessive nODule Evaluation using Radiomic Stratification) and CANARY (Computer-Aided Nodule Assessment and Risk Yield). Methods We retrospectively analyzed the CT images of 95 malignant lung nodules of the adenocarcinoma spectrum using BRODERS and CANARY. Cases were identified at Mayo Clinic (n = 45) and Vanderbilt University Medical Center and Nashville/Veteran Administration Tennessee Valley Health Care System (n = 50). Three observers with different training levels (medical student, internal medicine resident and thoracic radiology fellow) each performed lung nodule segmentation. All methods were carried out in accordance with relevant guidelines and regulations Results Interclass correlation coefficients (ICC) of 0.77, 0.98 and 0.97 for the average nodule volume, BRODERS cancer probability and Score Indicative of Lesion Aggression (SILA) which summarizes the distribution of the CANARY exemplars indicated good to excellent reliability, respectively. The dice similarity coefficient was 0.79 and 0.81 for the data sets from the two institutions. Conclusion: BRODERS and CANARY are robust radiomics tools with excellent interobserver variability. These tools are simple and reliable regardless the observer/operator’s level of training.
Assessment of Interstitial Lung Disease Using Lung Ultrasound Surface Wave Elastography
Purpose: Optimal strategies to detect early interstitial lung disease (ILD) are unknown. ILD is frequently subpleural in distribution and affects lung elasticity. Lung ultrasound surface wave elastography (LUSWE) is a noninvasive method of quantifying superficial lung tissue elastic properties. In LUWSE a handheld device applied at the intercostal space vibrates the chest at a set frequency, and the lung surface wave velocity is measured by an ultrasound probe 5 mm away in the same intercostal space. We explored LUWSE’s ability to detect ILD and correlated LUSWE velocity with physiological, quantitative, and visual radiologic features of subjects with known ILD and of healthy controls. Materials and Methods: Seventy-seven subjects with ILD, mostly caused by connective tissue disease, and 19 healthy controls were recruited. LUSWE was performed on all subjects in 3 intercostal lung regions bilaterally. Comparison of LUSWE velocities pulmonary function testing, visual assessment, and quantitative analysis of recent computed tomographic imaging with Computer-Aided Lung Informatics for Pathology Evaluation and Rating (CALIPER) software. Results: Sonographic velocities were higher in all lung regions for cases, with the greatest difference in the lateral lower lung. Median velocity in m/s was 5.84 versus 4.11 and 5.96 versus 4.27 (P<0.00001) for cases versus controls, left and right lateral lower lung zones, respectively. LUSWE velocity correlated negatively with vital capacity and positively with radiologist and CALIPER-detected interstitial abnormalities. Conclusions: LUSWE is a safe and noninvasive technique that shows high sensitivity to detect ILD and correlated with clinical, physiological, radiologic, and quantitative assessments of ILD. Prospective study in detecting ILD is indicated.
Predicting Outcomes in Idiopathic Pulmonary Fibrosis Using Automated Computed Tomographic Analysis
Abstract Rationale Quantitative computed tomographic (CT) measures of baseline disease severity might identify patients with idiopathic pulmonary fibrosis (IPF) with an increased mortality risk. We evaluated whether quantitative CT variables could act as a cohort enrichment tool in future IPF drug trials. Objectives To determine whether computer-derived CT measures, specifically measures of pulmonary vessel–related structures (VRSs), can better predict functional decline and survival in IPF and reduce requisite sample sizes in drug trial populations. Methods Patients with IPF undergoing volumetric noncontrast CT imaging at the Royal Brompton Hospital, London, and St. Antonius Hospital, Utrecht, were examined to identify pulmonary function measures (including FVC) and visual and computer-derived (CALIPER [Computer-Aided Lung Informatics for Pathology Evaluation and Rating] software) CT features predictive of mortality and FVC decline. The discovery cohort comprised 247 consecutive patients, with validation of results conducted in a separate cohort of 284 patients, all fulfilling drug trial entry criteria. Measurements and Main Results In the discovery and validation cohorts, CALIPER-derived features, particularly VRS scores, were among the strongest predictors of survival and FVC decline. CALIPER results were accentuated in patients with less extensive disease, outperforming pulmonary function measures. When used as a cohort enrichment tool, a CALIPER VRS score greater than 4.4% of the lung was able to reduce the requisite sample size of an IPF drug trial by 26%. Conclusions Our study has validated a new quantitative CT measure in patients with IPF fulfilling drug trial entry criteria—the VRS score—that outperformed current gold standard measures of outcome. When used for cohort enrichment in an IPF drug trial setting, VRS threshold scores can reduce a required IPF drug trial population size by 25%, thereby limiting prohibitive trial costs. Importantly, VRS scores identify patients in whom antifibrotic medication prolongs life and reduces FVC decline.
Abstract 3723: Assessing the reproducibility of computer-aided nodule assessment and risk yield (CANARY) method to characterize lung adenocarcinomas
Abstract This study assessed the inter-user reproducibility of Computer-Aided Nodule Assessment and Risk Yield (CANARY), a novel analytical tool that risk stratifies lung adenocarcinomas (ADCs) according to defined computed tomography (CT) structural characteristics. CANARY detects nine distinct voxel signatures in ADCs based on standard CT imaging, thereby defining nodule characteristics correlating with patient outcomes, and corresponding to invasion or lepidic growth histologically. A software user segments the borders of each ADC prior to voxel analysis, introducing potential variability into the assessment. While CANARY is a promising method, it requires validation of the analytical variability between users and prediction of accuracy in an independent cohort. To evaluate the reproducibility of CANARY analysis, three independent users who are not part of the CANARY development team segmented and analyzed 50 biopsy-confirmed primary lung ADCs from Vanderbilt University Medical Center. The CT scans of ADCs were selected retrospectively based on the following criteria: ADC-histology proven on biopsy, diagnosed between 2009-2015, less than 3cm in greatest diameter, and stages IA-IV. Users followed a standard operating procedure established at the Mayo Clinic, and were blinded to clinical characteristics and patient outcomes. Results: To measure inter-user variability of ADC voxel composition, the intraclass correlation coefficient (ICC) was calculated based upon the percentage of each voxel subtype within the whole ADC. An ICC of 1 reflects high reproducibility between users. Amongst all 50 ADCs, the average ICC for all nine voxel types was 0.828 (95% CI 0.76, 0.895). The ICC of the four voxel types associated with invasive features on CT was 0.865 (95% CI 0.805, 0.924). ICCs were also calculated using a logarithmic transformation for data normalization, generating an ICC of 0.745 (95% CI 0.663, 0.826) for all nine voxel types, and an ICC of 0.995 (95% CI 0.993, 0.997) for the four voxel types associated with invasion. Conclusions: (1) CANARY analysis of lung ADC voxel signatures appears to be reproducible amongst users, making it a reliable tool for the evaluation of ADC voxel density. (2) Correlation of invasive signatures associated with more aggressive ADCs was nearly perfect amongst users. Additional validation metrics for CANARY with larger datasets are being evaluated, including the accuracy of tumor prognostic predictions between users and analysis of ADC datasets from other institutions. Citation Format: Erica C. Nakajima, Michael P. Frankland, Tucker Johnson, Sanja L. Antic, Ronald A. Karwoski, Bennett Landman, Heidi Chen, Ronald C. Walker, Brian J. Bartholmai, Tobias Peikert, Srinivasan Rajagopalan, Pierre P. Massion, Fabien Maldonado. Assessing the reproducibility of computer-aided nodule assessment and risk yield (CANARY) method to characterize lung adenocarcinomas [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 3723. doi:10.1158/1538-7445.AM2017-3723
P103
Automated quantitative imaging correlates with lung function and exercise tolerance in patients with idiopathic pulmonary fibrosis.
Purpose: The aim of the study was to determine whether a novel computed tomography (CT) postprocessing software technique (CALIPER) is superior to visual CT scoring as judged by functional correlations in idiopathic pulmonary fibrosis (IPF). Materials and Methods: A total of 283 consecutive patients with IPF had CT parenchymal patterns evaluated quantitatively with CALIPER and by visual scoring. These 2 techniques were evaluated against: forced expiratory volume in 1 second (FEV1), forced vital capacity (FVC), diffusing capacity for carbon monoxide (DLco), carbon monoxide transfer coefficient (Kco), and a composite physiological index (CPI), with regard to extent of interstitial lung disease (ILD), extent of emphysema, and pulmonary vascular abnormalities. Results: CALIPER-derived estimates of ILD extent demonstrated stronger univariate correlations than visual scores for most pulmonary function tests (PFTs): (FEV1: CALIPER R 2=0.29, visual R 2=0.18; FVC: CALIPER R 2=0.41, visual R 2=0.27; DLco: CALIPER R 2=0.31, visual R 2=0.35; CPI: CALIPER R 2=0.48, visual R 2=0.44). Correlations between CT measures of emphysema extent and PFTs were weak and did not differ significantly between CALIPER and visual scoring. Intriguingly, the pulmonary vessel volume provided similar correlations to total ILD extent scored by CALIPER for FVC, DLco, and CPI (FVC: R 2=0.45; DLco: R 2=0.34; CPI: R 2=0.53). Conclusions: CALIPER was superior to visual scoring as validated by functional correlations with PFTs. The pulmonary vessel volume, a novel CALIPER CT parameter with no visual scoring equivalent, has the potential to be a CT feature in the assessment of patients with IPF and requires further exploration.
Noninvasive Computed Tomography–based Risk Stratification of Lung Adenocarcinomas in the National Lung Screening Trial
Abstract Rationale Screening for lung cancer using low-dose computed tomography (CT) reduces lung cancer mortality. However, in addition to a high rate of benign nodules, lung cancer screening detects a large number of indolent cancers that generally belong to the adenocarcinoma spectrum. Individualized management of screen-detected adenocarcinomas would be facilitated by noninvasive risk stratification. Objectives To validate that Computer-Aided Nodule Assessment and Risk Yield (CANARY), a novel image analysis software, successfully risk stratifies screen-detected lung adenocarcinomas based on clinical disease outcomes. Methods We identified retrospective 294 eligible patients diagnosed with lung adenocarcinoma spectrum lesions in the low-dose CT arm of the National Lung Screening Trial. The last low-dose CT scan before the diagnosis of lung adenocarcinoma was analyzed using CANARY blinded to clinical data. Based on their parametric CANARY signatures, all the lung adenocarcinoma nodules were risk stratified into three groups. CANARY risk groups were compared using survival analysis for progression-free survival. Measurements and Main Results A total of 294 patients were included in the analysis. Kaplan-Meier analysis of all the 294 adenocarcinoma nodules stratified into the Good, Intermediate, and Poor CANARY risk groups yielded distinct progression-free survival curves (P < 0.0001). This observation was confirmed in the unadjusted and adjusted (age, sex, race, and smoking status) progression-free survival analysis of all stage I cases. Conclusions CANARY allows the noninvasive risk stratification of lung adenocarcinomas into three groups with distinct post-treatment progression-free survival. Our results suggest that CANARY could ultimately facilitate individualized management of incidentally or screen-detected lung adenocarcinomas.