Abstract Rationale Usual interstitial pneumonia (UIP) is the defining morphology of idiopathic pulmonary fibrosis (IPF). Guidelines for IPF diagnosis conditionally recommend surgical lung biopsy for histopathology diagnosis of UIP when radiology and clinical context are not definitive. A “molecular diagnosis of UIP” in transbronchial lung biopsy, the Envisia Genomic Classifier, accurately predicted histopathologic UIP. Objectives We evaluated the combined accuracy of the Envisia Genomic Classifier and local radiology in the detection of UIP pattern. Methods Ninety-six patients who had diagnostic lung pathology as well as a transbronchial lung biopsy for molecular testing with Envisia Genomic Classifier were included in this analysis. The classifier results were scored against reference pathology. UIP identified on high-resolution computed tomography (HRCT) as documented by features in local radiologists’ reports was compared with histopathology. Measurements and Main Results In 96 patients, the Envisia Classifier achieved a specificity of 92.1% (confidence interval [CI],78.6–98.3%) and a sensitivity of 60.3% (CI, 46.6–73.0%) for histology-proven UIP pattern. Local radiologists identified UIP in 18 of 53 patients with UIP histopathology, with a sensitivity of 34.0% (CI, 21.5–48.3%) and a specificity of 96.9% (CI, 83.8–100%). In conjunction with HRCT patterns of UIP, the Envisia Classifier results identified 24 additional patients with UIP (sensitivity 79.2%; specificity 90.6%). Conclusions In 96 patients with suspected interstitial lung disease, the Envisia Genomic Classifier identified UIP regardless of HRCT pattern. These results suggest that recognition of a UIP pattern by the Envisia Genomic Classifier combined with HRCT and clinical factors in a multidisciplinary discussion may assist clinicians in making an interstitial lung disease (especially IPF) diagnosis without the need for a surgical lung biopsy.
Reply: Improving Care for Patients with Interstitial Lung Disease, Using Machine Learning, Requires Transparency and Reproducibility
Abstract Rationale Usual interstitial pneumonia (UIP) is the histopathologic hallmark of idiopathic pulmonary fibrosis. Although UIP can be detected by high-resolution computed tomography of the chest, the results are frequently inconclusive, and pathology from transbronchial biopsy (TBB) has poor sensitivity. Surgical lung biopsy may be necessary for a definitive diagnosis. Objectives To develop a genomic classifier in tissue obtained by TBB that distinguishes UIP from non-UIP, trained against central pathology as the reference standard. Methods Exome enriched RNA sequencing was performed on 283 TBBs from 84 subjects. Machine learning was used to train an algorithm with high rule-in (specificity) performance using specimens from 53 subjects. Performance was evaluated by cross-validation and on an independent test set of specimens from 31 subjects. We explored the feasibility of a single molecular test per subject by combining multiple TBBs from upper and lower lobes. To address whether classifier accuracy depends upon adequate alveolar sampling, we tested for correlation between classifier accuracy and expression of alveolar-specific genes. Results The top-performing algorithm distinguishes UIP from non-UIP conditions in single TBB samples with an area under the receiver operator characteristic curve (AUC) of 0.86, with specificity of 86% (confidence interval = 71–95%) and sensitivity of 63% (confidence interval = 51–74%) (31 test subjects). Performance improves to an AUC of 0.92 when three to five TBB samples per subject are combined at the RNA level for testing. Although we observed a wide range of type I and II alveolar–specific gene expression in TBBs, expression of these transcripts did not correlate with classifier accuracy. Conclusions We demonstrate proof of principle that genomic analysis and machine learning improves the utility of TBB for the diagnosis of UIP, with greater sensitivity and specificity than pathology in TBB alone. Combining multiple individual subject samples results in increased test accuracy over single sample testing. This approach requires validation in an independent cohort of subjects before application in the clinic.
The Impact of Benign Gene Expression Classifier Test Results on the Endocrinologist–Patient Decision to Operate on Patients with Thyroid Nodules with Indeterminate Fine-Needle Aspiration Cytopathology
Background: Seventy-five percent of thyroid nodules with indeterminate fine-needle aspiration (FNA) cytology are found to be benign postoperatively. A novel genomic test, the Afirma ® gene expression classifier (AGEC), has been available for clinical use in the United States, since late 2010. In 2010, two modest-sized validation studies showed that the AGEC could identify a benign gene expression signature in indeterminate cytology thyroid FNA samples with a negative predictive value >95%. The objective of this study was to evaluate how the AGEC impacted the joint decision of the endocrinologist and patient to operate when FNA cytology was indeterminate, but the AGEC reading of the nodule was benign. Methods: In this cross-sectional cohort study, data were contributed retrospectively by 51 endocrinologists at 21 practice sites that had previously obtained ≥3 benign AGEC readings in ≥1 cm nodules with indeterminate FNA cytology readings. Information regarding demographic data, nodule size and location, decision to operate, surgery type (hemithyroidectomy [HT] or total thyroidectomy [TT]), and reason for recommending surgery was retrospectively collected. Results: Compared to a 74% previous historical rate of surgery for cytologically indeterminate nodules, the operative rate fell to 7.6% during the period that AGEC were obtained in the clinical practices, a highly significant reduction in the decision to operate ( p <0.001). The rate of surgery on cytologically indeterminate nodules that were benign by the AGEC reading did not differ from the historically reported rate of operation on cytologically benign nodules ( p =0.41). The four primary reasons reported by the physicians for operating on nodules with a benign AGEC reading, in descending order: large nodule size (46.4%), symptomatic nodules (25.0%), rapidly growing nodules (10.7%), or a second suspicious or malignant nodule in the same patient (10.7%). These reasons are concordant with those typically given for operation on cytologically benign nodules. Conclusions: In a substantial group of medical practices, obtaining an AGEC test in patients with cytologically indeterminate nodules was associated with a striking reduction in the rate of diagnostic thyroidectomy. Approximately, one surgery was avoided for every two AGEC tests run on thyroid FNAs with indeterminate cytology.