Reeve, Bryce B.

PURPOSE There is increasing interest in implementing digital systems for remote monitoring of patients’ symptoms during routine oncology practice. Information is limited about the clinical utility and user perceptions of these systems. METHODS PRO-TECT is a multicenter trial evaluating implementation of electronic patient-reported outcomes (ePROs) among adults with advanced and metastatic cancers receiving treatment at US community oncology practices (ClinicalTrials.gov identifier: NCT03249090 ). Questions derived from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) are administered weekly by web or automated telephone system, with alerts to nurses for severe or worsening symptoms. To elicit user feedback, surveys were administered to participating patients and clinicians. RESULTS Among 496 patients across 26 practices, the majority found the system and questions easy to understand (95%), easy to use (93%), and relevant to their care (91%). Most patients reported that PRO information was used by their clinicians for care (70%), improved discussions with clinicians (73%), made them feel more in control of their own care (77%), and would recommend the system to other patients (89%). Scores for most patient feedback questions were significantly positively correlated with weekly PRO completion rates in both univariate and multivariable analyses. Among 57 nurses, most reported that PRO information was helpful for clinical documentation (79%), increased efficiency of patient discussions (84%), and was useful for patient care (75%). Among 39 oncologists, most found PRO information useful (91%), with 65% using PROs to guide patient discussions sometimes or often and 65% using PROs to make treatment decisions sometimes or often. CONCLUSION These findings support the clinical utility and value of implementing digital systems for monitoring PROs, including the PRO-CTCAE, in routine cancer care.

7018 Background: Standard reporting of symptomatic adverse events (AEs) in oncology relies on clinicians to rate patient (pt) experience using CTCAE; each symptom is represented by a single graded item. To capture direct pt experience, NCI developed PRO-CTCAE to supplement CTCAE. In PRO-CTCAE, the pt answers up to 3 questions per AE about a symptom’s frequency, severity and interference with daily activities. To align PRO-CTCAE with CTCAE, we developed an algorithm for mapping sets of questions for an AE to a single composite numerical grade. Methods: We used a 5-step process. (1) All 187 possible PRO-CTCAE score permutations were presented to clinical investigators to subjectively map permutations to single numerical grades (range 0-3). (2) Permutations with < 75% agreement were presented to investigator committees at a National Clinical Trials Network meeting to gain majority consensus via anonymous voting. (3) The resulting algorithm was refined via graphical and tabular approaches to assure directional consistency. (4) Validity, reliability and sensitivity were assessed in a national study dataset. (5) Accuracy for delineating AEs between study arms was measured in 2 phase III clinical trials (Alliance for Clinical Trials in Oncology A091105 and Exelixis COMET-2). Results: (1) 12/187 score permutations had < 75% initial agreement. (2) Majority consensus was reached for all permutations. (3) 5 mappings were adjusted to assure directional consistency. (4) Composite grades for 46/59 (78%) AEs were higher in pts with ECOG performance status 2-4 vs 0-1 (median effect size 0.23 [range -0.49-0.73]; 32/59 effect size ≥0.2; 25/59 p< 0.05), similar to when conducting analysis on individual unmapped items. The test-retest reliability for 24 selected composite grades ranged from 0.57-0.96 (median intraclass correlation coefficient [ICC] 0.77) with 18/24 (75%) grades having ICC ≥0.7. Median (range) standardized response means in pts reporting worsening, no change, and improvement were 0.20 (0.03-0.34), -0.06 (-0.20-0.03) and -0.12 (-0.32-0.06). (5) Pattern, directionality and statistical significance of between-arm differences in both trials were preserved with composite grades as compared to individual unmapped items. Conclusions: A composite grading algorithm for PRO-CTCAE was rigorously developed and validated. PRO-CTCAE composite grades may be useful in analyses to provide a single metric for each pt-reported AE for trial and real-world reporting. Support: UG1CA189823; U01CA233046; HHSN261200800043C; Bayer (A091105); https://acknowledgments.alliancefound.org .

9047 Background: Symptomatic adverse events (AE) in cancer trials are reported by clinicians using the National Cancer Institute's (NCI) Common Terminology Criteria for Adverse Events (CTCAE). To integrate the patient perspective into AE reporting, NCI contracted (HHSN261201000043C) to create a patient-reported outcomes companion tool (PRO-CTCAE). We report the validity and reliability of PRO-CTCAE’s 124 items reflecting 78 symptomatic AEs. Methods: English-speaking subjects (n=869; 44% male; median [mdn] age 59; 32% racial/ethnic minority; 34% high school or less; 17% ECOG Performance Status [PS] 2-4) receiving treatment for a range of cancers at 4 NCI-designated cancer centers and 5 sites in NCI's Community Cancer Centers Program completed a web-based survey in clinic including PRO-CTCAE and EORTC QLQ-C30. Pearson correlations were computed between PRO-CTCAE items and QLQ-C30 scales. Differences in PRO-CTCAE item scores between clinician-reported ECOG PS (0-1 vs 2-4) groups were computed. Test-retest reliability of 48 prespecified items was evaluated in a subset (n=79). Results: Correlations in the expected direction were observed for 116/124 PRO-CTCAE items with the QLQ-C30 global health scale (mdn r=-.21; range .08 to -.57). Stronger correlations were seen for PRO-CTCAE items with conceptually related QLQ-C30 domains: fatigue with physical, role and social functioning (-.54 to -.66); anxiety and depression with emotional functioning (-.54 to -.70); and concentration and memory problems with cognitive functioning (-.62 to -.72) [all p <0.001]. Fatigue, nausea, vomiting, pain, dyspnea, insomnia, appetite loss, constipation and diarrhea items strongly correlated with corresponding QLQ-C30 symptom scales (.69 to .79, all p <0.001). Scores for 98/124 PRO-CTCAE items were higher in the ECOG PS 2-4 vs 0-1 group (mdn effect size .3). Test-retest reliability was observed across all tested items (mdn intraclass correlation coeff .77; range .53 to .96). Conclusions: Results demonstrate favorable validity and reliability of PRO-CTCAE in a large, heterogeneous sample of patients undergoing cancer treatment. Further testing in multicenter treatment trials is underway.

Purpose The US National Cancer Institute (NCI) Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed to enable patient reporting of symptomatic adverse events in oncology clinical research. This study was designed to assess the feasibility and resource requirements associated with implementing PRO-CTCAE in a multicenter trial. Methods Patients with locally advanced rectal cancer enrolled in the National Cancer Institute–sponsored North Central Cancer Treatment Group (Alliance) Preoperative Radiation or Selective Preoperative Radiation and Evaluation before Chemotherapy and Total Mesorectal Excision trial were asked to self-report 30 PRO-CTCAE items weekly from home during preoperative therapy, and every 6 months after surgery, via either the Web or an automated telephone system. If participants did not self-report within 3 days, a central coordinator called them to complete the items. Compliance was defined as the proportion of participants who completed PRO-CTCAE assessments at expected time points. Results The prespecified PRO-CTCAE analysis was conducted after the 500th patient completed the 6-month follow-up (median age, 56 years; 33% female; 12% nonwhite; 43% high school education or less; 5% Spanish speaking), across 165 sites. PRO-CTCAE was reported by participants at 4,491 of 4,882 expected preoperative time points (92.0% compliance), of which 3,771 (77.2%) were self-reported by participants and 720 (14.7%) were collected via central coordinator backup. Compliance at 6-month post-treatment follow-up was 333 of 468 (71.2%), with 122 (26.1%) via backup. Site research associates spent a median of 15 minutes on PRO-CTCAE work for each patient visit. Work by a central coordinator required a 50% time commitment. Conclusion Home-based reporting of PRO-CTCAE in a multicenter trial is feasible, with high patient compliance and low site administrative requirements. PRO-CTCAE data capture is improved through centralized backup calls.

Aims: The US National Cancer Institute recently developed the PRO-CTCAE (Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events). PRO-CTCAE is a library of questions for clinical trial participants to self-report symptomatic adverse events (e.g. nausea). The objective of this study is to inform evidence-based selection of a recall period when PRO-CTCAE is included in a trial. We evaluated differences between 1-, 2-, 3-, and 4-week recall periods, using daily reporting as the reference. Methods: English-speaking patients with cancer receiving chemotherapy and/or radiotherapy were enrolled at four US cancer centers and affiliated community clinics. Participants completed 27 PRO-CTCAE items electronically daily for 28 days, and then weekly over 4 weeks, using 1-, 2-, 3-, and 4-week recall periods. For each recall period, mean differences, effect sizes, and intraclass correlation coefficients were calculated to evaluate agreement between the maximum of daily ratings and the corresponding ratings obtained using longer recall periods (e.g. maximum of daily scores over 7 days vs 1-week recall). Analyses were repeated using the average of daily scores within each recall period rather than the maximum of daily scores. Results: A total of 127 subjects completed questionnaires (57% male; median age: 57). The median of the 27 mean differences in scores on the PRO-CTCAE 5-point response scale comparing the maximum daily versus the longer recall period (and corresponding effect size) was −0.20 (−0.20) for 1-week recall, −0.36 (−0.31) for 2-week recall, −0.45 (−0.39) for 3-week recall, and −0.47 (−0.40) for 4-week recall. The median intraclass correlation across 27 items between the maximum of daily ratings and the corresponding longer recall ratings for 1-week recall was 0.70 (range: 0.54–0.82), for 2-week recall was 0.74 (range: 0.58–0.83), for 3-week recall was 0.72 (range: 0.61–0.84), and for 4-week recall was 0.72 (range: 0.64–0.86). Similar results were observed for all analyses using the average of daily scores rather than the maximum of daily scores. Conclusion: A 1-week recall corresponds best to daily reporting. Although intraclass correlations remain stable over time, there are small but progressively larger differences between daily and longer recall periods at 2, 3, and 4 weeks, respectively. The preferred recall period for the PRO-CTCAE is the past 7 days, although investigators may opt for recall periods of 2, 3, or 4 weeks with an understanding that there may be some information loss.

Examining the patient's subjective experience in prospective clinical comparative effectiveness research (CER) of oncology treatments or process interventions is essential for informing decision making. Patient-reported outcome (PRO) measures are the standard tools for directly eliciting the patient experience. There are currently no widely accepted standards for developing or implementing PRO measures in CER. Recommendations for the design and implementation of PRO measures in CER were developed via a standardized process including multistakeholder interviews, a technical working group, and public comments. Key recommendations are to include assessment of patient-reported symptoms as well as health-related quality of life in all prospective clinical CER studies in adult oncology; to identify symptoms relevant to a particular study population and context based on literature review and/or qualitative and quantitative methods; to assure that PRO measures used are valid, reliable, and sensitive in a comparable population (measures particularly recommended include EORTC QLQ-C30, FACT, MDASI, PRO-CTCAE, and PROMIS); to collect PRO data electronically whenever possible; to employ methods that minimize missing patient reports and include a plan for analyzing and reporting missing PRO data; to report the proportion of responders and cumulative distribution of responses in addition to mean changes in scores; and to publish results of PRO analyses simultaneously with other clinical outcomes. Twelve core symptoms are recommended for consideration in studies in advanced or metastatic cancers. Adherence to methodologic standards for the selection, implementation, and analysis/reporting of PRO measures will lead to an understanding of the patient experience that informs better decisions by patients, providers, regulators, and payers.