Citations

PURPOSE Automated curation of breast cancer treatment data with minimal human involvement could accelerate the collection of statewide and nationwide evidence for patient management and assessing the effectiveness of treatment pathways. The primary challenges are the complexity and inconsistency of structured clinical data streams and accurate extraction of this information from free-text clinical narratives. MATERIALS AND METHODS We proposed a hybrid two-phase information extraction framework that combined a Unified Medical Language System parser (phase-1) with a fine-tuned large language model (LLM; phase-2) to extract longitudinal treatment timelines from time-stamped clinical notes. Our framework was developed through end-to-end joint learning as a question-answering model, where the model was trained to simultaneously answer five questions, each corresponding to a specific treatment. RESULTS We fine-tuned and internally validated the model on 26,692 patients with breast cancer (diagnosed between 2013 and 2020) receiving treatment at Mayo Clinic and externally validated the model on 162 randomly selected patients from Stanford Healthcare. Zero-shot LLM (out-of-the-box) had high specificity but low sensitivity, indicating that although these frameworks are useful for generic language understanding, they are lacking in terms of targeted clinical tasks. The proposed model achieved 0.942 average AUROC on the internal and 0.924 on the external data, demonstrating only marginal drop in performance when evaluated on external. The proposed model also achieved better trade-off between sensitivity (average: 79.2%) and specificity (average: 76.2%) compared with rule-based (average sensitivity: 70.5%, average specificity: 68.1%) and structured codes (average sensitivity: 64.1%, average specificity: 83.5%). CONCLUSION The proposed framework can extract temporal information about cancer treatments from various time-stamped clinic notes, regardless of the setting of treatment administration (inpatient or outpatient) or time frame. To support the cancer research community for such data curation and longitudinal analysis, we have packaged the code as a docker image, which needs minimal system reconfiguration and shared with an open-source academic license.

Abstract Critical findings in radiology reports are life threatening conditions that need to be communicated promptly to physicians (“critical findings”) for timely man-agement of patients. Flagging radiology reports of such incidents could facilitate opportune communication of critical findings. With advancements in natural language processing (NLP), large language models (LLMs) can be trained with task-specific instructions and examples to mine information from narrative texts. We believe that similar methods can be applied to radiology reports to identify and extract critical findings from these reports. However, due to the rarity of such critical events, there is a dearth of manually labeled datasets of critical findings in radiology reports. To overcome this limitation, we train instruction-tuned MISTRAL-based language models in a two-phase weakly supervised fine-tuning setup on unlabeled radiology reports from Mayo Clinic (n=15000). The weakly fine-tuned model is then used to automatically extract critical terms from both internal and external test datasets - Mayo Clinic (n=80) and MIMIC-III (n=123) 1 respectively against the expert-annotation. We also evaluated performance of the models on a large-scale MIMIC-IV reports (n=5000) using automated LLM-aided evaluation metrics - G-eval and Prometheus. We observe that, for both manual and LLM-based evaluations, weakly supervised fine-tuning improves model performance, showing successful task-specific alignment. For community use, we released the trained model with open-source academic license∗

PURPOSE Patient-centered outcomes (PCOs) are pivotal in cancer treatment, as they directly reflect patients' quality of life. Although multiple studies suggest that factors affecting breast cancer–related morbidity and survival are influenced by treatment side effects and adherence to long-term treatment, such data are generally only available on a smaller scale or from a single center. The primary challenge with collecting these data is that the outcomes are captured as free text in clinical narratives written by clinicians. MATERIALS AND METHODS Given the complexity of PCO documentation in these narratives, computerized methods are necessary to unlock the wealth of information buried in unstructured text notes that often document PCOs. Inspired by the success of large language models (LLMs), we examined the adaptability of three LLMs, GPT-2, BioGPT, and PMC-LLaMA, on PCO tasks across three institutions, Mayo Clinic, Emory University Hospital, and Stanford University. We developed an open-source framework for fine-tuning LLM that can directly extract the five different categories of PCO from the clinic notes. RESULTS We found that these LLMs without fine-tuning (zero-shot) struggle with challenging PCO extraction tasks, displaying almost random performance, even with some task-specific examples (few-shot learning). The performance of our fine-tuned, task-specific models is notably superior compared with their non–fine-tuned LLM models. Moreover, the fine-tuned GPT-2 model has demonstrated a significantly better performance than the other two larger LLMs. CONCLUSION Our discovery indicates that although LLMs serve as effective general-purpose models for tasks across various domains, they require fine-tuning when applied to the clinician domain. Our proposed approach has the potential to lead more efficient, adaptable models for PCO information extraction, reducing reliance on extensive computational resources while still delivering superior performance for specific tasks.

4020 Background: High grade (G3) GEPNENs are a rare and heterogeneous disease entity for which there is little prospective treatment data. EP chemotherapy is the treatment standard but this may not be appropriate for all G3 GEPNEN pts. CAPTEM has demonstrated activity in G3 GEPNENs and may be a promising alternative. EA2142 aimed to determine if CAPTEM was superior to EP in pts with G3 GEPNENs. Methods: This was a multicenter, randomized (1:1) phase II trial for pts with a locally advanced and unresectable or metastatic well differentiated G3 neuroendocrine tumor (NET) or a poorly differentiated, non-small cell G3 neuroendocrine carcinoma (NEC) of suspected gastrointestinal origin and an ECOG PS of 0-2. Pathology must have demonstrated a Ki-67 of 20-100% or at least 10 mitoses/10 high powered field. Pts were randomized to receive capecitabine 750 mg/m2 orally every 12 hours on days 1-14 and temozolomide 200 mg/m2 orally once daily on days 10-14 of a 28-day treatment cycle (Arm A) or etoposide 100 mg/m2 daily on days 1-3 with either cisplatin 25 mg/m2 daily on days 1-3 or carboplatin AUC 5 on day 1 of a 21-day treatment cycle (Arm B). Restaging scans were performed every 8 weeks and toxicity monitored per CTCAEv4. Final statistical plan was to accrue 80 pts to detect a 67% improvement in progression free survival (PFS) (primary endpoint) with CAPTEM as compared to EP, 80% power and one-sided significance level of 0.10. A planned interim analysis for efficacy and futility was conducted. Results: A total of 67 pts were enrolled (Arm A, n=32; Arm B, n=35). Male 58%, African American 4%, Asian 3%. Mean age 61. Among 63 eligible pts, primary tumor site pancreatic 56%, non-pancreatic 43%. Poorly differentiated 57%, well differentiated 33%, unknown 10%. Mean Ki-67 48% (Arm A), 60% (Arm B). The study was closed prior to full accrual due to futility at 57.7% information time. In the interim analysis, among 62 eligible pts, PFS, overall survival and response rate with CAPTEM were 2.43 months (mos) (95% CI 2.04, 7.72), 12.6 mos, 9% respectively vs 5.36 mos (95% CI 2.14, 7.23), 13.6 mos and 10% with EP. Toxicity was evaluable in 57 pts with Grade (G) 3/4 events occurring in 29% of pts on Arm A, 66% of pts on Arm B. G3/4 events occurring in more than 5% of pts on Arm A—febrile neutropenia (n=2); abdominal pain (n=2); diarrhea (n=2); nausea (n=2); neutropenia (n=2); dehydration (n=2) and on Arm B—anemia (n=8); febrile neutropenia (n=2); fatigue (n=2); lymphopenia (n=2); neutropenia (n=12); thrombocytopenia (n=4); leukopenia (n=6). There was one G5 event on Arm A due to sepsis. Conclusions: CAPTEM does not appear to be superior to EP chemotherapy as front-line treatment for pts with G3 NENs but does demonstrate a more favorable toxicity profile. Studies assessing G3 NET independently of G3 NEC are needed. Clinical trial information: NCT02595424.

In June 2016, Diabetes Technology Society convened a panel of US experts in inpatient diabetes management to discuss the current and potential role of continuous glucose monitoring (CGM) in the hospital. This discussion combined with a literature review was a follow-up to a meeting, which took place in May 2015. The panel reviewed evidence on use of CGM in 3 potential inpatient scenarios: (1) the intensive care unit (ICU), (2) non-ICU, and (3) transitioning outpatient CGM use into the hospital setting. Panel members agreed that data from limited studies and theoretical considerations suggested that use of CGM in the hospital had the potential to improve patient clinical outcomes, and in particular reduction of hypoglycemia. Panel members discussed barriers to widespread adoption of CGM, which patients would benefit most from use of this technology, and what type of outcome studies are needed to guide use of CGM in the inpatient setting.

Abstract As anticancer therapies designed to target specific molecular pathways have been developed, it has become critical to develop methods to assess the response induced by such agents. Although traditional, anatomic CT, and MRI examinations are useful in many settings, increasing evidence suggests that these methods cannot answer the fundamental biologic and physiologic questions essential for assessment and, eventually, prediction of treatment response in the clinical trial setting, especially in the critical period soon after treatment is initiated. To optimally apply advances in quantitative imaging methods to trials of targeted cancer therapy, new infrastructure improvements are needed that incorporate these emerging techniques into the settings where they are most likely to have impact. In this review, we first elucidate the needs for therapeutic response assessment in the era of molecularly targeted therapy and describe how quantitative imaging can most effectively provide scientifically and clinically relevant data. We then describe the tools and methods required to apply quantitative imaging and provide concrete examples of work making these advances practically available for routine application in clinical trials. We conclude by proposing strategies to surmount barriers to wider incorporation of these quantitative imaging methods into clinical trials and, eventually, clinical practice. Our goal is to encourage and guide the oncology community to deploy standardized quantitative imaging techniques in clinical trials to further personalize care for cancer patients and to provide a more efficient path for the development of improved targeted therapies. Clin Cancer Res; 22(2); 284–90. ©2016 AACR.