Liu, Xingyi

ABSTRACT Background Mobility functional status is inconsistently recorded in electronic health records. Recent advances in Large Language Models (LLMs) enable automated extraction of functional information from unstructured clinical notes. Leveraging a validated functional LLM with high performance in mobility extraction, we aimed to study the association between LLM‐derived mobility functional status and clinical outcomes, including emergency department (ED) visits, hospitalizations, and mortality. Methods Individuals aged 60 years and older enrolled in the Mayo Clinic Study of Aging, with documented clinical notes in 2022 were included. Individuals without any identifiable functional information were excluded. The LLM‐derived mobility functional status categorized mobility as impaired or unimpaired, across the five mobility domains as classified by International Classification of Functioning, Disability, and Health. Clinical outcomes in 2023 were extracted from the Rochester Epidemiology Project database. Logistic regression models assessed associations between mobility impairment and outcomes, adjusted for age, sex, Elixhauser comorbidity index, and education. Results Three thousand three hundred thirty‐seven individuals were screened, of whom three hundred forty‐three had no functional information. Two thousand nine hundred ninety‐four individuals were analyzed. 79.4% had at least one mobility impairment. Higher mobility impairment is associated with older age and greater Elixhauser comorbidity. Individuals with 3 or more impairments had an increased adjusted odds of ED visits (Adjusted Odds Ratio (aOR): 2.18 (1.64, 2.90)) and hospitalizations (aOR: 2.35 (1.51, 3.65)) compared to unimpaired individuals ( p < 0.001), while mortality did not demonstrate a significant association with mobility impairment. Conclusion LLM‐derived mobility functional status assessment was associated with ED visits and hospitalization, but not with all‐cause mortality after adjustment. Prospective validation and evaluation of incremental discrimination, calibration, and clinical utility are needed before clinical implementation.

Abstract Objective This study evaluated sequential multi-site fine-tuning of large language models (LLMs), simulating incremental deployment across institutions for extracting mobility functional status from unstructured clinical notes. Materials and Methods We assembled a corpus of 600 clinical notes from 3 institutions, yielding 3810 annotated note sections (1200 training; 2610 testing). Building upon our previously established LLM mobility extraction pipeline, which utilized optimized prompt engineering, we applied parameter-efficient low-rank adaptation (LoRA) fine-tuning. We evaluated sequential multi-site fine-tuning, where the model is trained locally and only the updated LoRA weights are transferred to the next site. We compared this against joint fine-tuning, which requires pooling all multi-site patient data centrally before training. Performance was measured by micro F1-scores for 2 distinct subtasks: mobility extraction and impairment classification. Results Sequential multi-site fine-tuning yielded comparable results to centralized joint fine-tuning for mobility extraction (F1, 0.861 vs 0.894) and impairment classification (F1, 0.912 vs 0.900). Both sequential and joint fine-tuning outperformed an untuned 70B model. Additionally, single-site LoRA models showed robust cross-site generalization. Discussion Sequential multi-site fine-tuning offers a resource-friendly approach to adapt LLMs to clinical environments. By retaining previously acquired knowledge without severe interference, this strategy allows for local fine-tuning at each institution without the need to pool or share sensitive patient data required for joint fine-tuning. Conclusion Sequential multi-site fine-tuning provides a viable solution for the scalable, incremental deployment of LLMs across institutions, showing promising generalizability for clinical information extraction from electronic health record narratives.

PURPOSE Large language models (LLMs) have demonstrated remarkable versatility in oncology applications, such as cancer staging and survival analysis. Despite their potential, ethical concerns such as data privacy breaches, bias in training data, lack of transparency, and risks associated with erroneous outputs pose significant challenges to their adoption in high-stakes oncology settings. Therefore, we aim to explore the ethical challenges associated with LLM-based applications in oncology and evaluate emerging techniques designed to address these issues. METHODS Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework, a systematic review was conducted to evaluate publications related to ethical issues of LLMs in oncology across eight academic databases (eg, PubMed, Web of Science, and Embase) between January 1, 2019, and December 31, 2024. RESULTS The search retrieved 4,319 published articles, of which 65 publications were preserved and included in our analysis. We identified six prevalent ethical challenges in oncology, including trust, equity, privacy, transparency, nonmaleficence, and accountability. We then evaluated emerging technical solutions to mitigate ethical challenges and summarized evaluation metrics used to assess these solutions' effectiveness. CONCLUSION This review provides actionable recommendations for responsibly deploying LLMs in oncology, ensuring adherence to ethical guidelines, and fostering improved patient outcomes. By bridging technical and clinical perspectives, this review offers a foundational framework for advancing ethical artificial intelligence applications in oncology and highlights areas for future research.

Abstract With global aging, assessing functional status is vital for precision medicine. Electronic Health Records (EHRs), particularly unstructured data, hold abundant information on patient mobility. This study explores using Large Language Models (LLMs) to extract and standardize mobility status from unstructured EHR data (i.e., clinical notes). We annotated 600 clinical notes from three health care institutions located in southeastern Minnesota and west-central Wisconsin, focusing on expressions of mobility and associated impairment. Leveraging the open-source Llama 3 model, we tested various prompting strategies—including zero-shot, few-shot, and task decomposition—and evaluated their performance. Error analysis showed that while the model sometimes inferred impairments without explicit evidence, most errors were clinically reasonable, often reflecting borderline or ambiguous cases. While considering reasonable inference as correct, at the patient-level, Mobility Extraction achieves a micro-average accuracy of 0.952 with an F1-score of 0.962, and Impairment Classification produces a micro-average accuracy of 0.912 and an F1-score of 0.948. A local, deterministic setup improved trustworthiness by ensuring consistent outputs, safeguarding privacy, and demonstrating cross-institution generalizability. These findings highlight the feasibility of LLM-based solutions for extracting mobility functional status from unstructured EHR data, supporting both clinical applications and research.