Citations

Clear

93 citation(s) found for "Liu, Wei".

Generative AI, foundation models and large language models in radiation therapy physics: Clinical applications, challenges, and future directions
Qi, X. Sharon, Wang, Yi, Yang, Xiaofeng, Ren, Lei , Liu, Wei, Benedict, Stanley , Xiao, Ying , Xing, Lei , Naqa, Issam M. El, Qi, X. Sharon , Qi, X. Sharon, Yang, Xiaofeng

Mayo Clinic

Medical Physics | August 7, 2026 | Volume: 53 | Issue: 8 | Pages: No page available

10.1002/mp.70616 | PMID 42563397 Get It

Abstract Generative AI (Gen AI), Foundation Models (FMs), and Large Language Models (LLMs) are powerful emerging technologies that demonstrate exceptional capabilities in processing vast amounts of unstructured and structured data, including text, voice, images, video and other formats, and adapting to a wide range of specific tasks. Their immense potential to drive meaningful improvements in treatment outcomes is increasingly evident. The advent of these technologies has marked a transformative era in healthcare, including the fields of radiation oncology and medical physics. Specifically, these powerful technologies offer unprecedented opportunities to analyze domain‐specific data, process and synthesize medical images, automate routine tasks, support clinical decision‐making, optimize and streamline clinical workflows, and enhance the quality of clinical trials. While these emerging technologies present new opportunities to revolutionize radiation therapy practice, their implementation also raises important educational, ethical, and regulatory considerations. This scoping review highlights benefits, promises, risks, and challenges, such as interpretability, data privacy, regulatory compliance, reproducibility, hallucination, and integration into existing clinical workflow. Finally, emerging opportunities are outlined to guide future research directions. This review paper provides a timely overview of Gen AI, FMs and LLMs, aiming to inform medical physicists, clinicians, and researchers of the evolving role of these disruptive technologies in shaping the future of radiation therapy.
Imaging-Centered Artificial Intelligence for Precision Cancer Immunotherapy: Multimodal Biomarkers, Treatment-Effect Estimation, and Clinical Translation
Zhong, Shengyang , Li, Siyuan, Yang, Wanrong , Zheng, Zheng, Liu, Zhengliang, Xie, Jin, Li, Jin, By, Yi, Li, X C., Liu, Wei, Zhang, Lian, Liu, Tianming, Qiang, Ning , Yi, Bin, Zhao, Lin

Mayo Clinic

No container title available | July 27, 2026 | Volume: No volume available | Issue: No issue available | Pages: No page available

10.21203/rs.3.rs-10468208/v1 Get It

Abstract Immune checkpoint inhibitors (ICIs) can produce durable tumor control, yet many patients experience primary or acquired resistance, mixed lesion-level responses, or immune-related adverse events. Tissue and blood biomarkers provide biologically informative but spatially incomplete views of a heterogeneous and time-varying tumor–host system. Radiologic imaging complements these assays by repeatedly measuring whole-body tumor burden, organ-specific disease patterns, lesion-level heterogeneity, and treatment-related change. This critical narrative review evaluates imaging-centered artificial intelligence (AI) for precision cancer immunotherapy, including baseline and serial CT, MRI, and PET; radiomics and learned image representations; lesion-level and whole-body heterogeneity; toxicity assessment; and integration with pathology, genomics, circulating tumor DNA, laboratory measurements, and electronic health records. Representative cohorts range from early radiogenomic studies of approximately 100–200 patients to externally validated and longitudinal models involving hundreds or thousands of patients. Rather than treating all reported performance estimates as equally actionable, we appraise the clinical question, cohort design, endpoint, validation strategy, calibration, and intended clinical action. Current evidence supports noninvasive immune phenotyping, risk stratification, longitudinal monitoring, and multimodal biomarker integration. However, most models predict outcomes among patients already receiving ICIs and therefore remain prognostic under treatment rather than predictive of differential treatment benefit. Radiotherapy (RT) combined with ICI serves as a focused imaging use case because lesion identity, dose distribution, fractionation, sequence, and thoracic and normal-tissue exposure must be linked to defined actions. Existing patient-level AI studies estimate efficacy or toxicity after treatment selection; none has prospectively shown that an AI-selected RT strategy improves patient outcomes. Translation will require patient-level leakage control, calibration, geographic and temporal validation, robustness to missing modalities, and prospective evaluation of prespecified model-guided decisions.
Research through Evaluation for Large Language Model in Patient-Clinician Communications
Holmes, Jason, Hobson, Jared, Bennett, Alexandra, McKone, Elizabeth L. , Ebner, Daniel, Shiraishi, Satomi, Patel, Samir H. , Yu, Nathan Y. , Hallemeier, Chris L., Ball, Brooke E., Waddle, Mark, Liu, Wei, Hao, Yuexing , Holmes, Jason , Routman, David M. , Kalantari, Saleh , Ghassemi, Marzyeh

Mayo Clinic

No container title available | June 18, 2026 | Volume: No volume available | Issue: No issue available | Pages: No page available

10.21203/rs.3.rs-9735115/v1 | PMID 42370275 Get It

Abstract Large Language Models (LLMs) demonstrate strong capabilities in healthcare applications. However, recruiting human evaluators to assess usability and effectiveness becomes challenging in the large-scale LLM-generated outcomes. Existing metrics are often outdated for evaluating LLM-based healthcare technologies. This paper proposes a Research through Evaluation (RtE) approach that can refine metrics and improve assessments of LLM-generated outputs. We deploy RtE in a retrospective comparative study of prostate cancer patient inquiries, comparing responses from human clinical care teams with those generated by a GPT-4-based in-basket bot. Through three rounds of co-evaluations with clinical professionals and an LLM-as-graders study with four LLMs, the RtE method shows that starting with small, iterative evaluations helps human graders adjust metrics to suit their daily practices better. Additionally, in the LLM-as-graders study, LLMs exhibit similar behavior, preferring the in-basket bot's responses over the clinical care team. Our methodology and findings demonstrate that the RtE method can enhance the transparency and robustness of evaluation metrics for LLM-generated content, with the potential to generalize to evaluations in other clinical practices.
Noisy probing dose facilitated dose prediction for pencil beam scanning proton therapy: Physics enhances generalizability
Zhang, Lian , Zhang, Xiao, Li, Mingzhu, Vargas, Carlos , Keole, Sameer R., Stützer, Kristin , Liu, Tianming , Wong, William W. , Vora, Sujay A., Feng, Hongying , Holmes, Jason , Zhang, Xiao, Liu, Zhengliang , Li, Mingzhu, Sio, Terence T. , Li, Sheng , Shen, Jiajian , Liu, Wei

Mayo Clinic

Medical Physics | May 27, 2026 | Volume: 53 | Issue: 5 | Pages: No page available

10.1002/mp.70509 | PMID 42192246 Get It

Abstract Background Accurate and efficient dose calculation is essential for online adaptive planning in proton therapy. Deep learning (DL) has shown promising dose prediction results for pencil beam scanning proton therapy (PBSPT) in recent years, but existing DL‐based dose prediction methods still suffer from limited generalizability and an inability to effectively handle outlier clinical cases. This may lead to inaccurate dose delivery to targets or excessive irradiation to organs at risk (OARs), thereby compromising the safety and efficacy of online adaptive proton therapy. Purpose To design a physics‐aware and generalizable AI‐based PBSPT dose prediction method that incorporates underlying physics to enhance generalizability, particularly in handling outlier clinical cases. Methods This study analyzed PBSPT plans of 103 prostate (93 for training and 10 for testing) and 78 lung cancer patients (68 for training and 10 for testing) from our institution, with each case comprising CT images and structure sets. Using the doses generated by our Monte Carlo‐based dose engine as the reference standard, we compared three methods: the region of interest (ROI)‐based method, the beam mask and sliding window method, and the proposed noisy probing dose method, which rapidly generates a low‐statistics dose via uniformly weighted spots on an expanded spot‐placement target volume without optimization. To evaluate the generalizability of these methods to rare treatment planning scenarios, 12 cases with uncommon beam angles or prescription doses were used to assess their performance, which was evaluated using dose‐volume histogram (DVH) indices, 3D Gamma passing rates (3%/2 mm/10%), and Dice coefficients for dose agreement, while prediction times were measured to gauge model efficiency. Results The proposed noisy probing dose method consistently outperformed the ROI‐based and beam mask baselines across all evaluated metrics, with more accurate dose agreement and superior generalizability. For DVH indices, the noisy probing dose method achieved the smallest deviation in clinical target volume (CTV) dose coverage: in prostate cancer, CTV D98 deviation was reduced by 45% (from 0.53 ± 0.22 Gy [RBE] for ROI‐based) and 29% (from 0.41 ± 0.28 Gy [RBE] for beam mask) to 0.29 ± 0.06 Gy [RBE]; in lung cancer, similar improvements were observed, with CTV D98 deviations reduced to 0.34 ± 0.12 Gy [RBE]. The 3D Gamma passing rates improved to 99.65% ± 1.15% for prostate targets and 97.04% ± 1.17% for lung targets. The dice coefficients of the 90% iso‐dose lines were also the highest with the noisy probing dose method (prostate: 0.983 ± 0.005; lung: 0.967 ± 0.01). For the 12 outlier cases, the noisy probing dose method maintained superior generalizability, yielding higher 3D Gamma passing rates (prostate targets: 96.79% ± 0.83%, OARs: 94.29% ± 1.01%; lung targets: 93.38% ± 1.34%, OARs: 93.95% ± 1.32%), demonstrating robust generalizability to rare clinical scenarios. The dose predictions for all testing cases were completed within 0.3 seconds. Conclusions A novel noisy probing dose method was proposed for PBSPT dose prediction in prostate and lung cancer patients. By embedding more proton‐specific physics, this method demonstrated an improvement in the generalizability of dose prediction.
Retrieval-Augmented Generation in Oncology: Promises, Pitfalls, and Early Applications
Thaker, Nikhil G. , Liu, Wei, Waddle, Mark, SHOWALTER, TIMOTHY, Luh, Join, Levitt, Chad, Ning, Matthew, Loaiza-Bonilla, Arturo, Hong, Julian, Mastroleo, Federico , Showalter, Timothy N. , Luh, Join Y. , Loaiza‐Bonilla, Arturo

Mayo Clinic

AI in Precision Oncology | April 29, 2026 | Volume: 3 | Issue: 2-3 | Pages: 34-45

10.1177/2993091x261446348 | PMID 42305950 Get It

Retrieval-augmented generation (RAG) is rapidly emerging as a transformative paradigm for large language models (LLMs), especially in high-stakes domains like oncology that demand precision, factual grounding, and up-to-date knowledge. By pairing LLMs with external knowledge repositories, RAG systems explicitly ground model outputs in relevant retrieved documents, helping to reduce hallucinations and ensure responses reflect current evidence. In oncology, where clinical knowledge evolves continually with new research and drug approvals, RAG offers a way to integrate the latest data (e.g., trial results, guidelines, genomic databases) into decision-making. This review synthesizes the technical foundations of RAG, including its architecture and key components, and examines current applications in oncology such as clinical decision support, patient education, radiology reporting, pathology analysis, and genomics-driven precision medicine. We highlight recent studies that demonstrate RAG’s potential—for instance, improving treatment recommendations by incorporating genetic profiles and literature, and enhancing diagnostic accuracy by integrating guidelines. We also discuss emerging developments like multimodal RAG (combining text with imaging or other data), ensemble model approaches, and new explainability tools that trace model outputs to sources. Finally, we critically analyze the limitations and challenges of deploying RAG in healthcare, including computational costs, retrieval errors, noise or conflicts in retrieved information, and ethical and regulatory considerations. While RAG-based systems show promise in augmenting oncologists’ expertise with timely knowledge, careful implementation, high-quality curation of knowledge bases, and human oversight will be crucial for safe and effective adoption in clinical practice.
Abstract 5051: A multi-center hospital-based case-control study of lymphoid and myeloid neoplasms (AsiaLymph): Study design and initial findings for education, body mass index, and family history
Lan, Qing, Chan, John K., Lam, Tai Hing , Chen, Kexin, Kwong, Yok Lam, Xu, Caigang , Chiu, Brian CH, Liang, Raymond , Dennis, Ip, Hu, Wei, Bassig, Bryan A. , Purdue, Mark, Xu, Jun, Locke, Sarah J. , Wang, Sophia S., Berndt, Sonja, Hofmann, Jonathan N., Shi, Jianxin , Yu, Kai, Gadalla, Shahinaz, Jones, Rena, Dai, Hongji, Lyu, Zhangyan, Qiu, Lugui , Liu, Wei, zhang, huilai, Wang, Xianhuo, Chanock, Stephen, Linet, Martha, Vermeulen, Roel, Rothman, Nathaniel, Lan, Qing , Hurwitz, Lauren M. , Chan, John K.C., Chen, Kexin , Chiu, Brian C.‐H. , Cerhan, James R. , Hofmann, Jonathan N. , Gadalla, Shahinaz M. , McReynolds, Lisa J. , Jones, Rena R. , Qiu, Lugui , Liu, Wei , Zhang, Huilai , Wang, Xianhuo , Morton, Lindsay M. , Linet, Martha S. , Friesen, Melissa C. , Vermeulen, Roel , Rothman, Nathaniel

Mayo Clinic

Cancer Research | April 3, 2026 | Volume: 86 | Issue: 7_Supplement | Pages: 5051-5051

10.1158/1538-7445.am2026-5051 Get It

Abstract Background: Epidemiologic and other studies have identified environmental, occupational, and genetic risk factors for lymphoid and myeloid malignancies, but most studies have been conducted in Western populations. Investigations in populations with differing exposure patterns, distributions of disease subtypes, and genetic architecture are needed to fully understand hematopoietic tumor etiology. Methods: We conducted a large, multicenter, hospital-based case-control study of lymphoid and myeloid neoplasms in East Asia (AsiaLymph), including 5,671 lymphoid cases, 1,879 myeloid cases, and 3,858 controls. Participants completed a computer-assisted personal interview and provided biospecimens. Cases underwent central pathology review and were coded into the WHO Classification. Herein, we describe the study methods in detail and report association results for education, body mass index, and family history. Results: Greater BMI at age 20 but not at age 40 was associated with odds of total lymphoid neoplasm (per 5 kg/m2 increase: OR [95% CI]: 1.18 [1.09-1.27]), total myeloid neoplasm (OR [95% CI]: 1.17 [1.05-1.29]), and several subtypes. Greater educational attainment was associated with increased odds of total lymphoid neoplasm (for college vs. less than primary education: OR [95% CI]: 1.41 [1.21-1.65]) but not myeloid neoplasm (OR [95% CI]: 1.17 [0.95-1.45]; p-heterogeneity=0.02). There were also positive associations between family history of hematologic cancer in first degree relatives and odds of lymphoid neoplasm (OR [95% CI]: 1.53 [1.15-2.03]) and myeloid neoplasm (OR [95% CI]: 1.65 [1.11-2.44]) and specific subtypes. None of the evaluated risk factors were associated with NK/T-cell lymphoma, which supports a distinct etiology for this subtype. Conclusion: The AsiaLymph Study is one of the largest molecular epidemiology studies of both lymphoid and myeloid neoplasms with standardized World Health Organization classification of histopathologic subtypes and will serve as a valuable resource for etiologic investigations into risk factors for these malignancies. Citation Format: Qing Lan, Lauren M. Hurwitz, John K. Chan, Tai Hing Lam, Kexin Chen, Yok Lam Kwong, Xu Caigang, Brian CH Chiu, Raymond Liang, Ip Dennis, Wei Hu, Bryan Bassig, Mark Purdue, Jun Xu, Sarah Locke, Sophia S. Wang, James R. Cerhan, Sonja Berndt, Jonathan N. Hofmann, Jianxin Shi, Kai Yu, Shahinaz Gadalla, Lisa J. McReynolds, Rena Jones, Hongji Dai, Zhangyan Lyu, Lugui Qiu, Wei Liu, Huilai Zhang, Xianhuo Wang, Lindsay M. Morton, Stephen Chanock, Martha Linet, Melissa C. Friesen, Roel Vermeulen, Nathaniel Rothman. A multi-center hospital-based case-control study of lymphoid and myeloid neoplasms (AsiaLymph): Study design and initial findings for education, body mass index, and family history [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2026; Part 1 (Regular Abstracts); 2026 Apr 17-22; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2026;86(7 Suppl):Abstract nr 5051.
Med-VLM: Enhancing Medical Image Segmentation Accuracy Through Vision-Language Model
Zhao, Yihao, Zhong, Enhao, Yuan, Cuiyun, Li, Yang, Zhao, Man , Li, Chunxia, Hu, Jun, Liu, Wei, Liu, Chaoying

Mayo Clinic

2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) | February 23, 2026 | Volume: No volume available | Issue: No issue available | Pages: 7342-7352

10.1109/iccvw69036.2025.00756 Get It

Collaborative large language models (LLMs) are all you need for screening in systematic reviews
Parmar, Mihir, Naqvi, Syed Arsalan Ahmed , Warraich, Kainat , Saeidi, Amir, Rawal, Samarth, Faisal, Kunwer Sufyan , Kazmi, Syeda Zainab , Fatima, Maurish , He, Huan, Safdar, Mohammad, Liu, Wei, Haddad, Tufia C., Wang, Zhen , Murad, Mohammad Hassan, Baral, Chitta, Haddad, Tufia, Riaz, Irbaz Bin , Saeidi, Amir, Haddad, Tufia C.

Mayo Clinic

No container title available | February 18, 2026 | Volume: No volume available | Issue: No issue available | Pages: No page available

10.64898/2026.02.07.26345640 | PMID 41757212 Get It

Abstract Background The ability of large language models (LLMs) to work collaboratively and screen studies in a systematic review (SR) is under-explored. Hence, we aimed to evaluate the effectiveness of LLMs in automating the process of screening in systematic reviews. Methods This is an observational study which included labeled data (title and abstracts) for five SRs. Originally, two reviewers screened the citations independently for eligibility. A third reviewer cross-checked each citation for quality assurance. GPT-4, Claude-3-Sonnet, and Gemini-Pro-1.0 were used using zero-shot chain-of-thought prompting. Collaborative approaches included (i): conflict resolution using benefit of the doubt, (ii) majority voting using an independent third LLM and (iii) conflict resolution using an informed third LLM. Performance was assessed using accuracy, precision for exclusion, and recall for inclusion. Work saved over samples (WSS) was computed to estimate the reduction in manual human effort. Results A total of 11300 articles were included in this study. The individual models, GPT-4, Claude-3-Sonnet, and Gemini-Pro-1.0 exhibited a high precision for exclusion, achieving 99.7%, 99.7%, and 99.2% and high recall for inclusion achieving 95.5%, 96.6% and 85.7%, respectively. However, the collaborative approach utilizing the two best-performing models (GPT-4 and Claude-3S) achieved an average precision of 99.9% and a recall of 98.5% (across all collaborative approaches). Furthermore, the proposed collaborative approach resulted in an average WSS of 63.5%, compared to the average WSS of 45.2% for individual models. Conversational LLM interactions showed a consistent pattern of results. Limitations This study was limited due to reliance on proprietary models, and evaluation on oncology datasets. Conclusion Evidence shows that collaborative LLMs enable efficient, high-performing screening in systematic reviews, supporting continuous evidence updates. Primary funding source NIH (U24CA265879-01-1) and Carolyn-Ann-Kennedy-Bacon Fund.
Correlative biomarker analyses of etentamig in patients with Relapsed/Refractory multiple myeloma (RRMM) at optimal dose support its potential to maximize T-cell activity with rapid and transient proinflammatory cytokine responses
Ahsan, Aarif, Mantis, Christine, Luo, Adam , Zheng, Pingping , Liu, Wei , Zhang, Catherine, Rodriguez, Cesar, Voorhees, Peter M. , Safah, Hana, McKay, John, Korde, Neha , Vij, Ravi , Kumar, Shaji, Mian, Hira , Pothacamury, Rajvineeth Kumar, Svensson, Anders , Mian, Hira , D’Souza, Anita , Chung, Alfred , Tuchman, Sascha Alexander , Weisel, Katja C , Teipel, Raphael , Kumar, Shaji , Baljević, Muhamed , Talati, Chetasi , Ross, Jeremy A.

Mayo Clinic

Blood | December 4, 2025 | Volume: 146 | Issue: Supplement 1 | Pages: 2204-2204

10.1182/blood-2025-2204 Get It

Abstract Background Etentamig is a second-generation B-cell maturation antigen (BCMA) x CD3 bispecific antibody that has shown promising efficacy and a favorable safety profile in heavily pretreated patients with relapsed/refractory multiple myeloma (RRMM). Etentamig possesses a unique combination of 2 high-affinity BCMA binding domains, coupled to a low-affinity CD3 binding domain that have been shown to reduce the negative impact of soluble BCMA (sBCMA), decrease risk for cytokine release syndrome (CRS), and drive sustained T-cell activation with reduced immune exhaustion in preclinical models of MM (Blood 2023;142 [suppl 1]:4666). Here, we describe correlative biomarker analyses from Phase 1 (NCT03933735) and Phase 1b (NCT05650632) clinical trials of etentamig in RRMM designed to assess the impact of sBCMA on clinical response and pharmacodynamic changes in proinflammatory cytokines, and T-cell redistribution, activation, proliferation, and exhaustion. Methods Peripheral blood and serum samples from patients with RRMM enrolled in Phase 1 and Phase 1b open-label studies who received etentamig monotherapy were collected at baseline, on treatment, and at disease progression. The Phase 1b study consisted of dose-optimization (DO) and dose-expansion (DE) phases. During DO, patients received a single step-up dose (SUD) of intravenous etentamig (2 or 4 mg) on Cycle (C) 1 Day (D) 1, followed by the target dose of 60 mg on D4. In DE, based on DO findings, patients received the recommended etentamig SUD (2 mg) on C1D1, followed by 60 mg on D4 with modified dexamethasone predosing. Samples were analyzed by flow cytometry for immune cell populations, Luminex® for cytokines, and electrochemiluminescence for sBCMA. Peripheral blood mononuclear cells and bone marrow mononuclear cells (BMMC) at baseline and on-treatment were subjected to single-cell Cellular Indexing of Transcriptomes and Epitopes sequencing (scCITEseq) and single-cell T-cell receptors (scTCR) / B-cell receptors clonality analyses. Results While baseline serum sBCMA levels did not correlate with clinical response to etentamig at the selected dose level of 60 mg every 4 weeks (Q4W) with 2 mg SUD, a reduction in sBCMA levels over time was associated with best International Myeloma Working Group response. Etentamig treatment led to a rapid and transient increase of proinflammatory cytokines and promoted T-cell redistribution, activation, and proliferation. Maximum reduction in peak levels of CRS-related cytokines, including (interleukin [IL]-6, IL-8, IL-10, tumor necrosis factor–a), were observed in DE compared with DO and 60 mg Q4W without SUD. Peak induction levels of proinflammatory cytokines during C1 correlated with occurrence of CRS (≥Grade [G] 1) as well as clinical response across dose levels. A modified dexamethasone premedication schedule of 2 mg/60 mg Q4W cohort for C1 resulted in significantly reduced baseline levels of IL-6 and key macrophage-associated cytokine/chemokines (eg, monocyte chemoattractant protein-1 and macrophage inflammatory protein-3 beta). Incidence of CRS was lowest in the 2 mg/60 mg Q4W cohort (26% G1, 4% G2, 0% ≥G3). Higher T-cell to myeloma cell ratio in BMMCs at baseline correlated with better response to etentamig. scCITEseq and scTCR analysis revealed that at baseline, responders also had greater CD8+ T-cell cytotoxic activities, increased TCR clonality, and elevated inflammatory signatures in T-cells from both peripheral blood and bone marrow. Etentamig at a dosage of 60 mg Q4W resulted in significant clonal expansion of CD8+ effector T-cells and enhanced gene expression signatures related to activation and cytotoxicity in both CD8+ T effector and natural killer cells. Along treatment cycles, responders maintained their T-cell fitness, high clonality, and low T-cell exhaustion levels. Frequency of baseline CD4+ or CD8+ T-cell exhaustion (programmed cell death receptor 1+/ T cell immunoglobulin and mucin-domain containing-3 +) did not impact clinical response at optimal dose level of 2 mg/60 mg Q4W etentamig. Conclusions Responses to etentamig were independent of baseline sBCMA levels and resulted in rapid and transient proinflammatory cytokine production that promoted robust T-cell redistribution, activation, and proliferation indicating that the optimal dose of etentamig (2 mg/60 mg Q4W) maximizes its clinical potential as a convenient, safe, and effective therapy for MM.
Using large language models to automate summarization of CT simulation orders in radiation oncology
Cao, Meiyun, Hu, S.X. , Sharp, Jason, Clouser, Edward , Lam, Louis, Ding, Xiaoning, Toesca, Diego Santos, Raulston, Brianne C, Lindholm, Wendy S., Patel, Samir H. , Vora, Sujay A., Wang, P. , Clouser, Edward Lee , Lam, Linda L, Wang, Peilong , Liu, Wei , Toesca, Diego A.S. , Hu, Shaw, Sharp, Jason, Holmes, Jason , Liu, Wei

Mayo Clinic

Journal of Applied Clinical Medical Physics | October 28, 2025 | Volume: 26 | Issue: 11 | Pages: No page available

10.1002/acm2.70310 | PMID 41144781 Get It

Abstract Background In the current clinical workflow of radiation oncology, therapists manually summarize the physician‐issued Computed Tomography (CT) simulation orders to prepare patients' CT simulations. This process increases the workload, introduces variability in documentation quality, and is prone to human errors. Purpose This study aims to address these challenges using a large language model (LLM) to automate the generation of summaries from the CT simulation orders and evaluate its performance. Methods A total of 607 CT simulation orders were collected from the Aria database at our institution. A locally hosted LLaMa 3.1 405B model, accessed via the Application Programming Interface (API) service, was used to extract keywords from the CT simulation orders and generate summaries. The downloaded CT simulation orders were categorized into seven groups based on treatment modalities and disease sites. For each group, a customized instruction prompt was developed collaboratively with therapists to guide the LLaMa 3.1 405B model in generating summaries. The ground truth for the corresponding summaries was manually derived by carefully reviewing each CT simulation order and subsequently verified by therapists. The accuracy of the LLM‐generated summaries was evaluated by therapists using the ground truth. Results Over 98% of the LLM‐generated summaries aligned with the ground truth in terms of accuracy. Our evaluations showed an improved consistency in format and enhanced readability from the LLM‐generated summaries compared to the corresponding therapist‐generated summaries. This automated approach demonstrated consistent performance across all groups, regardless of treatment modality or disease site. Conclusions This study demonstrated the high precision and consistency of the LLaMa 3.1 405B model in extracting keywords and summarizing CT simulation orders, suggesting that LLMs have great potential to assist in this task, reduce the workload of CT simulation therapists and improve radiation oncology workflow efficiency.