Toesca, Diego Santos

Abstract Background In the current clinical workflow of radiation oncology, therapists manually summarize the physician‐issued Computed Tomography (CT) simulation orders to prepare patients' CT simulations. This process increases the workload, introduces variability in documentation quality, and is prone to human errors. Purpose This study aims to address these challenges using a large language model (LLM) to automate the generation of summaries from the CT simulation orders and evaluate its performance. Methods A total of 607 CT simulation orders were collected from the Aria database at our institution. A locally hosted LLaMa 3.1 405B model, accessed via the Application Programming Interface (API) service, was used to extract keywords from the CT simulation orders and generate summaries. The downloaded CT simulation orders were categorized into seven groups based on treatment modalities and disease sites. For each group, a customized instruction prompt was developed collaboratively with therapists to guide the LLaMa 3.1 405B model in generating summaries. The ground truth for the corresponding summaries was manually derived by carefully reviewing each CT simulation order and subsequently verified by therapists. The accuracy of the LLM‐generated summaries was evaluated by therapists using the ground truth. Results Over 98% of the LLM‐generated summaries aligned with the ground truth in terms of accuracy. Our evaluations showed an improved consistency in format and enhanced readability from the LLM‐generated summaries compared to the corresponding therapist‐generated summaries. This automated approach demonstrated consistent performance across all groups, regardless of treatment modality or disease site. Conclusions This study demonstrated the high precision and consistency of the LLaMa 3.1 405B model in extracting keywords and summarizing CT simulation orders, suggesting that LLMs have great potential to assist in this task, reduce the workload of CT simulation therapists and improve radiation oncology workflow efficiency.

Abstract Background The radiation oncology clinical practice involves many steps relying on the dynamic interplay of abundant text data. Large language models have displayed remarkable capabilities in processing complex text information. But their direct applications in specific fields like radiation oncology remain underexplored. Purpose This study aims to investigate whether fine‐tuning LLMs with domain knowledge can improve the performance on Task (1) treatment regimen generation, Task (2) treatment modality selection (photon, proton, electron, or brachytherapy), and Task (3) ICD‐10 code prediction in radiation oncology. Methods Data for 15 724 patient cases were extracted. Cases where patients had a single diagnostic record, and a clearly identifiable primary treatment plan were selected for preprocessing and manual annotation to have 7903 cases of the patient diagnosis, treatment plan, treatment modality, and ICD‐10 code. Each case was used to construct a pair consisting of patient diagnostics details and an answer (treatment regimen, treatment modality, or ICD‐10 code, respectively) for the supervised fine‐tuning of these three tasks. Open source LLaMA2‐7B and Mistral‐7B models were utilized for the fine‐tuning with the Low‐Rank Approximations method. Accuracy and ROUGE‐1 score were reported for the fine‐tuned models and original models. Clinical evaluation was performed on Task (1) by radiation oncologists, while precision, recall, and F‐1 score were evaluated for Task (2) and (3). One‐sided Wilcoxon signed‐rank tests were used to statistically analyze the results. Results Fine‐tuned LLMs outperformed original LLMs across all tasks with p value ≤ 0.001. Clinical evaluation demonstrated that over 60% of the fine‐tuned LLMs‐generated treatment regimens were clinically acceptable. Precision, recall, and F1‐score showed improved performance of fine‐tuned LLMs. Conclusion Fine‐tuned LLMs demonstrated statistically significant improvements over original LLMs upon three clinically important tasks in radiation oncology. This study explored the feasibility of applying fine‐tuned LLMs in radiation oncology, inspiring further development of utilizing LLMs to assist with radiation oncology tasks.

Abstract Purpose Lattice radiation therapy (LRT) is a form of spatially fractionated radiation therapy that allows increased total dose delivery aiming for improved treatment response without an increase in toxicities, commonly utilized for palliation of bulky tumors. The LRT treatment planning process is complex, while eligible patients often have an urgent need for expedited treatment start. In this study, we aimed to develop a simulation‐free workflow for volumetric modulated arc therapy (VMAT)‐based LRT planning via deep learning‐predicted synthetic CT (sCT) to expedite treatment initiation. Methods Two deep learning models were initially trained using 3D U‐Net architecture to generate sCT from diagnostic CTs (dCT) of the thoracic and abdomen regions using a training dataset of 50 patients. The models were then tested on an independent dataset of 15 patients using image similarity analysis assessing mean absolute error (MAE) and structural similarity index measure (SSIM) as metrics. VMAT‐based LRT plans were generated based on sCT and recalculated on the planning CT (pCT) for dosimetric accuracy comparison. Differences in dose volume histogram (DVH) metrics between pCT and sCT plans were assessed using the Wilcoxon signed‐rank test. Results The final sCT prediction model demonstrated high image similarity to pCT, with a MAE and SSIM of 38.93 ± 14.79 Hounsfield Units (HU) and 0.92 ± 0.05 for the thoracic region, and 73.60 ± 22.90 HU and 0.90 ± 0.03 for the abdominal region, respectively. There were no statistically significant differences between sCT and pCT plans in terms of organ‐at‐risk and target volume DVH parameters, including maximum dose (Dmax), mean dose (Dmean), dose delivered to 90% (D90%) and 50% (D50%) of target volume, except for minimum dose (Dmin) and (D10%). Conclusion With demonstrated high image similarity and adequate dose agreement between sCT and pCT, our study is a proof‐of‐concept for using deep learning predicted sCT for a simulation‐free treatment planning workflow for VMAT‐based LRT.

Abstract Background Inter‐fractional and intra‐fractional positioning and motion monitoring based on implanted fiducials through kilovoltage (kV) imaging is a cost‐effective approach to enhance intact prostate radiotherapy treatment accuracy. However, comprehensive studies for three‐dimensional (3D) position extraction and the corrective action threshold are lacking. Purpose To develop and verify a fiducial motion monitoring system with 3D position information based on triggered kV images taken during treatment delivery, and to comprehensively evaluate the recommended thresholds for corrective action during treatment. Methods An in‐house fiducial triangulation algorithm was developed to monitor fiducial positions using subsequent kV triggered images. The precision of fiducial triangulation and motion detection was validated on a pelvis phantom with four fiducial inserts. A retrospective study was conducted on prostate cancer patients who received either moderately hypofractionated radiotherapy (MHRT, n = 16) or stereotactic body radiotherapy (SBRT, n = 12), categorized by endo‐rectal balloon (ERB) use. Intra‐fractional positions relative to isocenter were computed, and these positions were analyzed in comparison to the original marker locations of the patient's treatment plan. A linear regression fit was used per fraction to determine correlation coefficients between motions in the left–right (LR), superior–inferior (SI), and anterior–posterior (AP) directions. Average fraction time (AFT) of treatment was reported based on the observed average time spent on steps in the workflow and the percentage of fiducials caught outside of the current two‐dimensional (2D) tolerance threshold of 5 mm. The observed 3D offsets were used to estimate AFT for various 3D tolerance thresholds. Results Overall, our triangulation method proved to be very precise for static cases, where phantom measurements revealed a fiducial position triangulation precision of mm for stationary targets, but had a spread of mm for targets with 1 mm of motion. Employing triangulation, true motion exceeding 5 mm was detected above 4.0 mm in magnitude 90.9% of the time, a noticeable improvement in comparison to the 28%–65% successful detection rate reported by 2D projection methods. Most detection errors were attributed to depth disparity. Similar to other reports, correlation coefficients for intra‐fractional motion generally indicated no LR/SI, no LR/AP, and weak positive SI/AP correlations for MHRT and SBRT patients. Offsets beyond a 3D tolerance threshold of 5 mm were observed at a rate of 4.25%–5.25%, while the 5 mm 2D out‐of‐tolerance catch‐rate was 1.6%. The AFT was 8.1–8.2 min using the 2D tolerance threshold of 5 mm. In comparison, the estimated AFT for the proposed 3D tolerance monitoring of offsets beyond 2–5 mm was slightly higher at 8.2–10.9 min due to the higher amount of out‐of‐tolerance instances for the higher precision intra‐fractional motion management. Conclusions Our study showcases a promising subsequent kV‐based triangulation method for intra‐fractional prostate motion monitoring. Acquiring 3D motion information results in higher out‐of‐tolerance catch‐rates particularly in the depth dimension of the kV images, which is perpendicular to the treatment beam. Failure to properly observe and catch these offsets would result in sub‐optimal conformality and accuracy of the dose delivery.

AbstractBackgroundDeep learning auto‐segmentation (DLAS) models have been adopted in the clinic; however, they suffer from performance deterioration owing to the clinical practice variability. Some commercial DLAS software provide an incremental retraining function that enables users to train a custom model using their institutional data to account for clinical practice variability.PurposeThis study was performed to evaluate and implement the commercial DLAS software with the incremental retraining function for definitive treatment of patients with prostate cancer in a multi‐user environment.MethodsCT‐based target organs and organs‐at‐risk (OAR) delineation of 215 prostate cancer patients were utilized. The performance of three commercial DLAS software built‐in models was validated with 20 patients. A retrained custom model was developed using 100 patients and evaluated on the remaining data (n = 115). Dice similarity coefficient (DSC), Hausdorff distance (HD), mean surface distance (MSD), and surface DSC (SDSC) were utilized for quantitative evaluation. A multi‐rater qualitative evaluation was blindly performed with a five‐level scale. Visual inspection was performed in consensus and non‐consensus unacceptable cases to identify the failure modes.ResultsThree commercial DLAS vendor built‐in models achieved sub‐optimal performance in 20 patients. The retrained custom model had a mean DSC of 0.82 for prostate, 0.48 for seminal vesicles (SV), and 0.92 for rectum, respectively. This represents a significant improvement over the built‐in model with DSC of 0.73, 0.37, and 0.81 for the corresponding structures. Compared to the acceptance rate of 96.5% and consensus unacceptable rate (i.e., both reviewers rated as unacceptable) of 3.5% achieved by manual contours, the custom model achieved a 91.3% acceptance rate and 8.7% consensus unacceptable rate. The failure modes of retrained custom model were attributed to the following: cystogram (n = 2), hip prosthesis (n = 2), low dose rate brachytherapy seeds (n = 2), air in endorectal balloon(n = 1), non‐iodinated spacer (n = 2), and giant bladder(n = 1).ConclusionThe commercial DLAS software with the incremental retraining function was validated and clinically adopted for prostate patients in a multi‐user environment. AI‐based auto‐delineation of the prostate and OARs is shown to achieve improved physician acceptance, overall clinical utility, and accuracy.