Citations

Abstract Objective In acupuncture therapy, the accurate location of acupoints is essential for its effectiveness. The advanced language understanding capabilities of large language models (LLMs) like Generative Pre-trained Transformers (GPTs) and Llama present a significant opportunity for extracting relations related to acupoint locations from textual knowledge sources. This study aims to explore the performance of LLMs in extracting acupoint-related location relations and assess the impact of fine-tuning on GPT’s performance. Materials and Methods We utilized the World Health Organization Standard Acupuncture Point Locations in the Western Pacific Region (WHO Standard) as our corpus, which consists of descriptions of 361 acupoints. Five types of relations (“direction_of”, “distance_of”, “part_of”, “near_acupoint”, and “located_near”) (n = 3174) between acupoints were annotated. Four models were compared: pre-trained GPT-3.5, fine-tuned GPT-3.5, pre-trained GPT-4, as well as pretrained Llama 3. Performance metrics included micro-average exact match precision, recall, and F1 scores. Results Our results demonstrate that fine-tuned GPT-3.5 consistently outperformed other models in F1 scores across all relation types. Overall, it achieved the highest micro-average F1 score of 0.92. Discussion The superior performance of the fine-tuned GPT-3.5 model, as shown by its F1 scores, underscores the importance of domain-specific fine-tuning in enhancing relation extraction capabilities for acupuncture-related tasks. In light of the findings from this study, it offers valuable insights into leveraging LLMs for developing clinical decision support and creating educational modules in acupuncture. Conclusion This study underscores the effectiveness of LLMs like GPT and Llama in extracting relations related to acupoint locations, with implications for accurately modeling acupuncture knowledge and promoting standard implementation in acupuncture training and practice. The findings also contribute to advancing informatics applications in traditional and complementary medicine, showcasing the potential of LLMs in natural language processing.

Abstract Alzheimer's disease and related dementias (ADRD) significantly impact patients and their caregivers, causing emotional stress, lack of training, and financial strain for the latter. Addressing the need for effective support, this research developed and validated ADQueryAid, a conversational AI system designed to empower ADRD caregivers. Built on a Large Language Model (LLM) and enriched with authoritative ADRD information through a knowledge graph, ADQueryAid uses Retrieval Augmented Generation (RAG) techniques to retrieve relevant information from both structured and unstructured data sources. Prompt engineering and conversation policies ensure the delivery of informative, empathetic, and personalized responses. Evaluated via a Blind Within-Subjects Design study with 20 ADRD caregivers interacting with both ADQueryAid and a baseline model (ChatGPT 3.5) in fictional caregiving scenarios, the system's usability was assessed using the Chatbot Usability Questionnaire (CUQ). Results showed ADQueryAid significantly outperformed the baseline model across all CUQ metrics, offering more contextually relevant information, accurate guidance, and emotional support, which enhanced the caregiving experience. This study highlights the potential of AI to support ADRD caregivers by providing personalized assistance, with future research focusing on expanding the knowledge base, refining conversational strategies, and exploring the long-term impact on caregiver well-being and patient outcomes.

Abstract Objectives Precise literature recommendation and summarization are crucial for biomedical professionals. While the latest iteration of generative pretrained transformer (GPT) incorporates 2 distinct modes—real-time search and pretrained model utilization—it encounters challenges in dealing with these tasks. Specifically, the real-time search can pinpoint some relevant articles but occasionally provides fabricated papers, whereas the pretrained model excels in generating well-structured summaries but struggles to cite specific sources. In response, this study introduces RefAI, an innovative retrieval-augmented generative tool designed to synergize the strengths of large language models (LLMs) while overcoming their limitations. Materials and Methods RefAI utilized PubMed for systematic literature retrieval, employed a novel multivariable algorithm for article recommendation, and leveraged GPT-4 turbo for summarization. Ten queries under 2 prevalent topics (“cancer immunotherapy and target therapy” and “LLMs in medicine”) were chosen as use cases and 3 established counterparts (ChatGPT-4, ScholarAI, and Gemini) as our baselines. The evaluation was conducted by 10 domain experts through standard statistical analyses for performance comparison. Results The overall performance of RefAI surpassed that of the baselines across 5 evaluated dimensions—relevance and quality for literature recommendation, accuracy, comprehensiveness, and reference integration for summarization, with the majority exhibiting statistically significant improvements (P-values <.05). Discussion RefAI demonstrated substantial improvements in literature recommendation and summarization over existing tools, addressing issues like fabricated papers, metadata inaccuracies, restricted recommendations, and poor reference integration. Conclusion By augmenting LLM with external resources and a novel ranking algorithm, RefAI is uniquely capable of recommending high-quality literature and generating well-structured summaries, holding the potential to meet the critical needs of biomedical professionals in navigating and synthesizing vast amounts of scientific literature.

Abstract Background: Oncofertility counseling in BRCA carriers has unique challenges due to a possible negative impact of germline pathogenic/likely pathogenic variants (PVs) on reproductive potential and the indication to undergo risk-reducing bilateral salpingo-oophorectomy. While previous studies demonstrated the safety of pregnancy in women with prior breast cancer (BC) history, limited evidence exists in BRCA carriers specifically. We report the results of the largest study to date investigating likelihood of pregnancy after BC, reproductive and disease outcomes specifically in BRCA carriers. Methods: This was an international, multicenter, hospital-based, retrospective cohort study including young BRCA carriers with history of BC (NCT03673306). Eligible patients were women harboring germline PVs in BRCA1 and/or BRCA2 and diagnosed with stage I-III invasive BC at the age of ≤ 40 years between January 2000 and December 2020. BRCA healthy carriers or patients with BRCA variants of unknown significance, non-invasive BC, lack of follow-up, prior ovarian cancer or other malignancies without history of BC were excluded. Primary endpoints were pregnancy rate and disease-free survival (DFS). Overall survival (OS), BC specific survival (BCSS), pregnancy, fetal and obstetric outcomes were secondary endpoints. To mitigate the impact of guaranteed time bias, two survival analyses were performed: 1) Extended Cox model with occurrence of pregnancy as a time-varying covariate; 2) Case-control analysis by matching 1:3 patients with and without a pregnancy for type of BRCA PV, hormone receptor status, nodal status and year at diagnosis; moreover, each control had to have a disease-free interval longer than the time occurring between BC diagnosis and conception of the matched case. Results: From 78 centers worldwide, 4,732 patients were included, of whom 659 had at least one pregnancy after BC (pregnancy cohort) and 4,073 did not (non-pregnancy cohort). Patients in the pregnancy cohort were significantly more likely to carry BRCA1 PVs, to be younger at BC diagnosis, to have node-negative and hormone receptor-negative (HR-) BC (all p< 0.01). Pregnancy rate at 10 years was 22% (95% CI 21%-24%) overall, 18% (95% CI 16%-21%) and 26% (95% CI 24%-26%) in patients with HR+ and HR- BC, respectively (p< 0.01). Median time from BC diagnosis to conception was 3.5 years (IQR 2.2-5.3 years), significantly longer in patients with HR+ than HR- BC (4.3 vs 3.2 years, p< 0.01). Among 659 patients with a pregnancy after BC, 45 (6.8%) and 63 (9.6%) experienced an induced abortion or a miscarriage, respectively. Of the 517 (78.5%) patients with completed pregnancy, 54 (10.4%) had twins and 406 (78.5%) delivered at term (delivery ≥37 weeks). Among the 571 babies born, 5 (0.9%) had congenital anomalies. Median follow-up was 7.8 years (IQR, 4.5-12.6 years). No significant difference in DFS was observed between the pregnancy and non-pregnancy cohorts (unadjusted HR 0.97, 95% CI 0.82-1.15, p=0.74; adjusted HR [aHR] 0.99, 95% CI 0.81-1.20, p=0.90). Patients in the pregnancy cohort showed significantly better BCSS (HR 0.53, 95% CI 0.38-0.74, p< 0.01; aHR 0.59, 95% CI 0.41-0.86, p< 0.01) and OS (HR 0.52, 95% CI 0.38-0.72, p< 0.01; aHR 0.58, 95% CI 0.40-0.85, p< 0.01). Similar results were obtained in the case-control analysis. Subgroup analyses of survival endpoints according to patient, tumor and treatment characteristics will be presented at the conference. Conclusions: In this global study, 1 out of 5 young BRCA carriers conceived within 10 years after a BC diagnosis, higher than rates previously reported in unselected young BC populations. Pregnancy following BC in BRCA carriers did not appear to adversely impact maternal prognosis or fetal outcomes. These findings should be incorporated into oncofertility counseling of young BRCA carriers with BC. Citation Format: Matteo Lambertini, Eva Blondeaux, Elisa Agostinetto, Anne-Sophie Hamy-Petit, Hee Jeong Kim, Camila Chiodi, Rinat Bernstein-Molho, Florentine Hilbers, Katarzyna Pogoda, Estela Carrasco Lopez, Kevin Punie, Jyoti Bajpai, Michail Ignatiadis, Halle Moore, Kelly-Anne Phillips, Angela Toss, Christine Rousset-Jablonski, Fedro Alessandro Peccatori, Tiphaine Renaud, Alberta Ferrari, Shani Paluch-Shimon, Robert Fruscio, Wanda Cui, Stephanie Wong, Claudio Vernieri, Kathryn Ruddy, Maria Vittoria Dieci, Alexios Matikas, Mariya Rozenblit, Cynthia Villareal Garza, Laura De Marchis, Lucia Del Mastro, Fabio Puglisi, Maria Del Pilar Estevez-Diz, Kenny A. Rodriguez-Wallberg, Bela Mrinakova, Sarah Meister, Luca Livraghi, Florian Clatot, Rinat Yerushalmi, Carmine De Angelis, Rodrigo Sánchez-Bayona, Icro Meattini, Natalia Cichowska-Cwalińska, Martine Berlière, Mahmoud Salama, Ugo De Giorgi, Amir Sonnenblick, Hatem Azim, Ann Partridge. Pregnancy after breast cancer in young women with germline BRCA pathogenic variants: results from an international cohort study [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr GS02-13.

AbstractBackground and AimEffective clinical event classification is essential for clinical research and quality improvement. The validation of artificial intelligence (AI) models like Generative Pre‐trained Transformer 4 (GPT‐4) for this task and comparison with conventional methods remains unexplored.MethodsWe evaluated the performance of the GPT‐4 model for classifying gastrointestinal (GI) bleeding episodes from 200 medical discharge summaries and compared the results with human review and an International Classification of Diseases (ICD) code‐based system. The analysis included accuracy, sensitivity, and specificity evaluation, using ground truth determined by physician reviewers.ResultsGPT‐4 exhibited an accuracy of 94.4% in identifying GI bleeding occurrences, outperforming ICD codes (accuracy 63.5%, P < 0.001). GPT‐4's accuracy was either slightly lower or statistically similar to individual human reviewers (Reviewer 1: 98.5%, P < 0.001; Reviewer 2: 90.8%, P = 0.170). For location classification, GPT‐4 achieved accuracies of 81.7% and 83.5% for confirmed and probable GI bleeding locations, respectively, with figures that were either slightly lower or comparable with those of human reviewers. GPT‐4 was highly efficient, analyzing the dataset in 12.7 min at a cost of 21.2 USD, whereas human reviewers required 8–9 h each.ConclusionOur study indicates GPT‐4 offers a reliable, cost‐efficient, and faster alternative to current clinical event classification methods, outperforming the conventional ICD coding system and performing comparably to individual expert human reviewers. Its implementation could facilitate more accurate and granular clinical research and quality audits. Future research should explore scalability, prompt and model tuning, and ethical implications of high‐performance AI models in clinical data processing.