Abdulnour, Raja-Elie E

Abstract Background Large language models (LLMs) are increasingly used in medication-related tasks despite limited evidence supporting their accuracy and safety. LLMs are vulnerable to adversarial attacks and may confabulate in response to fabricated information in the prompt. Given the substantial risk of harm from medication errors, the purpose of this study was to establish a benchmark task to evaluate LLM performance in distinguishing fabricated from real medications. Methods Two datasets (brand and generic), each consisting of 250 medication lists were developed, each including 4-6 medications and one fabricated medication (specifically a Pokémon character), with complete dosing information provided. Six LLMs were evaluated (GPT-5-Chat, GPT-4o-mini, DrugGPT, Gemma-3-27B-IT, Llama-3.3-70B-Instruct, and Qwen3-32B). For each medication list, models were queried for dosing information and disease indication across three experimental conditions: standard decoding, standard decoding with mitigation prompting, and deterministic decoding (temperature = 0). Each experiment was performed in triplicate. Outputs were deemed confabulations if the LLM failed to recognize and reject the fabricated medication. The primary outcome was the rate of confabulations. Exact paired-permutation tests were used to compare confabulation rates across LLMs and prompting approaches. Results Confabulation rates under standard conditions ranged from 2.7% to 99.6%. DrugGPT demonstrated the lowest baseline confabulation rates (2.7-6.4%), whereas other models demonstrated substantially higher rates (47.2-99.6%). Mitigation prompting significantly reduced confabulation rates compared to standard and deterministic decoding across all tasks and datasets (p < 0.001): GPT-5-Chat achieved confabulation rates of 0% with mitigation prompting from baseline rates of 66.4-88.5%. Deterministic decoding did not substantially impact confabulation rates. Conclusions LLMs performing medication-related tasks are susceptible to confabulation in response to invalid or fabricated inputs, raising important safety concerns for clinical use. While mitigation prompting reduced confabulation rates, this study underscores the importance of robust safeguards when deploying LLMs in medication-related decision-making.

Abstract Rationale Acute respiratory distress syndrome (ARDS) is a devastating illness with limited therapeutic options. A better understanding of early biochemical and immunological events in ARDS could inform the development of new preventive and treatment strategies. Objectives To determine select peripheral blood lipid mediator and leukocyte responses in patients at risk for ARDS. Methods Patients at risk for ARDS were randomized as part of a multicenter, double-blind clinical trial of aspirin versus placebo (the LIPS-A [Lung Injury Prevention Study with Aspirin] trial; NCT01504867). Plasma thromboxane B2 (TXB2), aspirin-triggered lipoxin A4 (15-epi-LXA4, ATL), and peripheral blood leukocyte number and activation were determined on enrollment and after treatment with either aspirin or placebo. Measurements and Main Results Thirty-three of 367 subjects (9.0%) developed ARDS after randomization. Baseline ATL levels, total monocyte counts, intermediate monocyte counts, and monocyte–platelet aggregates were associated with the development of ARDS. Peripheral blood neutrophil count and monocyte–platelet aggregates significantly decreased over time. Of note, nine subjects developed ARDS after randomization yet before study drug initiation, including seven subjects assigned to aspirin treatment. Subjects without ARDS at the time of first dose demonstrated a lower incidence of ARDS with aspirin treatment. Compared with placebo, aspirin significantly decreased TXB2 and increased the ATL/TXB2 ratio. Conclusions Biomarkers of intravascular monocyte activation in at-risk patients were associated with development of ARDS. The potential clinical benefit of early aspirin for prevention of ARDS remains uncertain. Together, results of the biochemical and immunological analyses provide a window into the early pathogenesis of human ARDS and represent potential vascular biomarkers of ARDS risk. Clinical trial registered with www.clinicaltrials.gov (NCT01504867).