Citations

Abstract Large Language Models (LLMs) demonstrate strong capabilities in healthcare applications. However, recruiting human evaluators to assess usability and effectiveness becomes challenging in the large-scale LLM-generated outcomes. Existing metrics are often outdated for evaluating LLM-based healthcare technologies. This paper proposes a Research through Evaluation (RtE) approach that can refine metrics and improve assessments of LLM-generated outputs. We deploy RtE in a retrospective comparative study of prostate cancer patient inquiries, comparing responses from human clinical care teams with those generated by a GPT-4-based in-basket bot. Through three rounds of co-evaluations with clinical professionals and an LLM-as-graders study with four LLMs, the RtE method shows that starting with small, iterative evaluations helps human graders adjust metrics to suit their daily practices better. Additionally, in the LLM-as-graders study, LLMs exhibit similar behavior, preferring the in-basket bot's responses over the clinical care team. Our methodology and findings demonstrate that the RtE method can enhance the transparency and robustness of evaluation metrics for LLM-generated content, with the potential to generalize to evaluations in other clinical practices.

Importance Currently, there are no standardized outcome domains or measures in clinical trials for facial aging. Heterogeneity in outcome domains and measurement instruments across clinical trials creates difficulty in directly comparing interventions, determining superior therapies, and developing high-quality meta-analyses. Objective To develop a core outcome set (COS) of essential domains to be reported in clinical trials evaluating the efficacy of interventions for facial aging. Evidence Review PubMed/Medline, Embase, Cochrane Central Register of Controlled Trials, and CINAHL were searched from September 2005 to September 2015. An updated search of the same databases was performed from September 2015 to February 2026. Studies were included if (1) they were randomized clinical trial or controlled clinical trial in design, (2) they assessed the efficacy or safety of an intervention for facial aging, (3) they were published in English, and (4) they involved human participants. Complementary sources, including patient interviews, were used to capture further relevant outcomes. Two rounds of Delphi surveys, followed by consensus meetings, were used to identify outcome domains considered most important by both patient and physician stakeholders. Findings The final COS consists of 6 outcome domains: (1) overall convenience of treatment; (2) time to return to normal work and social activity; (3) overall assessment of focused area of treatment (at the point in time when treatment is expected to provide peak benefit); (4) duration of treatment effect; (5) severity of persistent local or systemic adverse events, including pigmentary change, skin texture change, delayed healing, scarring, and serious adverse events; and (6) patient satisfaction with treatment. Conclusions and Relevance The 6 outcome domains identified through a Delphi consensus are recommended for reporting in future facial aging trials to ensure that outcomes that matter most to patients and clinicians are measured and that results are comparable across interventions.

Abstract Objective To evaluate otolaryngology (ENT) residency match trends before and after the USMLE Step 1 transition to pass/fail. Study Design Observational longitudinal analysis of annual, publicly available specialty‐level data (2010‐2025). Setting National Resident Matching Program (NRMP). Methods Data from 2010 to 2025 were used to calculate applicants per PGY‐1 position, fill rate, and nonmatch‐in‐specialty rate. ENT was compared with plastic surgery, orthopedic surgery, neurological surgery, and general surgery. We used ordinary least squares (OLS), segmented regression with a 2023 breakpoint, and difference‐in‐differences (DID) models. Results From 2010 to 2025, ENT programs increased from 105 to 139 and positions from 280 to 394. Applicants per position ranged from 1.06 to 1.60 (mean 1.38). OLS showed no significant long‐term trend in applicants per position (+0.002/year, P = .86), fill rate (+0.03 percentage points/year, P = .68), or nonmatch‐in‐specialty rate (+0.04 percentage points/year, P = .94). Segmented regression showed no significant post‐2023 changes in applicants per position (slope change +0.03/year, P = .78), and no significant changes in applicant counts ( P = .7). ENT had the largest 2022 to 2023 decline in this cohort, in both applicants (−14.1%) and applicants per position (−16.9%). However, in two‐way fixed‐effects DID models, there was no ENT‐specific post‐transition change in applicants per position ( P > .05 in primary and sensitivity analyses). Conclusion ENT remained highly competitive but trend‐stable from 2010 to 2025. Although 2023 showed a notable drop in applicants, we found no consistent evidence of a sustained ENT‐specific competitiveness shift after the Step 1 pass/fail transition. Level of Evidence 4.

Abstract Therapeutic options for Waldenström macroglobulinemia (WM) have expanded but improvements in overall-survival (OS) are difficult to interpret due to competing non–WM mortality. We conducted a large single-center retrospective study of 1,020 patients with WM to evaluate survival trends. Patients were stratified by era of active WM diagnosis using a two-era model (1996–2005 and post-2005) and a four-era model (pre-2001, 2001–2007, 2008–2013, and post-2013). OS did not differ significantly across eras in the overall cohort (p = 0.64 and p = 0.86 for the two- and four-era models). However, among patients aged < 65 years at the time of active WM diagnosis in the 2-era model, OS improved in the post-2006 era (median 18.8 vs 14.1 years, p = 0.018). Cause-specific-survival (CSS) improved across all eras, increasing from 15.6 years in 1996–2005 to 18.9 years post-2006 (p < 0.0001), and from 13.7 years pre-2001 to not reached in later periods in the four-era model (p < 0.0001). Competing-risk-analysis (CRA) demonstrated a reduction in WM-related mortality over time (p < 0.001). However, excess mortality persisted, with a standardized mortality ratio (SMR) of 2.79 (p < 0.001) among patients < 65 years compared with a matched U.S. population. Our findings highlight improvements in WM-specific survival, while gains in OS are observed in younger patients.

Abstract Mild traumatic brain injury (mTBI) commonly induces transient acute (APTH) or persistent (PPTH) post-traumatic headache (PTH) that often resembles migraine. As orexin B sensitizes male but not female murine, nonhuman primate, and human dorsal root ganglion neurons and supradural orexin B/orexin receptor 2 (OX2R) signaling elicits migraine-like pain in naïve male, but not female, mice we explored possible sexually dimorphic contributions of orexin B/OX2R to PTH. In mice of both sexes, mTBI-induced transient cephalic allodynia, a surrogate measure of APTH. After APTH resolution, allodynia was reinstated by exposure to normally innocuous stress or by inhalational delivery of a subthreshold concentration of umbellulone, a TRPA1 agonist, suggesting the expression of PPTH. In contrast to these nonselective stimuli, subthreshold supradural orexin B induced PPTH only in male mTBI mice. Intranasal delivery of a CRISPR/Cas9 plasmid to edit trigeminal OX2R expression prevented APTH and development of PPTH selectively in male mTBI mice. Daily oral suvorexant, a dual orexin receptor antagonist (DORA), beginning immediately after mTBI, prevented APTH as well as PPTH. Critically, starting suvorexant treatment after resolution of APTH also prevented stress- or umbellulone-induced PPTH. EEG/EMG-defined sleep architecture or immobility-defined sleep was not disrupted in this mTBI model suggesting that suvorexant benefits are unlikely related to sleep modulation. Our findings reveal a male-specific mechanism of PTH maintained by orexin B/OX2R signaling and suggest that approved DORAs may be beneficial in treating APTH and preventing transition to PPTH in men. Importantly, DORAs may also be effective in men with established PPTH.