Cheung, Joseph

Study objectives To evaluate and compare the performance of two proprietary frontier large language models (LLMs), ChatGPT-5 and Grok-4, on diagnostic reasoning and foundational knowledge tasks within the specialty domain of sleep medicine. Methods The models were evaluated on two tasks: case-based reasoning using 79 clinical vignettes from the AASM Case Book of Sleep Medicine and knowledge assessment using 897 multiple-choice questions (MCQs) from board review materials. For vignettes, final diagnosis was scored by concept-level exact match, and differential diagnosis (DDx) was scored on a fixed top-5 output using concept-level matching with synonym normalization to compute precision, recall, and F 1-score. MCQ performance was the proportion correct. Inter-model performance was compared using the Mann–Whitney U test. Results Both models achieved high accuracy for final diagnosis (92.4% for both; 95% CI 86.4, 98.4) and MCQs (ChatGPT-5: 93.0%; Grok-4: 92.8%). However, performance on generating a comprehensive differential diagnosis was suboptimal, with modest F 1-scores for both ChatGPT-5 (0.55 ± 0.20) and Grok-4 (0.59 ± 0.20). There were no statistically significant differences in performance between the two models across any metric ( p > 0.05). Conclusions Frontier LLMs demonstrated high accuracy in sleep medicine tasks requiring knowledge recall and direct pattern recognition but showed more limited performance in complex clinical reasoning tasks such as generating a comprehensive differential diagnosis. These findings suggest that current general-purpose models may be more reliable for focused knowledge support than for broad hypothesis generation. Future studies should evaluate whether domain-adapted models or clinician-in-the-loop workflows can improve real-world diagnostic usefulness and safety.

Abstract Objective Evaluate how RAG architecture, including corpus structure, retrieval strategy, and pipeline complexity, affects LLM-based medical problem solving and knowledge retrieval in sleep medicine. Materials and Methods We benchmarked four open-source LLMs (Llama-3-8B, Llama -3 -70B, Qwen 2.5-14B, and Qwen 2.5-235B) using a knowledge base of five sleep medicine textbooks. We compared performance across three dimensions: corpus structure (raw text vs table-of-contents aligned.), retrieval strategy (dense embedding vs hybrid sparse-dense), and pipeline complexity (baseline vs augmented). Evaluation metrics included board-style multiple choice question (MCQ) accuracy and clinical case vignette diagnostic ranking. Results RAG improved MCQ accuracy for all models. Llama-8B saw the largest gain of 10.6% (61.8% to 72.4%), while Qwen-235B reached 87.3%. In clinical cases, Llama-8B accuracy dropped by 7.1% when using raw text and dense retrieval due to context noise. This was corrected by using structured hybrid configurations. Hybrid retrieval consistently outperformed dense-only methods. Overall, structured corpora improved primary diagnosis accuracy by 6.1% on average, with Qwen-235B reaching a peak 10.2% increase. Discussion RAG effectiveness depends on the balance between model size and data structure. Large models handle uncurated text well, but smaller models are easily distracted by irrelevant data. Hybrid retrieval is necessary to maintain precision with specialized medical terms. A structured corpus paired with a baseline hybrid pipeline offers the best stability and speed for clinical use. Conclusion Rigorous data curation and hybrid retrieval are as essential as model scale for deploying safe, guideline-compliant AI in sleep medicine.

Abstract Objectives Patients with hypermobile Ehlers–Danlos syndrome (hEDS) and hypermobility spectrum disorders (HSD) experience a wide array of symptoms and system disorders. This study aimed to identify whether differences occurred in 115 self-reported symptoms and comorbidities in patients diagnosed with hEDS or HSD. Methods In this study we analysed self-reported data from an EDS Clinic intake questionnaire in patients diagnosed with hEDS, HSD or no hypermobile conditions. Results From 1 November 2019 to 7 March 2024, the EDS Clinic saw 2088 patients. Using the 2017 diagnostic criteria, 66.5% were diagnosed with HSD (n = 1389), 20.3% with hEDS (n = 423), 10.6% with historic HSD (H-HSD) or localized HSD (L-HSD) (n = 256) (hypermobile controls) and 2.6% were not diagnosed with hEDS, HSD, H-HSD or L-HSD (n = 55) (controls). Symptoms/comorbidities that occurred with high prevalence in both hEDS and HSD included joint pain (hEDS 82.0%, HSD 88.9%), allergy (hEDS 77.0%, HSD 77.0%), subluxations (hEDS 71.2%, HSD 72.6%), brain fog (hEDS 70.0%, HSD 74.7%), headache (hEDS 68.1%, HSD 69.1%), anxiety (hEDS 60.3%, HSD 69.3%), depression (hEDS 52.2%, HSD 58.0%), migraine (hEDS 53.7%, HSD 52.5%), nausea (hEDS 54.6%, HSD 59.5%) and constipation (hEDS 53.0%, HSD 57.2%). In contrast, 9/115 (8%) symptoms/comorbidities were self-reported significantly more often in hEDS but 42/115 (37%) in HSD. hEDS patients reported more symptoms that suggest a defect in collagen, such as dislocation, hernias and rectal prolapse, while HSD patients reported more joint, muscle, allergy, neurological, gastrointestinal, sleep and psychological symptoms/comorbidities. Conclusion Although we found an overlap in some symptoms and comorbidities self-reported by hEDS/HSD patients, such as allergy/atopy, headache/migraine and gastrointestinal symptoms, our findings suggest key differences exist between the two diagnoses, suggesting that hEDS and HSD may be distinct conditions.

Background Postprandial sleepiness refers to the state of excessive drowsiness that occurs after consuming a meal. This transient phenomenon is experienced by many individuals, often leading to decreased productivity and impaired cognitive performance. This study examined the effect of a whole-food plant-based (WFPB) diet on postprandial sleepiness. Methods Ten participants who had postprandial sleepiness who were on a typical American diet transitioned to a WFPB dietary pattern for 21 days. Postprandial sleepiness was assessed using the Epworth Sleepiness Scale (ESS) questionnaire, focusing on Q7 “sitting quietly after lunch without alcohol” as well as Q5 “lying down to rest in the afternoon when circumstances permit.” Results The participants had a mean age of 44 +/− 15 years, with 30% being male. There was a statistically significant reduction in sleepiness measured by ESS Q7 (median change = −1, P = .012) and ESS Q5 (median change = −1.5, P = .008). Moreover, there was a statistically significant decrease in overall ESS scores (median change = −8.0, P = .006), and weight (median change = −2.9 kg, P = .006). Conclusion Participants demonstrated significant improvement in postprandial sleepiness-related situations. Our findings showed that the WFPB diet may have potential benefits for reducing postprandial and overall daytime sleepiness.

Objectives: To assess the prevalence of sleep disturbance among patients evaluated at a clinic for patients afflicted with Post-acute sequelae of COVID-19 (PASC). Methods: Sleep disturbance was assessed with the Patient-Reported Outcomes Measurement Information System-Sleep Disturbance (PROMIS-SD) framework among adult patients of the PASC clinic. Results: Among 312 patients, the mean age was 46.2 years, and 70.2% were women. About 41.0% of patients had no sleep disturbance; sleep disturbance was mild to moderate in 51.3% and severe in 7.7%. PROMIS-SD score was negatively correlated with the time from the initial positive COVID-19 test to the initial consultation in the PASC clinic (Pearson r = −.094; r 2 = .0088). Conclusions: The PROMIS-Sleep Disturbance framework can serve as a tool to assess the burden of sleep disturbances in PASC patients.

Abstract In patients with breast hypertrophy, excessive breast weight applies pressure on the thorax, which may disrupt the normal breathing. The purpose of this study is to evaluate the impact of the breast hypertrophy and reduction mammoplasty on respiratory function. A comprehensive search of 3 databases, PubMed, Ovid, and Scopus databases, was performed. “Mammoplasty” and “respiration or pulmonary function tests” were the keywords used to search for relevant articles. Ten studies involving 280 patients with breast hypertrophy were included in the final review. Seven articles demonstrated an increase in at least 1 pulmonary function test value after the surgery. This systematic review revealed that, preoperatively, pulmonary function test values of the patients are usually in the normal range. Nonetheless, reduction mammoplasty still improves lung function parameters. Additionally, patients with respiratory complaints felt improvement in their symptoms after the surgery. However, future studies are needed, as heterogeneity among studies was observed.