Tang, Dennis M.

Abstract Objective To use an artificial intelligence (AI)‐powered large language model (LLM) to improve readability of patient handouts. Study Design Review of online material modified by AI. Setting Academic center. Methods Five handout materials obtained from the American Rhinologic Society (ARS) and the American Academy of Facial Plastic and Reconstructive Surgery websites were assessed using validated readability metrics. The handouts were inputted into OpenAI's ChatGPT‐4 after prompting: “Rewrite the following at a 6th‐grade reading level.” The understandability and actionability of both native and LLM‐revised versions were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Results were compared using Wilcoxon rank‐sum tests. Results The mean readability scores of the standard (ARS, American Academy of Facial Plastic and Reconstructive Surgery) materials corresponded to “difficult,” with reading categories ranging between high school and university grade levels. Conversely, the LLM‐revised handouts had an average seventh‐grade reading level. LLM‐revised handouts had better readability in nearly all metrics tested: Flesch‐Kincaid Reading Ease (70.8 vs 43.9; P < .05), Gunning Fog Score (10.2 vs 14.42; P < .05), Simple Measure of Gobbledygook (9.9 vs 13.1; P < .05), Coleman‐Liau (8.8 vs 12.6; P < .05), and Automated Readability Index (8.2 vs 10.7; P = .06). PEMAT scores were significantly higher in the LLM‐revised handouts for understandability (91 vs 74%; P < .05) with similar actionability (42 vs 34%; P = .15) when compared to the standard materials. Conclusion Patient‐facing handouts can be augmented by ChatGPT with simple prompting to tailor information with improved readability. This study demonstrates the utility of LLMs to aid in rewriting patient handouts and may serve as a tool to help optimize education materials. Level of Evidence Level VI.

AbstractObjectiveCritical components of the nasal endoscopic examination have not been definitively established for either the normal examination or for clinical disorders. This study aimed to identify concordance among rhinologists regarding the importance of examination findings for various nasal pathologies.Study DesignA consortium of 19 expert rhinologists across the United States was asked to rank the importance of findings on nasal endoscopy for 5 different sinonasal symptom presentations.SettingAn online questionnaire was distributed in July 2023.MethodsThe questionnaire utilized JotForm® software and featured 5 cases with a set of 4 identical questions per case, each covering a common indication for nasal endoscopy. Rankings were synthesized into Normalized Attention Scores (NASs) and Weighted Normalized Attention Scores (W‐NASs) to represent the perceived importance of each feature, scaled from 0 to 1.ResultsGeneral concordance was found for examination findings on nasal endoscopy within each case. The perceived features of importance differed between cases based on clinical presentation. For instance, in evaluating postnasal drip, the middle meatus was selected as the most important structure to examine (NAS, 0.73), with mucus selected as the most important abnormal finding (W‐NAS, 0.66). The primary feature of interest for mucus was whether it was purulent or not (W‐NAS, 0.67). Similar analyses were performed for features in each case.ConclusionThe implicit framework existing among rhinologists may help standardize examinations and improve diagnostic accuracy, augment the instruction of trainees, and inform the development of artificially intelligent algorithms to enhance clinical decision‐making during nasal endoscopy.