Namdari, Surena

Purpose To evaluate the extent to which experienced reviewers can accurately discern between artificial intelligence (AI)–generated and original research abstracts published in the field of shoulder and elbow surgery and compare this with the performance of an AI detection tool. Methods Twenty‐five shoulder‐ and elbow‐related articles published in high‐impact journals in 2023 were randomly selected. ChatGPT was prompted with only the abstract title to create an AI‐generated version of each abstract. The resulting 50 abstracts were randomly distributed to and evaluated by 8 blinded peer reviewers with at least 5 years of experience. Reviewers were tasked with distinguishing between original and AI‐generated text. A Likert scale assessed reviewer confidence for each interpretation, and the primary reason guiding assessment of generated text was collected. AI output detector (0%‐100%) and plagiarism (0%‐100%) scores were evaluated using GPTZero. Results Reviewers correctly identified 62% of AI‐generated abstracts and misclassified 38% of original abstracts as being AI generated. GPTZero reported a significantly higher probability of AI output among generated abstracts (median, 56%; interquartile range [IQR], 51%‐77%) compared with original abstracts (median, 10%; IQR, 4%‐37%; P < .01). Generated abstracts scored significantly lower on the plagiarism detector (median, 7%; IQR, 5%‐14%) relative to original abstracts (median, 82%; IQR, 72%‐92%; P < .01). Correct identification of AI‐generated abstracts was predominately attributed to the presence of unrealistic data/values. The primary reason for misidentifying original abstracts as AI was attributed to writing style. Conclusions Experienced reviewers faced difficulties in distinguishing between human and AI‐generated research content within shoulder and elbow surgery. The presence of unrealistic data facilitated correct identification of AI abstracts, whereas misidentification of original abstracts was often ascribed to writing style. Clinical Relevance With rapidly increasing AI advancements, it is paramount that ethical standards of scientific reporting are upheld. It is therefore helpful to understand the ability of reviewers to identify AI‐generated content.

ABSTRACT Musculoskeletal infections (MSKI) remain the bane of orthopedic surgery, and result in grievous illness and inordinate costs that threaten healthcare systems. As prevention, diagnosis, and treatment has remained largely unchanged over the last 50 years, a 2nd International Consensus Meeting on Musculoskeletal Infection (ICM 2018, https://icmphilly.com ) was completed. Questions pertaining to all areas of MSKI were extensively researched to prepare recommendations, which were discussed and voted on by the delegates using the Delphi methodology. The questions, including the General Assembly (GA) results, have been published ( GA questions ). However, as critical outcomes include: (i) incidence and cost data that substantiate the problems, and (ii) establishment of research priorities; an ICM 2018 research workgroup (RW) was assembled to accomplish these tasks. Here, we present the result of the RW consensus on the current and projected incidence of infection, and the costs per patient, for all orthopedic subspecialties, which range from 0.1% to 30%, and $17,000 to $150,000. The RW also identified the most important research questions. The Delphi methodology was utilized to initially derive four objective criteria to define a subset of the 164 GA questions that are high priority for future research. Thirty‐eight questions (23% of all GA questions) achieved the requisite > 70% agreement vote, and are highlighted in this Consensus article within six thematic categories: acute versus chronic infection, host immunity, antibiotics, diagnosis, research caveats, and modifiable factors. Finally, the RW emphasizes that without appropriate funding to address these high priority research questions, a 3rd ICM on MSKI to address similar issues at greater cost is inevitable.