Berkenstock, Meghan

Ophthalmic immune-related adverse events (Eye-irAEs) from immune checkpoint inhibitors can cause visual morbidity. The absence of standardized definitions for Eye-irAEs not only impedes the development of evidence-based treatments but also progress in translational research. The objective of this study was to develop consensus guidance for an approach to Eye-irAEs. Four ophthalmic physicians (uveitis specialists and neuro-ophthalmologists) drafted Eye-irAE consensus guidance and definitions, which were reviewed by the multidisciplinary Eye-irAE definition panel. The panel was divided into Group A (Neuro-ophthalmology/Orbital Disease) and Group B (Uveitis/Ocular Surface Disease). A modified Delphi consensus process was used, with two rounds of anonymous ratings by panelists and two meetings to discuss areas of controversy. For each disorder, five diagnostic components were evaluated: symptoms, examination findings, laboratory studies/imaging findings, diagnostic criteria, and treatment. Panelists rated content for usability, appropriateness and accuracy on 9-point scales in electronic surveys and provided free-text comments. Aggregated survey responses were incorporated into revised definitions. Consensus was based on numeric ratings using the RAND Corporation/ University of California Los Angeles Health Services Utilization Study (RAND/UCLA) Appropriateness Method with prespecified definitions. 29 panelists from 25 academic medical centers voted on 114 rating scales (66 neuro-ophthalmic/orbital disease components, 48 uveitis/ocular surface disease components); of these, 86.3% (57/66) in Group A and 89.6% (43/48) in Group B reached first-round consensus. After revisions, all items except 6.1% (4/66) in Group A and 1.6% (1/60) in Group B received second-round consensus. Consensus definitions were achieved for 10/11 neuro-ophthalmic/orbital disorders: optic neuritis, inflammatory optic disc edema, arteritic ischemic optic neuropathy, optic perineuritis, orbital inflammation, thyroid eye disease-like orbital inflammation, cavernous sinus syndrome, oculomotor mononeuritis, trochlear mononeuritis, and abducens mononeuritis. Consensus definitions were achieved for 9/10 uveitis/ocular surface disorders: anterior uveitis, intermediate uveitis, posterior uveitis, panuveitis, Vogt-Koyanagi-Harada-like syndrome, sarcoidosis-like syndrome, acute macular neuroretinopathy, dry eye disease, and scleritis. These disease definitions establish a standardized classification for Eye-irAE, highlighting differences between irAEs and other inflammatory disorders. Importantly, diagnostic certainty does not always align directly with the need to treat as an Eye-irAE. Given the consensus from this representative panel group, it is anticipated the definitions will be used broadly across clinical and research settings.

The NCCN Guidelines for the Management of Immunotherapy-Related Toxicities are intended to provide oncology practitioners with guidance on how to manage the wide-ranging and potentially fatal toxicities that may occur with cancer immunotherapy. The guidelines address immune-related adverse events related to immune checkpoint inhibitors, CAR T-cell therapies, and lymphocyte engagers (which include T-cell–engaging bispecific antibodies). These NCCN Guidelines Insights highlight recent guideline updates pertaining to the management of emerging toxicities related to cancer immunotherapy.

Purpose: To analyze the accuracy and thoroughness of three large language models (LLMs) to produce information for providers about immune checkpoint inhibitor ocular toxicities. Methods: Eight questions were created about the general definition of checkpoint inhibitors, their mechanism of action, ocular toxicities, and toxicity management. All were inputted into ChatGPT 4.0, Bard, and LLaMA programs. Using the six-point Likert scale for accuracy and completeness, four ophthalmologists who routinely treat ocular toxicities of immunotherapy agents rated the LLMs answers. Analysis of variance testing was used to assess significant differences among the three LLMs and a post hoc pairwise t-test. Fleiss kappa values were calculated to account for interrater variability. Results: ChatGPT responses were rated with an average of 4.59 for accuracy and 4.09 for completeness; Bard answers were rated 4.59 and 4.19; LLaMA results were rated 4.38 and 4.03. The three LLMs did not significantly differ in accuracy (P = 0.47) nor completeness (P = 0.86). Fleiss kappa values were found to be poor for both accuracy (−0.03) and completeness (0.01). Conclusion: All three LLMs provided highly accurate and complete responses to questions centered on immune checkpoint inhibitor ocular toxicities and management. Further studies are needed to assess specific immune checkpoint inhibitor agents and the accuracy and completeness of updated versions of LLMs.

Purpose: The purpose of this study was to assess the accuracy and completeness of 3 large language models (LLMs) to generate information about antibody–drug conjugate (ADC)-associated ocular toxicities. Methods: There were 22 questions about ADCs, tisotumab vedotin, and mirvetuximab soravtansine that were developed and input into ChatGPT 4.0, Bard, and LLaMa. Answers were rated by 4 ocular toxicity experts using standardized 6-point Likert scales on accuracy and completeness. ANOVA tests were conducted for comparison between the 3 subgroups, followed by pairwise t-tests. Interrater variability was assessed with Fleiss kappa tests. Results: The mean accuracy score was 4.62 (SD 0.89) for ChatGPT, 4.77 (SD 0.90) for Bard, and 4.41 (SD 1.09) for LLaMA. Both ChatGPT (P = 0.03) and Bard (P = 0.003) scored significantly better for accuracy when compared with LLaMA. The mean completeness score was 4.43 (SD 0.91) for ChatGPT, 4.57 (SD 0.93) for Bard, and 4.42 (SD 0.99) for LLaMA. There were no significant differences in completeness scores between groups. Fleiss kappa assessment for interrater variability was good (0.74) for accuracy and fair (0.31) for completeness. Conclusions: All 3 LLMs had relatively high accuracy and completeness ratings, showing LLMs are able to provide sufficient answers for niche topics of ophthalmology. Our results indicate that ChatGPT and Bard may be slightly better at providing more accurate answers than LLaMA. As further research and treatment plans are developed for ADC-associated ocular toxicities, these LLMs should be reassessed to see if they provide complete and accurate answers that remain in line with current medical knowledge.

The aim of the NCCN Guidelines for Management of Immunotherapy-Related Toxicities is to provide guidance on the management of immune-related adverse events resulting from cancer immunotherapy. The NCCN Management of Immunotherapy-Related Toxicities Panel is an interdisciplinary group of representatives from NCCN Member Institutions, consisting of medical and hematologic oncologists with expertise across a wide range of disease sites, and experts from the areas of dermatology, gastroenterology, endocrinology, neurooncology, nephrology, cardio-oncology, ophthalmology, pulmonary medicine, and oncology nursing. The content featured in this issue is an excerpt of the recommendations for managing toxicities related to CAR T-cell therapies and a review of existing evidence. For the full version of the NCCN Guidelines, including recommendations for managing toxicities related to immune checkpoint inhibitors, visitNCCN.org.