Citations

Abstract Error analysis is a critical step in evaluating and improving clinical concept extraction models, the most common clinical natural language processing (NLP) task. Unlike corpus annotation, which follows standardized protocols for creating gold-standard datasets, error analysis requires nuanced judgment grounded in both clinical expertise and NLP knowledge. This is especially important given the heterogeneity of clinical text, where variations in documentation style, note structure, and terminology can substantially influence model behavior. Despite its importance, there is currently no standardized, user-level framework to support systematic error analysis in clinical concept extraction task. In this study, we developed and validated MedError, a machine-assisted, human-in-the-loop framework designed to standardize and enhance error analysis for clinical concept extraction tasks. We collected and manually curated a corpus of 1,187 unique errors from a total of 4,237 notes across three different distinct hospitals. The error categories were defined using our previously validated error taxonomy and included 480 false negatives and 707 false positives across 25 error types and 48 clinical concept categories. We evaluated the performance of three proprietary and three open-source large language models (LLMs) in automatically classifying these errors into 26 and 15 predefined categories. We further developed a machine-assisted framework, MedError, which integrates best practices in error analysis, LLM-assisted classification and reasoning, and a user-friendly interface to enable more efficient, reproducible, and context-aware error analysis. The framework supports both single-site and federated multisite error analysis, facilitating the effective translation of clinical NLP systems into real-world settings.

Abstract Motivation Multiplex immunofluorescence (MxIF) enables the quantification of multiple protein markers at a single-cell level while preserving spatial information, offering a powerful tool for studying tissue microenvironments. However, the flexibility in MxIF panel design poses challenges in standardizing cell phenotyping. Results We present SpaFlow, an efficient, customizable pipeline for unsupervised clustering and classification of MxIF data, implemented using Nextflow. SpaFlow performs quality control, clustering, and postclustering analysis on segmented and quantified MxIF data, facilitating reproducible and scalable analyses across various computing platforms. The SpaFlow pipeline integrates three clustering and classification packages—Seurat, SCIMAP, and CELESTA—each providing unique methodologies for identifying cell types based on phenotypic markers. A novel “meta-clustering” approach condenses clusters across multiple regions of interest into common meta-clusters, streamlining the cell-type identification process in large datasets. SpaFlow’s robust quality control steps, including signal summation and cell density filtering, mitigate artifacts that may impact clustering accuracy. We demonstrate the utility of SpaFlow in a case study involving 297 ovarian tumor cores, where SpaFlow successfully identified biologically meaningful cell populations, including tumor-infiltrating lymphocytes, efficiently and rapidly. Additionally, SpaFlow’s reproducibility is validated using serial tonsil sections, confirming its capability to consistently identify distinctive cell populations across matched ROIs. Availability and implementation SpaFlow is freely available with detailed documentation and examples at https://github.com/dimi-lab/SpaFlow.

Background: The incidence of pediatric thyroid cancer has been increasing, and care varies due to socioeconomic disparities or differing practice patterns. Clinical guidelines call for care in multidisciplinary teams to minimize variance and provide protocols. Based on expert opinion, we hope to describe the form and function of such multidisciplinary teams for pediatric thyroid programs. Methods: A modified Delphi method to reach consensus statements over two rounds. Twenty-one experts with varying backgrounds responded to each statement on a 9-point Likert scale. Upon completion of the survey, the panel reviewed and shared the results and comments from participants and modified the statements accordingly. This process was repeated such that statements reached consensus, were deemed no consensus, or had no change in the mean. Results: There was an 88% and 83% completion rate for Rounds 1 and 2, respectively. A consensus was observed that there is a distinct definable model of care for pediatric thyroid patients. No consensus was reached for the age range of patients, but programs should care for children with medullary thyroid cancer, differentiated thyroid cancer, and patients with genetic predisposition syndromes. A comprehensive team includes, but is not limited to, a thyroid surgeon, a pediatric endocrinologist, a high-volume fine-needle aspiration (FNA) proceduralist, an oncologist, a nuclear medicine physician, a pediatric pathologist, a pediatric radiologist, and a nurse coordinator. Necessary support services involve care coordination, access to a multidisciplinary tumor board, ability to perform ultrasound-guided FNA, and access to molecular testing. The panel emphasized cross-institutional collaborative research prioritizing guidelines development, disease-specific outcomes, treatment toxicity, and the molecular landscape of thyroid cancer. Conclusions: These consensus statements can be beneficial in improving multidisciplinary care, by describing which elements of pediatric thyroid programs should be consistent across institutions. Overall, the panel agreed that pediatric thyroid centers should provide integrated care with defined team members, services, resources, and research priorities. This model has the potential to standardize various aspects of clinical care and enhance our ability to study patient outcomes, improve health care delivery, and increase scholarly collaboration.

Abstract Multiplexed Immunofluorescence (MxIF) enables detailed immune cell phenotyping, providing critical insights into cell behavior within the tumor immune microenvironment (TIME). However, signal integrity can be compromised due to the complex cyclic staining processes inherent to MxIF. Hematoxylin and Eosin (H&E) staining, on the other hand, offers complementary information through its depiction of cell morphology and texture patterns and is often visually cross-referenced with MxIF in clinical settings. In this study, we proposed a novel framework to align H&E and MxIF images for precise cross-modal cell feature validation. Using cell detection outputs from each modality as anchors, we formulated the multimodal image registration problem as point set alignment. Coherent Point Drift (CPD) is employed for initial alignment, followed by Graph Matching (GM) for refinement. Evaluations on ovarian cancer tissue microarrays (TMAs) demonstrate that our method achieves high alignment accuracy, enabling reliable validation of cell-level features across modalities for both restained and serial sections. Our results indicate that restained H&E enhances confidence in findings derived from MxIF. Additionally, we demonstrated the feasibility of generating high-quality virtual H&E images from MxIF data when restained H&E is unavailable, offering a viable alternative for integrated multimodal analysis.

Abstract Advances in genetic technology have led to the increasing use of genomic panels in precision oncology practice, with panels ranging from a couple to hundreds of genes. However, the clinical utilization and utility of oncology genomic panels, especially among vulnerable populations, is unclear. We examined the association of panel size with socioeconomic status and clinical trial matching. We retrospectively identified 9,886 eligible adult subjects in the Mayo Clinic Health System who underwent genomic testing between January 1, 2016 and June 30, 2020. Patient data were retrieved from structured and unstructured data sources of institutional collections, including cancer registries, clinical data warehouses, and clinical notes. Socioeconomic surrogates were approximated using the Area Deprivation Index (ADI) corresponding to primary residence addresses. Logistic regression was performed to analyze relationships between ADI or rural/urban status and (i) use of genomic test by panel size; (ii) clinical trial matching status. Compared with patients from the most affluent areas, patients had a lower odds of receiving a panel test (vs. a single-gene test) if from areas of higher socioeconomic deprivation [OR (95% confidence interval (CI): 0.71 (0.61–0.83), P < 0.01] or a rural area [OR (95% CI): 0.85 (0.76–0.96), P < 0.01]. Patients in areas of higher socioeconomic deprivation were less likely to be matched to clinical trials if receiving medium panel tests [(OR) (95% CI): 0.69 (0.49–0.97), P = 0.03]; however, there was no difference among patients receiving large panel tests (P > 0.05) and rural patients were almost 2x greater odds of being matched if receiving a large panel test [(OR) (95% CI): 1.76 (1.21–2.55), P < 0.01]. Significance: We identified socioeconomic and rurality disparities in the use of genomic tests and trial matching by panel size, which may have implications for equal access to targeted therapies. The lack of association between large panel tests and clinical trial matching by socioeconomic status, suggests a potential health equity impact, while removing barriers in access to large panels for rural patients may improve access to trials. However, further research is needed.

Multiplex immunofluorescence (MxIF) images provide detailed information of cell composition and spatial context for biomedical research. However, compromised data quality could lead to research biases. Comprehensive image quality checking (QC) is essential for reliable downstream analysis. As a reliable and specific staining of cell nuclei, 4′,6-diamidino-2-phenylindole (DAPI) signals were used as references for tissue localization and auto-focusing across MxIF staining–scanning–bleaching iterations and could potentially be reused for QC. To confirm the feasibility of using DAPI as QC reference, pixel-level DAPI values were extracted to calculate signal fluctuations and tissue content similarities in staining–scanning–bleaching iterations for identifying quality issues. Concordance between automatic quantification and human experts’ annotations were evaluated on a data set consisting of 348 fields of view (FOVs) with 45 immune and tumor cell markers. Cell distribution differences between subsets of QC-pass vs QC-failed FOVs were compared to investigate the downstream effects. Results showed that 87.3% FOVs with tissue damage and 73.4% of artifacts were identified. QC-failed FOVs showed elevated regional gathering in cellular feature space compared with the QC-pass FOVs. Our results supported that DAPI signals could be used as references for MxIF image QC, and low-quality FOVs identified by our method must be cautiously considered for downstream analyses.

ImportanceCytokine storm due to COVID-19 can cause high morbidity and mortality and may be more common in patients with cancer treated with immunotherapy (IO) due to immune system activation.ObjectiveTo determine the association of baseline immunosuppression and/or IO-based therapies with COVID-19 severity and cytokine storm in patients with cancer.Design, Setting, and ParticipantsThis registry-based retrospective cohort study included 12 046 patients reported to the COVID-19 and Cancer Consortium (CCC19) registry from March 2020 to May 2022. The CCC19 registry is a centralized international multi-institutional registry of patients with COVID-19 with a current or past diagnosis of cancer. Records analyzed included patients with active or previous cancer who had a laboratory-confirmed infection with SARS-CoV-2 by polymerase chain reaction and/or serologic findings.ExposuresImmunosuppression due to therapy; systemic anticancer therapy (IO or non-IO).Main Outcomes and MeasuresThe primary outcome was a 5-level ordinal scale of COVID-19 severity: no complications; hospitalized without requiring oxygen; hospitalized and required oxygen; intensive care unit admission and/or mechanical ventilation; death. The secondary outcome was the occurrence of cytokine storm.ResultsThe median age of the entire cohort was 65 years (interquartile range [IQR], 54-74) years and 6359 patients were female (52.8%) and 6598 (54.8%) were non-Hispanic White. A total of 599 (5.0%) patients received IO, whereas 4327 (35.9%) received non-IO systemic anticancer therapies, and 7120 (59.1%) did not receive any antineoplastic regimen within 3 months prior to COVID-19 diagnosis. Although no difference in COVID-19 severity and cytokine storm was found in the IO group compared with the untreated group in the total cohort (adjusted odds ratio [aOR], 0.80; 95% CI, 0.56-1.13, and aOR, 0.89; 95% CI, 0.41-1.93, respectively), patients with baseline immunosuppression treated with IO (vs untreated) had worse COVID-19 severity and cytokine storm (aOR, 3.33; 95% CI, 1.38-8.01, and aOR, 4.41; 95% CI, 1.71-11.38, respectively). Patients with immunosuppression receiving non-IO therapies (vs untreated) also had worse COVID-19 severity (aOR, 1.79; 95% CI, 1.36-2.35) and cytokine storm (aOR, 2.32; 95% CI, 1.42-3.79).Conclusions and RelevanceThis cohort study found that in patients with cancer and COVID-19, administration of systemic anticancer therapies, especially IO, in the context of baseline immunosuppression was associated with severe clinical outcomes and the development of cytokine storm.Trial RegistrationClinicalTrials.gov Identifier: NCT04354701

e18790 Background: Patients with SARS-CoV-2 with a diagnosis of cancer have increased risk of severe COVID-19 outcomes compared to patients without cancer. However, little is known regarding outcomes of patients with COVID-19 and cancer in the setting of human immunodeficiency virus (HIV). Given the unique risks of this population, we sought to understand COVID-19 outcomes using registry data. Methods: This is a descriptive research study utilizing the CCC19 registry, an international multi-institutional registry with healthcare provider-reported cases of patients with cancer and COVID-19. Between March 2020-December 2021, 116 persons with HIV (PWH) and 10,642 persons without HIV (PWOH) with laboratory-confirmed SARS-CoV-2 infection were identified as eligible for the analysis. Results: Median follow-up time for both groups was 90 days, with interquartile range (IQR) 30-180 days. Most PWH were actively receiving antiretroviral therapy (ART) at the time of COVID-19 diagnosis, with 71% (n = 82) having named drug information available; bictegravir/emtricitabine/tenofovir was the most common ART (n = 25). PWH were of younger age (median 57.5 yrs [IQR 46.5-63.25] vs 65 yrs [IQR 55-74]), male (81% vs 47%), and either non-Hispanic Black or Hispanic (71% vs 34%) compared to PWOH. 12% of PWH (n = 14) were current smokers compared to 6% of PWOH (n = 638), and more than half in each group were never smokers (51% of PWH and 53% of PWOH). The following comorbidities were identified in PWH vs PWOH: cardiovascular (16% vs 20%), pulmonary (16% vs 20%), renal (15% vs 14%), and diabetes mellitus (18% vs 27%). A higher proportion of PWH had hematologic malignancy compared to PWOH (33% vs 19%). More PWH had active cancer which was progressing at the time of SARS-CoV-2 infection compared to PWOH (24% vs 14%). 44% of PWH (n = 51) had received active systemic anticancer therapy within the 3 months preceding SARS-CoV-2 infection (including cytotoxic, targeted, endocrine therapies, and immunotherapy) compared to 51% of PWOH (n = 5,420). PWH had an increased rate of hospitalization (58% vs 55%) compared to PWOH. Although a lower proportion of PWH required supplemental oxygen during hospitalization compared to PWOH (34% vs 38%) and ICU admission rates were identical between the two groups (16% vs 16%), PWH had an increased rate of mechanical ventilation (14% vs 10%) and death (24% vs 18%) compared to PWOH. Conclusions: This is the first known study describing outcomes of patients with cancer and COVID-19 in the PWH population from a large multinational dataset. PWH have characteristics associated with adverse outcomes in prior analyses (male sex, non-Hispanic Black or Hispanic, hematologic malignancy, progressing cancer) but are notably younger and have fewer comorbidities. HIV infection may portend increased risk of severe COVID-19 and death; however, additional analyses, including multivariable regression, are warranted.