Alfasly, Saghir

Integrating artificial intelligence (AI) with healthcare data is rapidly transforming medical diagnostics and driving progress toward precision medicine. However, effectively leveraging multimodal data, particularly digital pathology whole slide images (WSIs) and genomic sequencing, remains a significant challenge due to the intrinsic heterogeneity of these modalities and the need for scalable and interpretable frameworks. Existing diagnostic models typically operate on unimodal data, overlooking critical cross-modal interactions that can yield richer clinical insights. We introduce MarbliX (Multimodal Association and Retrieval with Binary Latent Indexed matriX), a self-supervised framework that learns to embed WSIs and immunogenomic profiles into compact, scalable binary codes, termed “monogram.” By optimizing a triplet contrastive objective across modalities, MarbliX captures high-resolution patient similarity in a unified latent space, enabling efficient retrieval of clinically relevant cases and facilitating case-based reasoning. In lung cancer, MarbliX achieves 85%–89% across all evaluation metrics, outperforming histopathology (69%–71%) and immunogenomics (73%–76%). In kidney cancer, real-valued monograms yield the strongest performance (F1: 80%–83%, Accuracy: 87%–90%), with binary monograms slightly lower (F1: 78%–82%).

Abstract Cutaneous squamous cell carcinoma (cSCC) poses significant clinical challenges due to its rising incidence and potential for metastasis. Histopathologic risk stratification is further limited by substantial inter-observer variability. Unsupervised AI approaches based on content-based image retrieval offer scalable and interpretable decision support for diagnostic pathology. The objective of this study was to evaluate the use of image retrieval within histopathology atlases to stratify cSCC tumor differentiation from whole-slide images (WSIs), while comparing different patch selection and feature extraction strategies. This retrospective study included 552 archived WSIs comprising 385 well-differentiated, 102 moderately differentiated, and 66 poorly differentiated cases collected across Mayo Clinic sites in Arizona, Florida, and Minnesota. Image atlases were constructed using multiple patch aggregation strategies (Mosaic, Collage, and Montage) and deep learning encoders (KimiaNet, PathDino, and H-Optimus-0). A leave-one-WSI-out evaluation framework was used to assess differentiation classification performance using accuracy, specificity, sensitivity, and F1 score. Mosaic combined with KimiaNet achieved the highest Top-1 accuracy (74.9%) and specificity (92.6%), while Mosaic with H-Optimus-0 yielded the best Top-5 accuracy (79.0%) and macro-F1 score (62.6%). Collage combined with KimiaNet produced the highest Top-5 specificity (99.5%). The generalizability of the evaluated AI models varied across hospitals, reflecting differences in imaging protocols, staining practices, and patient populations. Overall, unsupervised image search and retrieval provides effective, annotation-free support for cSCC differentiation and has the potential to enhance dermatopathology workflows when appropriate combinations of patch selection and feature ex-traction methods are employed.