Advancing Computational Pathology with Multimodal Deep Learning

Histopathological assessment of tissue biopsies remains the cornerstone of cancer diagnosis, yet it is limited by diagnostic subjectivity and an incomplete view of the molecular heterogeneity that drives outcomes. This dissertation presents a suite of artificial intelligence (AI) frameworks that integrate deep learning, computer vision, and multimodal data to enhance diagnostic precision, prognostication, and therapeutic guidance across multiple cancer settings. The research first establishes a method for improving classification robustness through HistopathAI, a framework employing supervised contrastive learning and a hybrid deep feature fusion architecture. This approach yields more discriminative and generalizable feature representations, particularly in the context of class imbalance common in medical datasets. Building on this, the work demonstrates the power of integrating visually interpretable features from the tumor microenvironment. An AI-driven biomarker for Head and Neck Squamous Cell Carcinoma (HNSCC), which quantifies the spatial architecture of immune and stromal cells, was developed, and its prognostic value was validated on independent patient cohorts, confirming its strong potential for clinical risk stratification. A central contribution is the development of “virtual spatial transcriptomics” that bridges histology and genomics. The BrSTNet framework predicts spatial gene expression from routine H&E images in breast cancer, establishing a strong benchmark for image-to-transcriptome inference. This capability is translated to a direct clinical application in prostate cancer with ST-DoxPCa, a predictive signature for docetaxel benefit. The validation strategy was explicitly tailored to disease context: for the RTOG0521 trial (localized disease), a prognostic model trained independently in the Cleveland Clinic prostatectomy cohort (CCF) was applied in a fully blinded manner, without refitting to stratify patients and assess treatment effect; for the CHAARTED trial (metastatic disease), a separate treatment-response model was trained exclusively on the standard-of-care (ADT-only) arm. In both settings, ST-DoxPCa stratified patients into subgroups that either derived a significant survival benefit from docetaxel or showed no advantage, supporting treatment de-escalation for likely non-responders. Finally, to address the critical challenge of data scarcity, the thesis introduces MR-PHE, a novel framework for zero-shot learning in histopathology. By leveraging vision-language models with a multi-resolution, prompt-guided approach, MR-PHE enables accurate classification without task-specific training data, often outperforming fully supervised methods and offering a scalable solution for rare diseases. Collectively, these contributions advance the frontier of computational pathology, delivering a suite of validated, AI-driven tools that translate complex tissue-based data into clinically actionable insights. The methodologies developed herein represent a significant step toward a new paradigm of precision oncology, where patient care is personalized based on a deeper, quantitative understanding of tumor biology derived from standard pathology slides.