Deep DiveMedicine & Health

Precision Oncology Goes Multimodal: When AI Integrates Omics, Pathology, and Radiology

A review in npj Digital Medicine examines how machine learning and deep learning integrate multi-omic, spatial pathology, and radiomic data to map tumor molecular pathways and guide treatment selectionโ€”moving precision oncology from genomic-only profiling toward genuinely multimodal decision-making.

By Sean K.S. Shin
This blog summarizes research trends based on published paper abstracts. Specific numbers or findings may contain inaccuracies. For scholarly rigor, always consult the original papers cited in each post.

Precision oncology began with a straightforward premise: sequence the tumor's genome, find actionable mutations, match them to targeted therapies. This approach has produced genuine clinical successesโ€”imatinib for BCR-ABL-positive chronic myeloid leukemia, trastuzumab for HER2-amplified breast cancer, EGFR inhibitors for EGFR-mutant lung cancer. But for the majority of cancer patients, genomic profiling alone does not identify an effective targeted therapy. Tumors are not defined solely by their mutations; they are shaped by gene expression, protein networks, metabolic states, spatial organization within tissue, and the immune microenvironment.

A 2025 review in npj Digital Medicine examines how artificial intelligenceโ€”specifically machine learning (ML) and deep learning (DL)โ€”is enabling the integration of these multiple data dimensions into unified models for understanding tumor biology and selecting treatments.

The Multimodal Data Landscape

The review describes how ML/DL methods integrate what the authors characterize as multi-dimensional, multi-omic, spatial pathology, and radiomic data. Each data type captures a different aspect of tumor biology.

Multi-omics encompasses genomics (DNA mutations, copy number alterations), transcriptomics (gene expression), proteomics (protein levels), epigenomics (methylation, chromatin accessibility), and metabolomics (metabolic profiles). Each layer provides partial information; their integration offers a more complete molecular portrait.

Spatial pathology refers to digitized histopathology images analyzed at cellular resolution, often combined with spatial transcriptomics or multiplexed immunofluorescence. These data reveal not just which cells are present but where they are relative to each otherโ€”the spatial relationship between tumor cells, immune cells, and stromal cells.

Radiomics extracts quantitative features from clinical imaging (CT, MRI, PET) that are invisible to the human eyeโ€”texture patterns, shape irregularities, intensity distributions that correlate with underlying tumor biology.

<
ClaimSourceVerification Status
ML/DL integrates multi-dimensional, multi-omic, spatial pathology, and radiomic datanpj Digital Medicine 2025 abstractStated in abstract
Purpose: understand tumor molecular pathways and optimize treatment selectionnpj Digital Medicine 2025 abstractStated in abstract
Moving precision oncology from genomic-only to genuinely multimodalnpj Digital Medicine 2025 abstractStated in abstract

Critical Analysis

Why Integration Is Hard

The conceptual case for multimodal integration is compelling: more data should yield better predictions. The technical reality is considerably more difficult. Each data modality has different dimensionality, noise characteristics, and missingness patterns. Genomic data consists of discrete variants across ~20,000 genes. Transcriptomic data is continuous and high-dimensional. Pathology images contain billions of pixels. Radiomic features are extracted through hand-crafted or learned algorithms with their own biases.

ML/DL models that combine these modalities must handle what is sometimes called the "curse of multimodality": each additional data type increases model complexity, data requirements, and the risk of overfitting. A model trained on 200 patients with complete multi-omic, pathology, and imaging data may capture dataset-specific patterns that do not generalize to new populations.

The Molecular Pathway Question

The review describes how AI helps understand tumor molecular pathwaysโ€”the signaling cascades and regulatory networks that drive tumor growth. This framing positions AI not merely as a pattern-matching tool (input: data; output: prediction) but as a means of biological discovery. Graph neural networks that model protein-protein interaction networks, attention mechanisms that identify which genomic features interact with which pathology features, and multimodal autoencoders that learn shared representations across data types all contribute to this goal.

The distinction matters because treatment selection based on understood mechanisms is more robust than treatment selection based on correlative prediction. If a model learns that a specific combination of genomic alterations, expression patterns, and spatial immune features predicts response to immunotherapy, and if the underlying biological mechanism is understood (e.g., the combination indicates a T-cell-inflamed microenvironment with intact antigen presentation), clinicians can have greater confidence in the prediction.

From Genomic-Only to Multimodal: Where Are We Really?

The review characterizes a transition from genomic-only precision oncology to genuinely multimodal approaches. This transition is real but uneven. In clinical practice today, most precision oncology decisions still rely on genomic panels (FoundationOne, Tempus, Guardant360) that test for known actionable mutations. Transcriptomic tests exist for specific applications (Oncotype DX for breast cancer recurrence risk, Decipher for prostate cancer). But truly multimodal AI systems that integrate genomics, pathology, and radiology for treatment selection remain largely in the research domain.

The gap between research capability and clinical deployment reflects several barriers: regulatory pathways for multimodal AI diagnostics are underdeveloped; clinical workflows are not designed to ingest and act on multimodal AI outputs; and prospective validation studies demonstrating that multimodal AI improves patient outcomes (not just prediction accuracy) are scarce.

Data Infrastructure Requirements

Multimodal AI in oncology presupposes data infrastructure that most healthcare systems lack. Training these models requires linked datasets: the same patient's genomic data, pathology slides, imaging studies, treatment records, and outcomes, all connected and accessible. In practice, these data often reside in separate systems (genomics in the lab, pathology in the pathology department, imaging in PACS, outcomes in the EHR), with inconsistent identifiers and varying data quality.

Federated learning approachesโ€”where models are trained across institutions without centralizing dataโ€”offer a partial solution, but they introduce their own challenges: statistical heterogeneity across sites, communication overhead, and governance complexity.

Open Questions

  • Clinical validation gap: Which multimodal AI systems have been validated in prospective, randomized trials showing improved patient outcomes (survival, quality of life) compared to standard-of-care treatment selection?
  • Interpretability: Can multimodal AI models explain their treatment recommendations in terms that oncologists can evaluate and trust? Or do they function as opaque prediction engines?
  • Equity: Multimodal data generation (whole-genome sequencing, spatial transcriptomics, high-resolution imaging) is expensive. Will multimodal precision oncology widen the gap between well-resourced and under-resourced healthcare systems?
  • Temporal dynamics: Tumors evolve under treatment pressure. How do static multimodal snapshots account for clonal evolution, treatment resistance, and metastatic progression?
  • Minimal viable modality: For a given tumor type and clinical decision, what is the minimum combination of data modalities that captures most of the predictive value? Not every patient needs every data type.
  • Where This Leaves Us

    The npj Digital Medicine review maps a field in transition. The technical machinery for multimodal AI in oncology exists and is producing research results that consistently outperform single-modality approaches in retrospective analyses. The clinical translation machineryโ€”regulatory frameworks, data infrastructure, workflow integration, prospective validationโ€”lags behind. Precision oncology's next chapter will be written not in the capabilities of AI models, but in the willingness and ability of healthcare systems to generate, link, and act on multimodal data at scale.


    References (2)

    AI-driven multi-omic integration for precision oncology. (2025). npj Digital Medicine.
    Fountzilas, E., Pearce, T., Baysal, M. A., Chakraborty, A., & Tsimberidou, A. M. (2025). Convergence of evolving artificial intelligence and machine learning techniques in precision oncology. npj Digital Medicine, 8(1).

    Explore this topic deeper

    Search 290M+ papers, detect research gaps, and find what hasn't been studied yet.

    Click to remove unwanted keywords

    Search 7 keywords โ†’