Foundation Models in Computational Pathology: A Comprehensive Guide for Biomedical Research

Abstract

Foundation models are transforming computational pathology by providing versatile AI trained on massive datasets of histopathology images. This article explores how these models, pretrained via self-supervised learning on millions of whole slide images, enable powerful applications in cancer diagnosis, biomarker prediction, and patient prognosis with minimal task-specific labeling. We detail the core methodologies, from vision-only to multimodal architectures, and address key implementation challenges like data scarcity and computational cost. Through rigorous benchmarking and validation studies, we compare leading models like Virchow, CONCH, and PathOrchestra, providing insights for researchers and drug development professionals to select and optimize these tools for precision medicine and therapeutic R&D.

What Are Pathology Foundation Models? Core Concepts and Evolutionary Shift

Foundation models (FMs) are transforming computational pathology by serving as large-scale, adaptable artificial intelligence (AI) systems trained on extensive datasets. These models leverage self-supervised learning on diverse histopathology images and, in many cases, paired textual data, to develop general-purpose feature representations. Once trained, they can be efficiently adapted to a wide array of downstream clinical and research tasks with minimal task-specific labeling, thereby addressing critical challenges such as data scarcity, annotation costs, and the need for generalizable tools in diagnostic pathology. This whitepaper delineates the core architectural principles, pretraining methodologies, and adaptation techniques of pathology FMs. It further provides a quantitative analysis of current state-of-the-art models, detailed experimental protocols for their application, and a curated toolkit of essential research resources, offering a comprehensive technical guide for researchers and drug development professionals in the field.

The field of computational pathology has been revolutionized by the advent of whole-slide scanners, which convert glass slides into high-resolution digital images [1]. Traditional AI models in pathology were typically designed for a single, specific task—such as cancer grading or metastasis detection—and required large, expensively annotated datasets for training [2] [3]. This paradigm proved difficult to scale across the thousands of possible diagnoses and complex tasks in pathology.

Foundation models represent a fundamental shift. As defined by a Stanford AI institute, a foundation model is "any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks" [2]. In contrast to traditional deep learning models, FMs are characterized by their very large model size, use of transformer architectures, and ability to achieve state-of-the-art performance on adapted tasks while also demonstrating medium to high performance on untrained tasks [2]. As shown in Table 1, the key differentiators of FMs include their pretraining on very large datasets without labeled data and their adaptability to many tasks.

Table 1: Core Differentiators: Foundation Models vs. Non-Foundation Models

Characteristics	Foundation Model	Non-Foundation Model (Deep Learning Model)
Model Architecture	(Mainly) Transformer	Convolutional Neural Network
Model Size	Very Large	Medium to Large
Applicable Task Scope	Many	Single
Performance on Untrained Tasks	Medium to High	Low
Data Amount for Model Training	Very Large	Medium to Large
Use of Labeled Data for Training	No	Yes

In computational pathology, FMs are trained on hundreds of thousands of whole-slide images (WSIs) and histopathology region-of-interests (ROIs) [4] [5]. This large-scale pretraining captures the vast morphological diversity of tissue structures and cellular patterns, encoding them into versatile and transferable feature representations [4]. These representations serve as a "foundation" for models that predict clinical endpoints from WSIs, such as diagnosis, biomarker status, or patient prognosis [4] [1]. The resulting models demonstrate remarkable capabilities in low-data regimes and for rare diseases, which are common challenges in clinical practice [4] [6].

Architectural Frameworks and Training Methodologies

Core Model Architectures

Pathology foundation models employ sophisticated architectures designed to handle the unique challenges of gigapixel whole-slide images.

• Visual Transformer (ViT) based Slide Encoders: Models like TITAN (Transformer-based pathology Image and Text Alignment Network) use a Vision Transformer as a core component to create a general-purpose slide representation [4]. A pivotal innovation is handling the long and variable input sequences of WSIs, which can exceed 10,000 tokens. To manage this computational complexity, TITAN constructs an input embedding space by dividing each WSI into non-overlapping patches, from which features are extracted using a pre-trained patch encoder. These features are spatially arranged into a 2D grid, and the model uses attention with linear bias (ALiBi) to enable long-context extrapolation during inference [4].

• Multimodal Visual-Language Models: Models like CONCH (CONtrastive learning from Captions for Histopathology) and the multimodal version of TITAN integrate image and text data [4] [6]. CONCH, for instance, is based on the CoCa (Contrastive Captioners) framework and comprises an image encoder, a text encoder, and a multimodal fusion decoder [6]. It is trained using a combination of contrastive alignment objectives, which align the image and text modalities in a shared representation space, and a captioning objective that learns to generate captions corresponding to an image [6].

The following diagram illustrates the high-level conceptual workflow of a multimodal foundation model like CONCH or TITAN, from data processing to task-agnostic pretraining.

Large-Scale Pretraining Strategies

The pretraining of pathology FMs is a multi-stage process that leverages vast datasets to instill robust, generalizable knowledge.

• Unimodal Visual Pretraining: The initial stage often involves self-supervised learning (SSL) on large collections of WSIs without labels. For example, TITAN's first stage is a vision-only pretraining on 335,645 WSIs using the iBOT framework, which combines masked image modeling and knowledge distillation [4]. This teaches the model fundamental histomorphological patterns.

• Multimodal Alignment: To equip the model with language capabilities, a subsequent stage aligns visual features with textual descriptions. TITAN undergoes two cross-modal alignment stages: one with 423,122 synthetic fine-grained ROI captions generated by a generative AI copilot, and another with 182,862 slide-level clinical reports [4]. This process enables capabilities like text-to-image retrieval and pathology report generation.

PathOrchestra, another comprehensive FM, was trained on 287,424 slides from 21 tissue types across three centers [5]. The diversity of the pretraining data—covering multiple organs, stains, scanner types, and specimen types (FFPE and frozen)—is a critical factor in the model's subsequent generalization capability [4] [5].

Quantitative Analysis of State-of-the-Art Foundation Models

The performance of pathology FMs has been rigorously evaluated across a wide spectrum of tasks. The table below summarizes the scale and key capabilities of several leading models.

Table 2: Comparative Analysis of Pathology Foundation Models

Model	Training Data Scale	Key Architectures/Techniques	Reported Performance Highlights
TITAN [4]	335,645 WSIs; 423k synthetic captions; 183k reports	Vision Transformer (ViT), iBOT SSL, ALiBi, multimodal alignment	Outperforms ROI and slide FMs in linear probing, few-shot/zero-shot classification, rare cancer retrieval, and report generation.
PathOrchestra [5]	287,424 WSIs from 21 tissues	Self-supervised vision encoder	Accuracy >0.950 in 47/112 tasks, including pan-cancer classification and lymphoma subtyping; generates structured reports.
CONCH [6]	1.17M image-caption pairs	Contrastive learning & captioning (CoCa framework)	SOTA zero-shot accuracy: 90.7% (NSCLC), 90.2% (RCC), 91.3% (BRCA); excels at cross-modal retrieval and segmentation.
Tissue Concepts [7]	912,000 patches from 16 tasks	Supervised multi-task learning	Matches self-supervised FM performance on major cancers using only ~6% of the training data.

Key performance insights from these models include:

• CONCH demonstrated a significant leap in zero-shot classification, outperforming other visual-language models like PLIP and BiomedCLIP by over 10-35% on tasks such as non-small cell lung cancer (NSCLC) and renal cell carcinoma (RCC) subtyping [6].
• PathOrchestra showcased exceptional breadth, achieving high accuracy across 112 tasks including digital slide preprocessing, pan-cancer classification, and biomarker assessment, demonstrating its clinical readiness [5].
• TITAN emphasizes handling resource-limited scenarios, showing strong performance in rare disease retrieval and cancer prognosis without requiring fine-tuning or clinical labels [4].
• The Tissue Concepts model highlights that supervised multi-task learning can be a data-efficient path to building capable FMs, achieving performance comparable to large-scale self-supervised models with drastically less data [7].

Experimental Protocols for Downstream Task Adaptation

Applying a pretrained foundation model to a specific problem involves several established protocols. The choice of method depends on the amount of labeled data available for the downstream task.

Zero-Shot and Few-Shot Evaluation

In the zero-shot setting, the model performs a task without any further task-specific training. For classification, this is typically achieved by leveraging the model's multimodal alignment.

• Protocol: Class names are converted into a set of text prompts (e.g., "invasive ductal carcinoma of the breast," "breast IDC"). The image (or image tiles from a WSI) is then matched with the most similar text prompt in the model's shared image-text representation space using cosine similarity [6]. For WSIs, an aggregation method like MI-Zero can be used to combine tile-level scores into a slide-level prediction [6].
• Application: This method is ideal for rapid prototyping, generating initial hypotheses, or applications where acquiring labeled data is impossible.

Linear Probing and Fine-Tuning

For tasks with limited labeled data, linear probing and fine-tuning are standard approaches.

• Linear Probing Protocol: The weights of the pretrained FM are frozen. A simple linear classifier (e.g., a single fully connected layer) is then trained on top of the fixed features extracted by the FM for the new task. This tests the quality of the learned representations [4] [6].
• Full Fine-Tuning Protocol: The entire FM (or a significant portion of it) is further trained on the labeled data for the downstream task. This allows the model to adjust its pre-learned features to the specifics of the new task. Fine-tuning requires more data than linear probing but can achieve higher performance.
• Application: These methods are the most common for adapting a general FM to a specific clinical task, such as grading a specific cancer type or predicting a specific biomarker.

The following diagram outlines the decision workflow for selecting the appropriate adaptation protocol based on data availability and task goals.

The development and application of pathology FMs rely on a suite of key "reagent" solutions— computational tools, datasets, and infrastructure.

Table 3: Essential Research Reagents for Pathology Foundation Model Research

Research Reagent	Function/Description	Exemplars in Literature
Pre-trained Patch Encoders	Extracts foundational feature representations from small image patches; often used as a preprocessing step for slide-level FMs.	CONCH [6]
Large-Scale WSI Datasets	Diverse, multi-organ collections of whole-slide images used for large-scale self-supervised pretraining.	Mass-340K (335,645 WSIs) [4], PathOrchestra Dataset (287,424 WSIs) [5]
Multimodal Datasets (Image-Text Pairs)	Paired histopathology images and textual descriptions (reports, synthetic captions) for training visual-language models.	1.17M image-caption pairs (CONCH) [6], 423k synthetic captions + 183k reports (TITAN) [4]
Synthetic Caption Generators	Multimodal generative AI copilots that generate fine-grained morphological descriptions for ROIs, providing scalable supervision.	PathChat (used by TITAN) [4]
Benchmark Suites	Curated collections of public and private datasets for standardized evaluation of FMs across multiple tasks.	14 diverse benchmarks (CONCH) [6], 112 tasks (PathOrchestra) [5]
Multiple Instance Learning (MIL) Frameworks	Algorithms for aggregating patch-level or tile-level predictions to form a slide-level diagnosis or score.	Attention-based MIL (ABMIL) used in PathOrchestra [5]

Foundation models represent a paradigm shift in computational pathology, moving from a one-model-one-task approach to a versatile, scalable framework where a single, broadly trained model can be efficiently adapted to countless downstream applications. Their demonstrated success in tasks ranging from pan-cancer classification and rare disease retrieval to biomarker assessment and report generation underscores their transformative potential for both research and clinical practice [4] [5] [6].

The future of pathology FMs lies in several key directions: the development of generalist medical AI that integrates pathology FMs with models from other medical domains (e.g., radiology, genomics) [2]; continued scaling of model and dataset size; improved efficiency for clinical deployment; and rigorous real-world validation to address challenges related to diagnostic accuracy, cost, patient confidentiality, and regulatory ethics [1]. As these models continue to evolve, they are poised to become an indispensable tool in the pathologist's arsenal, enhancing diagnostic precision, personalizing treatment plans, and ultimately improving patient outcomes.

The field of computational pathology stands at a pivotal moment, poised to revolutionize cancer diagnosis, prognosis, and treatment planning through artificial intelligence. However, for years, progress has been constrained by the fundamental limitations of traditional supervised learning approaches. These models, which learn from vast amounts of meticulously labeled data, face particular challenges in histopathology where annotation costs are prohibitive and inter-observer variability among pathologists complicates ground truth establishment [2]. The average pathologist earns approximately $149 per hour, with annotation costs reaching $12 per slide when assuming just five minutes of annotation time [2]. This economic reality, coupled with the gigapixel complexity of whole-slide images (WSIs), has created a significant bottleneck in developing robust AI systems for histopathology.

Foundation models represent a paradigm shift in computational pathology, moving from task-specific models to general-purpose AI systems trained on broad data that can be adapted to diverse downstream tasks [8] [2]. These models leverage self-supervised learning (SSL) to learn transferable feature representations from unlabeled pathology images, fundamentally overcoming the annotation dependency that has plagued traditional supervised approaches. The emergence of models like TITAN [4], UNI [9], and Virchow [9] demonstrates how this new paradigm enables applications ranging from rare disease retrieval to cancer prognosis without task-specific fine-tuning.

Table 1: Comparison of Traditional Supervised Learning vs. Foundation Models in Computational Pathology

Characteristic	Traditional Supervised Learning	Foundation Models
Model Architecture	Convolutional Neural Networks (CNN) [2]	Transformer-based [4] [2]
Training Data	Medium to large labeled datasets [2]	Very large unlabeled datasets (335k+ WSIs) [4] [9]
Applicable Tasks	Single task [2]	Many downstream tasks [2]
Performance on Untrained Tasks	Low [2]	Medium to high [2]
Annotation Dependency	High (pathologist-intensive) [2]	Minimal (self-supervised) [4] [9]
Generalization	Limited to training distribution	Strong out-of-distribution performance [4]

The Limitations of Traditional Supervised Learning

Annotation Burden and Scalability Challenges

In traditional supervised learning, each diagnostic task requires pathologists to manually annotate thousands of histopathology images, creating an unsustainable scalability barrier. This process is not only time-consuming but also economically prohibitive for healthcare institutions [2]. The problem intensifies for rare diseases where cases are scarce, and for complex tasks like predicting patient prognosis or genetic mutations from histology, where ground truth labels may require expensive molecular testing or long-term clinical follow-up.

Technical Limitations in Model Performance

Beyond resource constraints, supervised learning models face fundamental technical limitations. These models frequently suffer from overfitting—learning patterns too specifically from training data and failing to generalize to unseen data [10]. In dynamic clinical environments where data distribution frequently changes, supervised models often fail to adapt without retraining [10]. Additionally, these models demonstrate limited adaptability to completely new scenarios unseen during training, unlike human pathologists who can apply reasoning to novel cases [10].

Foundation Models: A New Paradigm for Computational Pathology

Core Architectural Principles

Foundation models in computational pathology represent a fundamental shift in approach, centered on three key principles: self-supervised learning on massive unlabeled datasets, transformer architectures for whole-slide representation, and multimodal alignment.

The cornerstone of this paradigm is self-supervised learning (SSL), which enables models to learn visual representations from the inherent structure of histopathology data without manual annotations [9]. Algorithms like DINOv2 [9], iBOT [4], and masked autoencoders [9] have proven particularly effective for pathology images. These methods create learning objectives from the data itself, such as predicting missing parts of an image or identifying different augmentations of the same tissue region.

Transformer architectures form the backbone of modern pathology foundation models, enabling long-range context modeling across gigapixel whole-slide images [4]. Unlike traditional convolutional neural networks that process local regions independently, transformers use self-attention mechanisms to capture relationships between distant tissue regions, mirroring how pathologists integrate local cytological features with global architectural patterns.

Multimodal learning represents the third pillar, with models like TITAN aligning visual representations with corresponding pathology reports and synthetic captions [4]. This cross-modal alignment enables capabilities such as text-based image retrieval, pathology report generation, and zero-shot classification without explicit training.

Implementation Framework: The TITAN Model

The TITAN (Transformer-based pathology Image and Text Alignment Network) model exemplifies the foundation model paradigm in practice [4]. Its implementation involves a sophisticated three-stage framework:

Stage 1: Vision-only Pretraining TITAN employs the iBOT framework for visual self-supervised learning on 335,645 whole-slide images [4]. The model processes WSIs by first dividing them into non-overlapping 512×512 pixel patches at 20× magnification, encoding each patch into 768-dimensional features using a pretrained patch encoder [4]. These features are spatially arranged in a 2D grid preserving tissue topography. To handle variable WSI sizes, the model randomly crops 16×16 feature regions (covering 8,192×8,192 pixels), then samples multiple global (14×14) and local (6×6) crops for self-supervised learning [4]. The architecture uses Attention with Linear Biases (ALiBi) extended to 2D, enabling extrapolation to long contexts at inference by biasing attention based on Euclidean distance between features in the tissue [4].

Stage 2: ROI-Level Cross-Modal Alignment The vision model is aligned with 423,122 synthetic fine-grained captions generated using PathChat, a multimodal generative AI copilot for pathology [4]. This enables the model to understand regional morphological descriptions.

Stage 3: WSI-Level Cross-Modal Alignment Finally, the model aligns whole-slide representations with 182,862 clinical pathology reports, enabling slide-level reasoning and report generation capabilities [4].

Figure 1: TITAN Foundation Model Architecture and Training Pipeline

Experimental Validation and Benchmarking

Performance Across Clinical Tasks

Rigorous evaluation of pathology foundation models reveals their substantial advantages over traditional supervised approaches. In comprehensive benchmarks assessing disease detection and biomarker prediction across multiple institutions, SSL-trained pathology models consistently outperform models pretrained on natural images [9]. For disease detection tasks, foundation models achieve AUCs above 0.9 across all evaluated tasks, demonstrating robust diagnostic capability [9].

The TITAN model specifically excels in resource-limited clinical scenarios including rare disease retrieval and cancer prognosis, operating without any fine-tuning or clinical labels [4]. It outperforms both region-of-interest (ROI) and slide foundation models across multiple machine learning settings: linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval, and pathology report generation [4]. This generalizability is particularly valuable for rare cancers where collecting large annotated datasets is impractical.

Table 2: Benchmark Performance of Public Pathology Foundation Models on Clinical Tasks

Model	Training Data	Key Architectures	Clinical Task Performance
TITAN [4]	335,645 WSIs + 423K synthetic captions + 183K reports	ViT with ALiBi, iBOT, VLA	Superior performance on zero-shot classification, rare cancer retrieval, report generation
UNI [9]	100M tiles from 20 tissue types	ViT-L, DINO	State-of-the-art across 33 tasks including classification, segmentation, retrieval
Phikon [9]	43.3M tiles from 13 anatomic sites	ViT-B, iBOT	Strong performance on 17 downstream tasks across 7 cancer indications
Virchow [9]	2B tiles from 1.5M slides	ViT-H, DINO	SOTA on tile-level and slide-level benchmarks for tissue classification and biomarker prediction
Prov-GigaPath [9]	1.3B tiles from 171K WSIs	DINO, MAE, LongNet	Strong performance on 17 genomic prediction and 9 cancer subtyping tasks

Methodologies for Model Evaluation

Robust evaluation of pathology foundation models requires standardized benchmarking protocols. Recent initiatives have established comprehensive clinical benchmarks using real-world data from multiple medical centers [9]. The evaluation methodology typically encompasses:

Linear Probing: Assessing representation quality by training a linear classifier on frozen features while varying training set sizes (from 1% to 100% of available labels) [9]. This measures how well the model captures diagnostically relevant features.

Few-Shot Learning: Evaluating model performance with very limited labeled examples (e.g., 1-100 samples per class) to simulate rare disease scenarios [4].

Zero-Shot Classification: Testing model capability to recognize disease categories without any task-specific training, particularly for multimodal models using text prompts [4].

Cross-Modal Retrieval: Measuring the model's ability to retrieve relevant histology images given text queries, and vice versa [4].

Slide Retrieval: Assessing retrieval of diagnostically similar slides from a database, valuable for identifying rare cases and clinical decision support [4].

External validation across multiple institutions is crucial for assessing model generalizability and mitigating dataset-specific biases [9]. Performance should be measured on clinical data generated during standard hospital operations rather than curated research datasets alone.

Essential Research Toolkit for Pathology Foundation Models

Table 3: Research Reagent Solutions for Pathology Foundation Model Development

Component	Function	Examples & Specifications
Whole-Slide Image Data	Model pretraining and validation	Mass-340K (335,645 WSIs) [4], TCGA [9], multi-institutional clinical cohorts [9]
Computational Infrastructure	Handling gigapixel images and model training	High-end GPUs, distributed training frameworks, patch encoding pipelines [4] [9]
Patch Encoders	Feature extraction from image patches	CONCH [4], self-supervised models (DINOv2, iBOT) [9]
Annotation Platforms	Limited supervision for fine-tuning	Digital pathology annotation tools, slide-level labels from reports [4]
Multimodal Data	Vision-language pretraining	Pathology reports [4], synthetic captions [4], genomic data [2]
Benchmarking Frameworks	Standardized model evaluation	Clinical benchmark datasets [9], automated evaluation pipelines [9]

Future Directions and Clinical Implementation

Emerging Research Frontiers

The development of pathology foundation models continues to evolve rapidly, with several promising research directions emerging. Increased scale and diversity in pretraining data remains a priority, with recent models expanding to millions of slides across hundreds of tissue types [9]. Multimodal integration represents another frontier, with models incorporating not only pathology images and reports but also genomic, transcriptomic, and clinical data to enable more comprehensive patient characterization [2].

The rise of generative capabilities in pathology foundation models opens new possibilities for synthetic data generation, augmentation of rare diseases, and educational applications [11]. Additionally, research into explainability and interpretability is crucial for clinical adoption, helping pathologists understand model predictions and building trust in AI-assisted diagnoses [12].

Pathways to Clinical Deployment

Translating pathology foundation models from research to clinical practice requires addressing several critical challenges. Regulatory approval pathways must be established for these general-purpose models, which differ fundamentally from single-task devices [1]. Integration with clinical workflows presents technical and usability challenges, requiring seamless incorporation into digital pathology platforms and laboratory information systems.

Ongoing validation and monitoring is essential to ensure model performance generalizes across diverse patient populations and institution-specific practices [9]. Finally, education and training for pathologists will be crucial for effective human-AI collaboration, ensuring that clinicians can appropriately interpret model outputs and maintain ultimate diagnostic responsibility.

The ultimate vision is the development of generalist medical AI that integrates pathology foundation models with models from other medical domains (radiology, genomics, electronic health records) to provide comprehensive diagnostic support and enable truly personalized medicine [2]. As these technologies mature, they have the potential to transform pathology from a predominantly descriptive discipline to a quantitative, predictive science that enhances patient care through more accurate diagnoses, prognostic insights, and tailored treatment recommendations.

Computational pathology is undergoing a revolutionary transformation, driven by the emergence of foundation models capable of analyzing gigapixel whole-slide images (WSIs) with unprecedented sophistication [11]. These models represent a paradigm shift from task-specific algorithms to general-purpose visual encoders that learn transferable feature representations from vast repositories of histopathology data [13]. The development of these models is propelled by three interconnected technological forces: unprecedented data scale, advanced self-supervised learning (SSL) algorithms, and specialized transformer architectures [4] [13]. This convergence addresses critical challenges in pathology artificial intelligence (AI), including data imbalance, annotation dependency, and the need for robust generalization across diverse tissue types and disease conditions [12]. Foundation models are increasingly demonstrating remarkable capabilities across diagnostic, prognostic, and predictive tasks, establishing a new cornerstone for computational pathology research and clinical application [1].

The Critical Role of Data Scale in Model Performance

The performance of pathology foundation models exhibits a strong correlation with the scale and diversity of their pretraining datasets. Model generalization improves significantly when trained on larger datasets encompassing varied tissue types, staining protocols, and scanner variations [13].

Table 1: Data Scale of Representative Pathology Foundation Models

Model Name	Tiles (Billions)	Whole-Slide Images (WSIs)	Tissue Types	Primary Algorithm
Virchow2 [13]	1.7	3.1 million	~200	DINOv2
Prov-GigaPath [13]	1.3	171,189	31	DINOv2, LongNet
UNI [13]	0.1	100,000	20	DINOv2
TITAN [4]	-	335,645	20	iBOT, Vision-Language
Phikon [13]	0.043	6,093	13	iBOT

Massive datasets enable models to learn invariant features across technical variations (e.g., stain heterogeneity) and biological variations (e.g., tissue morphology) [14]. For instance, the Virchow2 model, trained on 3.1 million WSIs across nearly 200 tissue types, achieves state-of-the-art performance by capturing a vast spectrum of histopathological patterns [13]. Similarly, the TITAN model leverages 335,645 WSIs and 423,122 synthetic captions to create general-purpose slide representations applicable to rare disease retrieval and cancer prognosis without further fine-tuning [4]. This scaling trend underscores the critical importance of large-scale, curated data repositories for developing robust pathology foundation models.

Self-Supervised Learning Algorithms for Annotation-Efficient Training

Self-supervised learning has emerged as the dominant paradigm for pretraining pathology foundation models, effectively addressing the annotation bottleneck in medical imaging. SSL algorithms learn powerful feature representations by formulating pretext tasks from unlabeled data, eliminating the need for costly manual annotations [13] [14].

Core SSL Algorithms and Their Adaptations

The table below summarizes key SSL algorithms and their implementation in computational pathology.

Table 2: Self-Supervised Learning Algorithms in Computational Pathology

Algorithm	Core Mechanism	Key Pathology Adaptations	Representative Models
DINOv2 [13] [14]	Self-distillation with noise-resistant loss functions	Multi-magnification training, stain normalization augmentations	UNI, Virchow, Prov-GigaPath, Phikon-v2
iBOT [4] [13]	Combines masked image modeling with online tokenizer	Hierarchical masking strategies, tissue-aware cropping	TITAN, Phikon
Masked Autoencoders (MAE) [13]	Reconstructs randomly masked image patches	Semantic-aware masking preserving tissue structures	Prov-GigaPath (slide-level)

Domain-Specific Optimizations for Histopathology

Effective application of SSL in pathology requires domain-specific optimizations. Unlike natural images, WSIs exhibit unique characteristics including gigapixel resolutions, known physical scale of pixels, and redundant morphological patterns across populations [14]. Key adaptations include:

• Morphology-Preserving Augmentations: Replacement of standard crop-and-resize operations with Extended-Context Translation (ECT) to prevent distortion of cell and tissue shapes [14]. ECT uses a larger source field of view to minimize resizing artifacts while maintaining view overlap.
• Multi-Scale Learning: Training on tiles extracted from multiple magnifications (5×, 10×, 20×, 40×) to capture features manifesting at different resolutions [13] [14].
• Stain-Invariant Color Augmentations: Color perturbation strategies specifically designed to address variations in staining protocols and scanner types without reflecting underlying pathological differences [14].

These domain-specific modifications enable models to learn features that are invariant to technical artifacts while remaining sensitive to biologically relevant morphological patterns.

Transformer Architectures for Whole-Slide Image Analysis

Transformer architectures have revolutionized computational pathology by enabling long-range context modeling across gigapixel WSIs. While convolutional neural networks (CNNs) remain effective for local feature extraction, transformers excel at capturing global tissue microenvironment relationships [4] [13].

Architectural Innovations for Gigapixel Images

Standard transformer architectures face computational challenges when processing WSIs due to the quadratic complexity of self-attention mechanisms. Several innovative approaches have emerged to address this limitation:

• Hierarchical Processing: Models like TITAN employ a two-stage architecture where a patch encoder first extracts features from local image regions, followed by a slide-level transformer that aggregates these features into a whole-slide representation [4].
• Long-Range Context Modeling: The TITAN model incorporates Attention with Linear Biases (ALiBi) extended to 2D, enabling extrapolation to longer contexts during inference based on relative Euclidean distances between tissue regions [4].
• Multi-Resolution Architectures: Some frameworks implement hierarchical backbones that capture both cellular-level details and tissue-level context, optimized for transformer-based processing of WSIs [15].

Vision-Language Multimodal Integration

Multimodal transformer architectures represent a significant advancement in pathology AI. The TITAN model demonstrates how vision-language pretraining aligns image representations with pathological concepts [4]. By incorporating pathology reports and synthetic captions generated from AI copilots, these models enable cross-modal retrieval, zero-shot classification, and pathology report generation [4]. This multimodal alignment creates more clinically relevant representations that capture the semantic relationships between morphological features and diagnostic interpretations.

Experimental Protocols and Benchmarking

Rigorous evaluation frameworks are essential for assessing the clinical relevance and generalizability of pathology foundation models. Standardized benchmarks enable comparative analysis across different architectures and training approaches.

Performance Metrics and Downstream Tasks

Comprehensive model evaluation encompasses diverse clinical tasks including cancer subtyping, biomarker prediction, survival analysis, and rare cancer retrieval [4] [13]. The table below summarizes key evaluation metrics and benchmarks for pathology foundation models.

Table 3: Performance Benchmarks of Pathology Foundation Models on Clinical Tasks

Model	Linear Probing Accuracy	Few-Shot Learning	Zero-Shot Classification	Slide Retrieval	Report Generation
TITAN [4]	Outperforms ROI & slide foundations	State-of-the-art	Enabled via vision-language	Superior rare cancer retrieval	Generates clinical reports
Virchow2 [13]	State-of-the-art on 12 tasks	High data efficiency	Not specified	Not specified	Not applicable
UNI [13]	Strong performance across 33 tasks	Effective with limited labels	Limited capability	Good performance	Not applicable
Prov-GigaPath [13]	Excellent for genomics & subtyping	Not specified	Not specified	Not specified	Not applicable

Clinical Workflow Integration and Efficiency Gains

Beyond traditional performance metrics, foundation models demonstrate significant value in clinical workflow optimization. Studies show AI integration can reduce diagnostic time by approximately 90% in pathology and radiology fields [16]. These efficiency gains manifest through various mechanisms:

• Workload Reduction: AI models filter non-diagnostic regions, reducing data volume requiring pathologist review by significant margins [16].
• Decision Support: Foundation models provide annotated images highlighting suspicious regions, enhancing diagnostic confidence without fully automating the process [16].
• Independent Diagnosis: In limited scenarios, AI systems can achieve diagnostic performance comparable to human experts, particularly for well-defined classification tasks [16].

Successful development and application of pathology foundation models requires specialized computational resources and data infrastructure. The following toolkit outlines essential components for researchers in this field.

Table 4: Research Reagent Solutions for Pathology Foundation Model Development

Resource Category	Specific Examples	Function & Application
Pretrained Models	TITAN, UNI, Virchow, Prov-GigaPath, Phikon, CTransPath	Transfer learning, feature extraction, model fine-tuning for specific tasks
SSL Algorithms	DINOv2, iBOT, Masked Autoencoders (MAE)	Self-supervised pretraining on unlabeled whole-slide images
Architecture Components	Vision Transformers (ViT), Swin Transformers, ALiBi Positional Encoding	Long-range context modeling, gigapixel image processing
Pathology Datasets	TCGA, CAMELYON16, PANNUKE, Proprietary Institutional Repositories	Model training, validation, and benchmarking across diverse tissue types
Computational Infrastructure	High-memory GPU clusters, Distributed Training Frameworks, Large-scale Storage	Handling gigapixel images, training billion-parameter models
Domain-Specific Augmentations	Extended-Context Translation, Stain Normalization, Multi-Magnification Sampling	Preserving histological semantics while enhancing data diversity

The convergence of data scale, self-supervised learning, and transformer architectures has established a powerful foundation for computational pathology research. These technological drivers enable models that generalize across diverse clinical scenarios, particularly in resource-limited settings such as rare disease diagnosis [4]. As the field evolves, key challenges remain in standardizing model evaluation, ensuring regulatory compliance, and addressing ethical considerations around data privacy and algorithmic bias [1]. Future research directions will likely focus on multimodal integration with genomic and clinical data, federated learning to leverage decentralized data sources, and developing more efficient architectures for real-time clinical deployment. The ongoing maturation of pathology foundation models promises to significantly enhance diagnostic accuracy, personalize treatment strategies, and deepen our understanding of disease biology through AI-powered histomorphological analysis.

The field of computational pathology is undergoing a fundamental transformation, moving from specialized, task-specific deep learning models toward large-scale, adaptable foundation models (FMs). This shift mirrors the revolution witnessed in natural language processing and computer vision, representing a new paradigm for developing artificial intelligence (AI) in healthcare [2] [17]. Traditional deep learning models have provided substantial benefits in automating pathology tasks but face inherent limitations in scalability, generalization, and annotation dependency. Foundation models, trained on vast quantities of unlabeled data through self-supervised learning (SSL), overcome these barriers by learning universal histopathological representations that can be adapted to numerous downstream tasks with minimal fine-tuning [18] [17]. This whitepaper provides an in-depth technical analysis of the core differences between these two approaches, focusing on architectural principles, training methodologies, performance characteristics, and practical implementation for researchers, scientists, and drug development professionals engaged in precision oncology and computational pathology research.

Fundamental Architectural and Methodological Divergences

The distinction between foundation models and traditional deep learning in pathology begins at the most fundamental level of architecture, training data utilization, and learning paradigms. These differences explain the divergent capabilities and applications of each approach.

Model Architecture and Scalability

Traditional deep learning models in pathology typically employ Convolutional Neural Networks (CNNs) as their backbone architecture. These models are designed with a specific task in mind, such as tumor classification or cell segmentation, and their architecture is optimized accordingly [2] [19]. CNNs excel at capturing local spatial features through their convolutional filters but have limited capacity for modeling long-range dependencies in whole-slide images (WSIs) due to their inherent locality bias.

Foundation models predominantly utilize Vision Transformer (ViT) architectures, which leverage self-attention mechanisms to capture global context across entire image regions [4] [18]. The transformer architecture enables processing of variable-length sequences of image patches, making it particularly suitable for gigapixel WSIs that must be divided into thousands of patches. This architectural advantage allows FMs to model relationships between geographically distant tissue structures that may be pathologically significant but are missed by CNN-based approaches [20] [18].

Table 1: Core Architectural Differences Between Traditional Deep Learning and Foundation Models

Characteristic	Traditional Deep Learning Model	Foundation Model
Primary Architecture	Convolutional Neural Networks (CNNs)	Vision Transformer (ViT)
Model Size	Medium to large	Very large
Context Processing	Local receptive fields	Global self-attention
Input Flexibility	Fixed input dimensions	Variable sequence length
Parameter Count	Millions to hundreds of millions	Hundreds of millions to billions

Training Paradigms and Data Requirements

The training methodologies for these two approaches differ radically in their fundamental objectives and data requirements:

Traditional deep learning models rely exclusively on supervised learning, requiring large datasets of histopathology images with expert annotations for each specific task [2] [21]. This creates a significant bottleneck in development, as pathology annotations are time-consuming and expensive to acquire. The annotation cost alone is substantial—approximately $12 per slide assuming a pathologist salary of $149 per hour and 5 minutes of annotation time per slide [2]. These models learn exclusively from the labeled examples provided, with their knowledge strictly bounded by the diversity and quality of the annotated dataset.

Foundation models employ self-supervised learning (SSL) during pre-training, allowing them to learn from massive volumes of unlabeled histopathology images [18] [17]. SSL techniques include:

• Masked Image Modeling (MIM): Randomly masking portions of input images and training the model to reconstruct the missing parts [4] [18].
• Contrastive Learning: Learning representations by maximizing agreement between differently augmented views of the same image while distinguishing them from other images [20] [18].
• Self-Distillation: Using a student-teacher framework where the student network learns to match the output of the teacher network when presented with different augmented views of the same image [20].

This self-supervised pre-training phase allows FMs to learn general-purpose visual representations of histopathological morphology without any manual annotations. Once pre-trained, FMs can be adapted to specific tasks with minimal labeled examples through fine-tuning, few-shot learning, or even zero-shot learning in some cases [4] [22].

Table 2: Training Paradigm Comparison

Aspect	Traditional Deep Learning Model	Foundation Model
Learning Paradigm	Supervised learning	Self-supervised learning + transfer learning
Data Requirements	Large labeled datasets for each task	Massive unlabeled datasets + minimal labels for adaptation
Annotation Dependency	High	Low
Primary Training Objective	Task-specific loss minimization	Pre-training: SSL objective; Fine-tuning: Task-specific objective
Example SSL Algorithms	Not applicable	DINO, iBOT, Masked Autoencoding, Contrastive Learning

Multimodal Integration Capabilities

A distinctive capability of foundation models is their inherent capacity for multimodal integration, which remains challenging for traditional deep learning approaches.

Traditional deep learning models typically operate on single data modalities (e.g., H&E stained WSIs) and require specialized architectures to incorporate additional data types. Integrating pathology images with genomic data or clinical text often requires complex, custom-designed fusion networks that are difficult to optimize and scale [2] [23].

Foundation models can be designed from the ground up to process and align multiple data modalities through architectures that create joint embedding spaces [4] [18]. For example:

• CONCH and PLIP create aligned representations of histopathology images and textual descriptions [20].
• TITAN integrates whole-slide images with pathology reports and synthetic captions through cross-modal alignment [4].
• CLOVER integrates pathology images with multi-omics data for improved prognosis prediction [17].

This multimodal capability allows FMs to capture complex relationships between tissue morphology, clinical context, and molecular features, enabling more comprehensive pathological analysis [2] [17].

Performance and Functional Capabilities

The architectural and methodological differences between traditional deep learning and foundation models translate directly to divergent performance characteristics and functional capabilities across various pathology tasks.

Accuracy and Generalization

Comprehensive benchmarking studies reveal significant performance differences between these approaches:

Traditional deep learning models typically achieve high performance on the specific tasks and datasets they were trained on but often suffer from performance degradation when applied to data from different institutions, staining protocols, or scanner types [19] [22]. This limited generalization capability stems from their narrower training data distribution and architectural constraints.

Foundation models demonstrate superior generalization across diverse datasets and tissue types [22]. In a comprehensive benchmark evaluating 31 AI foundation models across 41 tasks, pathology-specific foundation models consistently outperformed general vision models and traditional approaches [22]. Notably, Virchow2 achieved the highest performance across multiple tasks and datasets, demonstrating the generalization capability of large-scale FMs [22]. FMs also show remarkable performance in low-data regimes, achieving state-of-the-art results on rare cancer types with minimal fine-tuning [4] [17].

Table 3: Performance Comparison on Pathology Tasks

Performance Metric	Traditional Deep Learning Model	Foundation Model
Task Specificity	Single task	Multiple downstream tasks
Performance on Trained Tasks	High to state-of-the-art	State-of-the-art
Performance on Untrained Tasks	Low	Medium to high
Data Efficiency	Requires large labeled datasets per task	High efficiency with few-shot learning
Cross-Institutional Generalization	Limited without explicit domain adaptation	Superior due to diverse pre-training
Rare Disease Performance	Limited by annotated examples	Strong even with minimal examples

Operational Efficiency and Clinical Applicability

The operational characteristics of these models have significant implications for their clinical integration:

Traditional deep learning models incur high initial development costs due to annotation requirements but may have lower computational demands during inference. However, developing separate models for each task creates maintenance challenges and workflow integration complexity in clinical environments [2] [21].

Foundation models have extremely high pre-training costs—reaching millions of dollars for the largest models—but offer significantly reduced adaptation costs for new tasks [17]. Once deployed, a single FM can serve multiple clinical applications, simplifying integration and maintenance. The emerging capability for zero-shot and few-shot learning further enhances their operational efficiency in clinical settings where labeled data may be scarce [4].

Experimental Framework and Validation

Rigorous experimental validation is essential for evaluating both traditional deep learning models and foundation models in pathology. This section outlines key methodologies and benchmarks used to assess model performance and robustness.

Representative Experimental Protocols

Benchmarking Foundation Models: A comprehensive evaluation framework for pathology FMs should include multiple assessment dimensions [22]:

• Linear Probing: Freezing the pre-trained backbone and training a linear classifier on top of the extracted features to evaluate representation quality.
• End-to-End Fine-Tuning: Unfreezing all parameters and fine-tuning the entire model on downstream tasks.
• Few-Shot Learning: Evaluating performance with limited labeled examples (e.g., 1, 5, 10, or 20 samples per class).
• Cross-Modal Retrieval: For multimodal FMs, testing the ability to retrieve relevant images given text queries and vice versa.
• Slide-Level Representation Learning: Assessing the quality of whole-slide representations for tasks such as cancer subtyping, biomarker prediction, and survival prognosis [4].

Representational Similarity Analysis (RSA): This methodology, borrowed from computational neuroscience, enables quantitative comparison of the internal representations learned by different models [20]. RSA involves:

• Extracting embeddings from multiple models for a standardized set of histopathology image patches.
• Computing Representational Dissimilarity Matrices (RDMs) that capture pairwise distances between embeddings.
• Comparing RDMs across models to quantify representational similarity and diversity.
• Analyzing slide-dependence and disease-dependence of representations to understand model robustness.

Recent RSA studies have revealed that FMs with similar training paradigms (vision-only vs. vision-language) do not necessarily learn similar representations, and that stain normalization can reduce slide-specific biases in FM representations [20].

The Scientist's Toolkit: Essential Research Reagents

Table 4: Key Experimental Resources for Pathology Foundation Model Research

Resource Category	Examples	Function and Application
Pre-Trained Models	UNI, Virchow, CONCH, PLIP, Prov-GigaPath, TITAN	Provide foundational representations for downstream pathology tasks without training from scratch
Benchmark Datasets	TCGA, CPTAC, Camelyon, internal validation sets	Standardized evaluation of model performance and generalization capabilities
Evaluation Frameworks	Linear probing, few-shot evaluation, cross-modal retrieval, survival analysis	Systematic assessment of model capabilities across diverse task types
Computational Infrastructure	High-end GPUs (NVIDIA A100/H100), distributed training frameworks, cloud computing platforms	Enable model training, fine-tuning, and deployment at scale
Pathology-Specific Libraries	TIAToolbox, QuPath, Whole-Slide Imaging processing libraries	Facilitate preprocessing, annotation, and analysis of whole-slide images

Implementation Workflows

The implementation of foundation models in pathology research follows distinct workflows that differ significantly from traditional deep learning approaches. The following diagram illustrates the core architectural and workflow differences between these paradigms:

Diagram 1: Architectural and Workflow Comparison Between Traditional Deep Learning and Foundation Models in Pathology

Technical Implementation Considerations

Implementing foundation models in pathology research requires addressing several technical considerations:

Data Preprocessing and Augmentation:

• Whole-slide images must be divided into patches (typically 256×256 or 512×512 pixels at 20× magnification) [4] [18].
• Stain normalization techniques are often applied to reduce domain shift between institutions [20].
• For multimodal FMs, text data (pathology reports, captions) requires tokenization and preprocessing to align with image patches.

Model Selection Criteria:

• Vision-only vs. Vision-Language: Vision-language models (e.g., CONCH, PLIP) enable zero-shot capabilities and cross-modal retrieval, while vision-only models (e.g., UNI, Virchow) may offer superior performance on standard vision tasks [20] [22].
• Model Scale: Larger models generally perform better but require more computational resources for fine-tuning and inference [22].
• Pre-training Data: Models pre-trained on diverse, multi-institutional datasets typically generalize better [4] [22].

Fine-tuning Strategies:

• Linear Probing: Training only a linear classifier on frozen features provides a rapid performance baseline.
• Full Fine-tuning: Updating all parameters typically achieves the best performance but requires more computational resources and careful regularization to prevent overfitting.
• Parameter-Efficient Fine-tuning (PEFT): Techniques like LoRA (Low-Rank Adaptation) can adapt large FMs with minimal parameter updates.

Future Directions and Research Opportunities

The development of foundation models in computational pathology is rapidly evolving, with several promising research directions emerging:

Generalist Medical AI: The integration of pathology FMs with foundation models from other medical domains (radiology, genomics, clinical notes) to create comprehensive diagnostic systems that leverage complementary information [2] [23].

Improved Multimodal Alignment: Developing more sophisticated techniques for aligning histopathological image features with textual descriptions, genomic data, and clinical outcomes to enhance model interpretability and clinical utility [4] [18].

Federated Learning: Enabling multi-institutional collaboration on FM development without sharing sensitive patient data, addressing data privacy concerns while improving model generalization [23] [17].

Explainability and Interpretability: Developing specialized explainable AI (XAI) techniques tailored to the unique characteristics of pathology FMs, enabling pathologists to understand and trust model predictions [21] [17].

Resource-Efficient Adaptation: Creating methods to adapt large FMs for clinical deployment in resource-constrained environments, including model compression, knowledge distillation, and efficient fine-tuning techniques.

Foundation models represent a paradigm shift in computational pathology, offering significant advantages over traditional deep learning approaches in terms of generalization, data efficiency, and multimodal capabilities. While traditional CNN-based models excel at specific tasks with sufficient labeled data, their specialized nature limits their scalability and adaptability across the diverse challenges of pathological diagnosis and research. Foundation models, built on transformer architectures and pre-trained through self-supervised learning on massive datasets, provide a versatile foundation that can be efficiently adapted to numerous downstream tasks with minimal fine-tuning. The emerging capabilities of these models in whole-slide representation learning, cross-modal understanding, and few-shot adaptation position them as transformative tools for advancing precision oncology and pathology research. However, challenges remain in computational requirements, explainability, and clinical validation that warrant continued research and development. As the field evolves, foundation models are poised to become indispensable components of the pathology research toolkit, enabling more accurate, efficient, and comprehensive analysis of histopathological data for drug development and clinical research.

How Pathology Foundation Models Work: Architectures and Real-World Applications

Computational pathology foundation models (CPathFMs) are large-scale deep learning models trained on vast amounts of unlabeled histopathology data using self-supervised learning (SSL) techniques [24]. Unlike traditional models that require extensive manual annotations for each specific task, foundation models learn general-purpose representations of tissue morphology that can be adapted to various downstream applications through transfer learning [25]. The emergence of CPathFMs represents a paradigm shift in digital pathology, enabling robust performance across diverse diagnostic tasks including tumor detection, subtyping, grading, and biomarker prediction [26] [27].

The development of effective CPathFMs faces significant challenges due to the inherent complexity of histopathological data. Whole-slide images (WSIs) are gigapixel-sized, present remarkable variability in tissue morphology, and exhibit differences in staining protocols and scanning equipment across institutions [24] [25]. Two core SSL techniques have proven particularly effective in addressing these challenges: contrastive learning and masked image modeling. These approaches enable models to learn rich, generalizable feature representations without relying on costly manual annotations, forming the technical foundation for next-generation computational pathology tools [26] [27] [25].

Technical Foundations

Self-Supervised Learning in Computational Pathology

Self-supervised learning has emerged as a foundational paradigm for pre-training CPathFMs by leveraging the inherent structure of unlabeled histopathological images [25]. SSL methods create supervisory signals directly from the data itself, bypassing the need for manual annotations that are expensive, time-consuming, and subject to inter-observer variability [24]. This approach is particularly valuable in computational pathology, where expert pathologist annotations are a scarce resource [25].

The SSL framework typically involves two phases: pre-training on large-scale unlabeled datasets to learn general visual representations, followed by fine-tuning on smaller labeled datasets for specific downstream tasks [25]. This paradigm has demonstrated remarkable success in natural image analysis and has been effectively adapted to the computational pathology domain [27] [25]. Among various SSL techniques, contrastive learning and masked image modeling have shown the most promise for histopathology image analysis due to their ability to capture both global and local tissue patterns [25].

Contrastive Learning Frameworks

Contrastive learning operates on the principle of measuring similarities and differences between data points [25]. In computational pathology, this typically involves maximizing agreement between differently augmented views of the same image while minimizing agreement with other images [28] [25]. Several specialized contrastive frameworks have been adapted for pathology image analysis:

DINO (self-Distillation with NO labels) employs a student-teacher paradigm where the student network learns to match the output of a teacher network after centering operations [25]. The teacher network is updated via an exponential moving average (EMA) of the student weights, providing stable training without labels [25]. This approach has been successfully scaled to million-image datasets in pathology, demonstrating strong representation learning capabilities [26] [27].

DINOv2 enhances DINO by integrating iBOT, which incorporates Masked Image Modeling (MIM) [25]. MIM randomly masks portions of input images and trains the model to reconstruct the masked areas, enabling the model to learn valuable representations by understanding both local cellular structures and broader tissue contexts [25]. This combination has proven particularly effective for histopathology applications, improving generalization across diverse pathology datasets [25].

Supervised Contrastive Learning (SCL) extends the contrastive framework to leverage available labels by pulling together samples from the same class while pushing apart samples from different classes [29]. HistopathAI implements this approach through a hybrid network that merges SCL strategies with cross-entropy loss, specifically tailored for imbalanced histopathology datasets [29].

Prototypical Contrastive Learning, as implemented in the SongCi model for forensic pathology, learns a set of prototype representations that capture both tissue-specific and cross-tissue features [30]. This approach distills redundant information from high-resolution WSIs into a lower-dimensional prototype space, enabling efficient representation of diverse tissue patterns [30].

Masked Image Modeling Architectures

Masked Image Modeling (MIM) has emerged as a powerful self-supervised pre-training strategy inspired by masked language modeling in natural language processing [25]. In MIM, random portions of input images are masked, and the model is trained to reconstruct the missing portions based on the surrounding context [25]. This approach forces the model to learn meaningful representations of tissue structures and their spatial relationships.

The Masked Autoencoder (MAE) architecture implements MIM through an asymmetric encoder-decoder design [25]. The encoder operates only on visible patches, making the process computationally efficient, while the lightweight decoder reconstructs the original image from the encoded representations and mask tokens [25]. For histopathology images, this approach enables models to learn hierarchical features spanning cellular morphology to tissue architecture.

iBOT integrates MIM with online tokenization, combining the benefits of masked reconstruction with the representation stability of contrastive learning [25]. This hybrid approach has been incorporated into DINOv2, which has served as the foundation for several state-of-the-art pathology models, including UNI and Virchow [26] [27] [25].

Key Methodologies and Implementations

Foundation Models in Computational Pathology

Recent years have witnessed the development of several pioneering foundation models that implement contrastive learning and MIM at unprecedented scales in computational pathology:

Virchow is a 632 million parameter Vision Transformer (ViT) model trained on approximately 1.5 million H&E-stained WSIs from 100,000 patients at Memorial Sloan Kettering Cancer Center using the DINOv2 algorithm [26]. This represents 4-10× more WSIs than prior training datasets in pathology [26]. The model employs a multiview student-teacher self-supervised approach that leverages both global and local regions of tissue tiles to learn embeddings of WSI tiles [26]. Virchow has demonstrated exceptional performance in pan-cancer detection, achieving 0.95 specimen-level area under the receiver operating characteristic curve (AUC) across nine common and seven rare cancers [26].

UNI is a general-purpose self-supervised model pretrained on the "Mass-100K" dataset comprising more than 100 million tissue patches from 100,426 diagnostic H&E WSIs across 20 major tissue types [27]. Using DINOv2 with a ViT-Large architecture, UNI was evaluated on 34 computational pathology tasks of varying diagnostic difficulty [27]. The model demonstrates capabilities such as resolution-agnostic tissue classification and slide classification using few-shot class prototypes, achieving superior performance in classifying up to 108 cancer types in the OncoTree classification system [27].

SongCi introduces a visual-language model specifically tailored for forensic pathology applications, leveraging advanced prototypical cross-modal self-supervised contrastive learning [30]. Pretrained on a multi-center dataset comprising over 16 million high-resolution image patches and 471 unique diagnostic outcomes, SongCi employs a prototypical contrastive learning strategy that distills WSI patches into a lower-dimensional prototype space [30]. The model then uses cross-modal contrastive learning to align image representations with textual descriptions of gross findings and diagnostic outcomes [30].

HistopathAI implements a hybrid network structure that merges supervised contrastive learning strategies with cross-entropy loss, specifically designed for imbalanced histopathology datasets [29]. The framework employs Hybrid Deep Feature Fusion (HDFF) to combine feature vectors from both EfficientNetB3 and ResNet50, creating comprehensive representations of histopathology images [29]. Using a stepwise methodology that transitions from feature learning to classifier learning, HistopathAI has achieved state-of-the-art classification accuracy across multiple public datasets [29].

Quantitative Comparison of Foundation Models

Table 1: Performance Comparison of Major Foundation Models on Pan-Cancer Detection

Model	Architecture	Pretraining Data	Pan-Cancer AUC	Rare Cancer AUC	Key Innovation
Virchow [26]	ViT (632M params)	1.5M WSIs, 100K patients	0.950	0.937	Largest pathology foundation model; DINOv2 training
UNI [27]	ViT-Large	100M patches, 100K WSIs	0.940	-	General-purpose model; 34 downstream tasks
HistopathAI [29]	EfficientNetB3 + ResNet50	7 public + 1 private dataset	State-of-art on all tested datasets	-	Supervised contrastive learning; hybrid feature fusion
SongCi [30]	Visual-language model	16M patches, 2,228 vision-language pairs	-	-	Prototypical cross-modal contrastive learning

Table 2: Technical Specifications of Major Foundation Model Training Approaches

Model	SSL Algorithm	Multi-modal	Batch Size	Training Iterations	Embedding Dimension
Virchow [26]	DINOv2	No	-	-	-
UNI [27]	DINOv2	No	-	50K-125K	-
SongCi [30]	Prototypical CL + Cross-modal CL	Yes (vision + language)	-	-	933 prototypes
CTransPath [27]	Contrastive Learning	No	-	-	-

Experimental Protocols and Methodologies

Model Pre-training with DINOv2

The DINOv2 self-supervised learning framework has been widely adopted for pre-training computational pathology foundation models [26] [27] [25]. The following protocol outlines the key methodological steps:

Data Preparation: Collect large-scale whole-slide images (WSIs) from diverse tissue types and preparation protocols. For Virchow, this involved 1.5 million H&E-stained WSIs from 100,000 patients, while UNI utilized 100,426 diagnostic H&E WSIs across 20 major tissue types [26] [27]. Extract tissue patches at multiple magnification levels (typically 20×, 10×, and 5×) to capture both cellular and architectural features.

Multi-crop Data Augmentation: Generate multiple augmented views for each patch using random combinations of transformations including color jittering, Gaussian blur, solarization, and random resized crops [25]. This creates the "student" and "teacher" views essential for the self-distillation process.

Self-Distillation Training: Implement the student-teacher framework where the student network learns to match the output distributions of the teacher network for different augmented views of the same image [25]. The teacher network parameters are updated via an exponential moving average (EMA) of the student parameters [25]. The training objective minimizes the cross-entropy loss between the student and teacher output distributions.

Multi-scale Feature Learning: Process image patches at multiple resolutions to capture both fine-grained cellular details and broader tissue architecture. This is particularly important in histopathology where diagnostic features span multiple scales [26].

Masked Image Modeling Integration: For DINOv2, incorporate masked patch modeling where random portions of input patches are masked and the model is trained to reconstruct the missing content [25]. This encourages the model to learn robust representations based on contextual understanding of tissue structures.

Prototypical Contrastive Learning for Forensic Pathology

The SongCi model introduces a specialized prototypical contrastive learning approach for forensic pathology applications [30]:

Prototype Learning: Each WSI is segmented into a collection of patches, and an image encoder extracts patch-level representations. These representations are projected into a low-dimensional space defined by shared prototypes across WSIs [30]. SongCi learned 933 prototypes using this self-supervised method.

Prototype Visualization and Analysis: Organize prototypes using dimensionality reduction techniques (UMAP) to identify both intra-tissue prototypes (encoding tissue-specific features) and inter-tissue prototypes (encoding shared histopathological features across organs) [30]. This enables the model to capture both specialized and generalizable patterns.

Cross-modal Alignment: Implement a gated-attention-boosted multi-modal block that integrates representations from paired WSI and gross key findings to align with forensic examination outcomes [30]. This creates a shared embedding space for visual and textual representations.

Zero-shot Inference: For unseen subjects, use gross key findings and corresponding WSIs to generate potential outcomes as textual queries. The model predicts final diagnostic results and provides explanatory factors highlighting critical elements associated with predictions [30].

Evaluation Methodologies for Foundation Models

Robust evaluation of computational pathology foundation models requires diverse tasks and datasets:

Pan-Cancer Detection: Evaluate model performance on detecting both common and rare cancers across various tissues [26]. Use specimen-level labels and measure area under the receiver operating characteristic curve (AUC) at both slide and specimen levels [26]. Include out-of-distribution data from external institutions to assess generalization capability.

Large-scale Multi-class Classification: Construct hierarchical cancer classification tasks following established oncology classification systems (e.g., OncoTree) [27]. Include both common and rare cancer types to evaluate model performance across diverse disease entities. Report top-K accuracy (K = 1, 3, 5), weighted F1 score, and AUROC performance metrics [27].

Few-shot and Zero-shot Learning: Assess model capability to adapt to new tasks with limited labeled examples [27]. Use class prototypes for prompt-based slide classification and evaluate performance with varying numbers of training examples [27].

Biomarker Prediction: Evaluate model performance on predicting molecular biomarkers from routine H&E images [26] [28]. This includes predicting genetic mutations, gene expression levels, and molecular subtypes based solely on morphological patterns in H&E-stained sections [28].

Performance Comparison and Scaling Laws

Foundation models in computational pathology demonstrate clear scaling laws where performance improves with increased model size, data diversity, and training duration [27].

Data Scaling: UNI demonstrated a +4.2% performance increase in top-1 accuracy when scaling from Mass-1K (1 million images) to Mass-22K (16 million images), and a further +3.7% increase when scaling to Mass-100K (100 million images) on the 43-class OncoTree cancer type classification task [27]. Similar trends were observed for the more challenging 108-class OncoTree code classification task [27].

Model Scaling: Comparing ViT-Base and ViT-Large architectures revealed that larger model architectures continue to benefit from increased data size, while smaller models may plateau in performance with very large datasets [27]. This highlights the importance of matching model capacity to dataset scale.

Algorithm Selection: DINOv2 consistently outperformed alternative self-supervised learning algorithms like MoCoV3 across various data scales and model architectures [27], establishing it as the current leading approach for pathology foundation models.

Table 3: Impact of Scaling on Model Performance (OncoTree Classification Tasks)

Model Scale	Data Scale	OT-43 Top-1 Accuracy	OT-108 Top-1 Accuracy	Training Iterations
ViT-L/ Mass-1K [27]	1M images, 1,404 WSIs	Baseline	Baseline	50,000
ViT-L/ Mass-22K [27]	16M images, 21,444 WSIs	+4.2%	+3.5%	50,000
ViT-L/ Mass-100K [27]	100M images, 100,426 WSIs	+7.9%	+6.5%	125,000

The Scientist's Toolkit

Table 4: Essential Research Reagents and Computational Resources for Pathology Foundation Models

Resource Category	Specific Tools/Components	Function/Purpose	Examples/Notes
Model Architectures	Vision Transformer (ViT) [26] [27]	Base architecture for processing image patches using self-attention mechanisms	Scalable to hundreds of millions of parameters
	Convolutional Neural Networks (CNNs) [29]	Alternative backbone for feature extraction, often used in hybrid approaches	EfficientNetB3, ResNet50 in HistopathAI [29]
SSL Frameworks	DINO/DINOv2 [26] [27] [25]	Self-distillation with no labels; combines contrastive learning with masked image modeling	Used in Virchow, UNI, and other state-of-the-art models
	Prototypical Contrastive Learning [30]	Learns prototype representations for efficient encoding of diverse tissue patterns	Implemented in SongCi for forensic pathology
	Supervised Contrastive Learning (SCL) [29]	Leverages available labels to improve feature separation in embedding space	Used in HistopathAI for imbalanced datasets
Data Processing Tools	WSInfer [31]	Toolbox for deep learning model deployment on whole-slide images	Provides end-to-end workflow for patch extraction and inference
	QuPath [31]	Open-source platform for digital pathology image analysis	Used for visualization of model predictions as heatmaps
Computational Resources	High-Performance GPUs [31]	Accelerate training of large foundation models	AMD Radeon Instinct MI210 (64GB RAM) or similar [31]
	Network-Attached Storage (NAS) [31]	Store and manage large whole-slide image collections	Qnap NAS or similar systems with high-speed connectivity

Foundation models (FMs) in computational pathology are large-scale artificial intelligence models pre-trained on vast datasets of histopathology images and, in some cases, associated text. These models learn universal feature representations that can be adapted to diverse downstream diagnostic tasks with minimal additional training, thereby addressing critical challenges such as data imbalance and heavy annotation dependency in medical AI [32] [12]. The development of these models is primarily organized around three distinct architectural paradigms: vision-only encoders that process image data alone, vision-language models (VLMs) that align visual and textual information, and whole-slide encoders designed to handle gigapixel whole-slide images (WSIs). Each architecture offers unique capabilities and addresses different aspects of the computational pathology workflow, from basic morphological analysis to integrated diagnostic reporting [33] [32].

Vision-Only Encoders: Learning from Images Alone

Core Architecture and Pre-training Methodology

Vision-only encoders are foundational components trained exclusively on histopathology images without textual supervision. These models typically employ self-supervised learning (SSL) objectives on large collections of unlabeled image patches, learning to capture salient morphological patterns in tissue structures [6] [4]. The pre-training process often utilizes techniques such as masked image modeling and knowledge distillation, which force the model to learn robust feature representations by predicting missing parts of images or distilling knowledge from a teacher network to a student network [4]. One prominent example is the UNI model, a state-of-the-art vision encoder pre-trained on over 100 million histology image patches from more than 100,000 whole-slide images using self-supervised learning [34]. This extensive pre-training enables the model to develop a comprehensive understanding of cellular and tissue-level morphology across diverse disease states and tissue types.

The typical workflow for these models involves processing input images divided into smaller patches, converting them into feature embeddings through a convolutional neural network (CNN) or Vision Transformer (ViT) backbone, and then applying SSL objectives to learn meaningful representations. For instance, the iBOT framework employs masked image modeling and knowledge distillation to create powerful feature extractors that can be transferred to various downstream tasks [4]. This approach has proven highly effective for capturing the intricate visual patterns present in histopathology images, from cellular atypia to complex tissue architecture.

Key Experimental Protocols and Evaluation

Vision-only encoders are typically evaluated through linear probing, where a simple classifier is trained on top of frozen features extracted by the pre-trained encoder, and fine-tuning, where the entire model is adapted to specific tasks [6] [35]. This evaluation methodology tests the quality and generalizability of the learned representations. For example, PathOrchestra, a comprehensive vision-only foundation model, was trained on 287,424 slides from 21 tissue types across three centers and evaluated on 112 tasks encompassing digital slide preprocessing, pan-cancer classification, lesion identification, multi-cancer subtype classification, biomarker assessment, gene expression prediction, and structured report generation [5]. The model demonstrated remarkable performance, achieving over 0.950 accuracy in 47 tasks, including challenging domains like lymphoma subtyping and bladder cancer screening [5].

Table 1: Performance of PathOrchestra on Select Pan-Cancer Classification Tasks

Task Description	Dataset	AUC	Accuracy	F1 Score
17-class Tissue Classification	In-house FFPE	0.988	0.879	0.863
32-class Classification	TCGA FFPE	0.964	0.666	0.667
32-class Classification	TCGA Frozen	0.950	0.577	0.577

Recent benchmarking studies have comprehensively evaluated vision-only models against other architectures. One large-scale assessment of 31 AI foundation models revealed that pathology-specific vision models (Path-VM) often outperform both pathology-specific vision-language models (Path-VLM) and general vision models, securing top rankings across diverse tasks [35]. Notably, Virchow2, a pathology foundation model, delivered the highest performance across TCGA, CPTAC, and external tasks in these evaluations, highlighting the effectiveness of specialized vision-only architectures in diverse histopathological applications [35].

Vision-Language Models: Bridging Images and Text

Architectural Framework and Alignment Techniques

Vision-language models (VLMs) represent a significant advancement in computational pathology by integrating visual processing with natural language understanding. These models are designed to align histopathology images with corresponding textual descriptions, enabling capabilities such as zero-shot classification, cross-modal retrieval, and natural language interaction [6] [34]. The core architecture typically consists of three key components: an image encoder that processes visual inputs, a text encoder that handles linguistic information, and a multimodal fusion mechanism that integrates both modalities into a shared representation space [6].

CONCH (CONtrastive learning from Captions for Histopathology) exemplifies this approach, employing a framework based on CoCa, a state-of-the-art visual-language foundation pre-training framework [6]. The model uses an image encoder, a text encoder, and a multimodal fusion decoder, trained using a combination of contrastive alignment objectives that seek to align the image and text modalities in the model's representation space and a captioning objective that learns to predict the caption corresponding to an image [6]. This dual-objective approach enables the model to not only understand the relationship between visual patterns and textual descriptions but also generate coherent captions for histopathological findings.

Diagram 1: Vision-Language Model Architecture in Computational Pathology

Experimental Protocols and Performance Benchmarking

VLMs are typically evaluated through zero-shot transfer learning, where the pre-trained model is applied to downstream tasks without any task-specific fine-tuning [6]. This evaluation approach tests the model's ability to generalize to new tasks and datasets based solely on its pre-trained knowledge. The experimental protocol involves representing class names using predetermined text prompts, with each prompt corresponding to a class. An image is then classified by matching it with the most similar text prompt in the model's shared image-text representation space [6].

CONCH has demonstrated state-of-the-art performance across multiple benchmarks, achieving a zero-shot accuracy of 90.7% for non-small cell lung cancer (NSCLC) subtyping and 90.2% for renal cell carcinoma (RCC) subtyping, outperforming the next-best model by 12.0% and 9.8%, respectively [6]. On the more challenging breast carcinoma (BRCA) subtyping task, CONCH achieved 91.3% accuracy, while other models performed at near-random chance levels [6]. These results highlight the powerful capabilities of VLMs in recognizing and distinguishing complex histopathological patterns without task-specific training.

Table 2: Zero-shot Classification Performance of CONCH on Slide-Level Tasks

Task	Dataset	CONCH Accuracy	Next-Best Model Accuracy	Performance Gap
NSCLC Subtyping	TCGA NSCLC	90.7%	78.7% (PLIP)	+12.0%
RCC Subtyping	TCGA RCC	90.2%	80.4% (PLIP)	+9.8%
BRCA Subtyping	TCGA BRCA	91.3%	55.3% (BiomedCLIP)	+36.0%

Another innovative approach in this domain is PathChat, a multimodal generative AI copilot built by fine-tuning a visual-language model on over 456,000 diverse visual-language instructions consisting of 999,202 question and answer turns [34]. When evaluated on multiple-choice diagnostic questions from cases with diverse tissue origins and disease models, PathChat achieved state-of-the-art performance, with accuracy improving from 78.1% in image-only settings to 89.5% when clinical context was provided [34]. This demonstrates the significant value of integrating multimodal information in pathology AI systems.

Whole-Slide Encoders: Processing Gigapixel Images

Architectural Innovations for Whole-Slide Analysis

Whole-slide encoders represent a specialized architectural paradigm designed to handle the unique challenges of processing gigapixel whole-slide images (WSIs). These models address the computational complexity of analyzing extremely high-resolution images while capturing both local morphological details and global tissue architecture [4]. Unlike patch-based approaches that process individual image regions independently, whole-slide encoders aim to model long-range dependencies and spatial relationships across entire slides, enabling more comprehensive histopathological analysis.

TITAN (Transformer-based pathology Image and Text Alignment Network) exemplifies this architectural approach, employing a Vision Transformer (ViT) that creates general-purpose slide representations [4]. The model is pretrained on 335,645 whole-slide images using a three-stage strategy: (1) vision-only unimodal pretraining on region-of-interest (ROI) crops, (2) cross-modal alignment of generated morphological descriptions at the ROI-level (423k pairs of 8k×8k ROIs and captions), and (3) cross-modal alignment at the WSI-level (183k pairs of WSIs and clinical reports) [4]. To handle computational complexity, TITAN uses a novel approach of dividing each WSI into non-overlapping patches of 512×512 pixels at 20× magnification, extracting 768-dimensional features for each patch, and then processing these features through the transformer architecture.

Experimental Framework and Capabilities

The evaluation of whole-slide encoders typically involves assessing their performance on slide-level tasks such as cancer subtyping, biomarker prediction, outcome prognosis, and slide retrieval, particularly in low-data regimes and for rare cancers [4]. These models are especially valuable in scenarios with limited training data, as they can leverage their comprehensive pre-training to make accurate predictions without extensive fine-tuning. TITAN, for instance, has demonstrated strong performance across diverse clinical tasks, outperforming both region-of-interest (ROI) and slide foundation models in machine learning settings including linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval, and pathology report generation [4].

A critical innovation in whole-slide encoders is their ability to generate pathology reports without any fine-tuning or requiring clinical labels. This capability stems from their multimodal pretraining, which aligns visual patterns with textual descriptions at both the regional and whole-slide levels [4]. By learning the relationship between morphological features and their corresponding diagnostic descriptions, these models can generate coherent clinical reports that accurately summarize histopathological findings, demonstrating a significant step toward automated pathology analysis and reporting.

Diagram 2: Whole-Slide Encoder Processing Workflow

Table 3: Key Research Reagents and Computational Resources for Pathology Foundation Models

Resource Category	Specific Examples	Function and Application
Pathology Datasets	TCGA (The Cancer Genome Atlas), CPTAC, CAMELYON, DigestPath	Provide diverse, annotated whole-slide images for model training and validation across multiple cancer types and tissue sites [5] [6].
Pre-trained Patch Encoders	CONCH, UNI, CTransPath	Serve as feature extractors for whole-slide images, converting image patches into meaningful morphological representations [6] [4].
Multiple Instance Learning Frameworks	CLAM (Clustering-constrained Attention Multiple-instance Learning)	Enable weakly supervised learning from slide-level labels without manual region-of-interest annotation [36].
Vision-Language Alignment Tools	PLIP, BiomedCLIP	Facilitate alignment between histopathology images and textual descriptions for zero-shot learning and retrieval tasks [6].
Whole-Slide Processing Libraries	OpenSlide, HistomicsUI	Enable efficient handling and processing of gigapixel whole-slide images for large-scale analysis [36].
Synthetic Data Generators	PathChat-based caption generation	Create synthetic image-caption pairs to augment training data and enhance model generalization [4].

Comparative Analysis and Future Directions

The three architectural paradigms for pathology foundation models each offer distinct advantages and face unique challenges. Vision-only encoders excel at learning rich morphological representations from vast image collections but lack the semantic understanding provided by language alignment. Vision-language models enable powerful zero-shot capabilities and natural language interaction but require carefully curated image-text pairs for training. Whole-slide encoders address the computational challenges of processing gigapixel images while capturing slide-level context but represent the most complex and resource-intensive approach.

Recent benchmarking studies reveal that model size and data size do not consistently correlate with improved performance in pathology foundation models, challenging assumptions about scaling in histopathological applications [35]. Instead, factors such as data diversity, pre-training objectives, and architectural specialization appear to be more critical determinants of model performance. Fusion models that integrate top-performing foundation models have demonstrated superior generalization across external tasks and diverse tissues, suggesting that hybrid approaches may offer the most promising path forward [35].

Future research directions in pathology foundation models include developing more efficient architectures that can handle the computational demands of whole-slide analysis, improving multimodal alignment techniques to better capture the nuances of histopathological diagnosis, and creating more sophisticated evaluation frameworks that assess clinical utility beyond technical metrics [32] [35]. As these models continue to evolve, they hold the potential to transform pathology practice by enhancing diagnostic accuracy, enabling personalized treatment strategies, and democratizing access to expert-level pathological analysis.

Foundation models are large-scale deep neural networks trained on vast datasets using self-supervised learning algorithms, generating versatile feature representations (embeddings) that generalize across diverse predictive tasks without task-specific training [37] [38]. In computational pathology, these models address critical limitations of traditional task-specific approaches, which require extensive labeled datasets and struggle with rare conditions and open-set identification [38]. The transition from task-specific models to foundation models represents a paradigm shift, enabling more robust and generalizable artificial intelligence (AI) tools for clinical diagnostics and research [11].

Foundation models in pathology are typically trained on hundreds of thousands to millions of whole-slide images (WSIs), learning to capture a comprehensive spectrum of histomorphological patterns including cellular morphology, tissue architecture, nuclear features, and tumor microenvironment characteristics [26] [37]. Their value is particularly pronounced in pan-cancer detection and rare cancer diagnosis, where they can identify subtle morphological patterns that may elude human observation or traditional computational methods [26] [38]. By learning from massive-scale multimodal data, these models achieve unprecedented performance in classifying cancer types, predicting biomarkers, and identifying rare malignancies, thereby advancing precision oncology [38].

Technical Architecture of Pathology Foundation Models

Model Architectures and Pretraining Strategies

Current pathology foundation models employ diverse architectural frameworks and training methodologies. The Virchow model utilizes a 632 million parameter Vision Transformer (ViT) trained using the DINO v2 algorithm on approximately 1.5 million H&E-stained WSIs [26]. This self-supervised approach leverages both global and local regions of tissue tiles to learn hierarchical representations of histopathological features [26]. The TITAN (Transformer-based pathology Image and Text Alignment Network) framework introduces a multimodal architecture pretrained on 335,645 WSIs through a three-stage process: (1) visual self-supervised learning on region-of-interest (ROI) crops, (2) cross-modal alignment with synthetic fine-grained morphological descriptions (423,122 caption-ROI pairs), and (3) cross-modal alignment at the whole-slide level with clinical reports [4] [39]. The UNI model employs a self-supervised learning approach pretrained on more than 100 million images from over 100,000 diagnostic H&E-stained WSIs across 20 major tissue types [40].

These models share a common capability to process gigapixel WSIs by dividing them into smaller patches or tiles, encoding them into feature representations, and aggregating these features to generate slide-level predictions. The most advanced models incorporate transformer architectures to capture long-range dependencies within tissue structures, enabling more comprehensive understanding of tissue microenvironment organization [4] [26].

Whole-Slide Image Processing Workflow

The following diagram illustrates the technical workflow for processing whole-slide images in foundation models:

Diagram: Whole-Slide Image Processing in Foundation Models

Pan-Cancer Detection Performance and Evaluation

Experimental Protocol for Pan-Cancer Detection

The evaluation of foundation models for pan-cancer detection follows a standardized protocol involving large-scale multimodal datasets. In the Virchow model evaluation, the pan-cancer detection model was trained using specimen-level labels across multiple cancer types [26]. The model infers cancer presence using Virchow embeddings as input to a weakly supervised aggregator model that groups tile embeddings to generate slide-level predictions [26]. Performance is assessed on slides from both internal institutions and external consultation cases to evaluate generalizability across diverse populations and scanner types [26].

For the TITAN model, evaluation encompasses diverse clinical tasks including linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval, and pathology report generation [4]. The model is tested on resource-limited clinical scenarios to assess its robustness in real-world settings where labeled data may be scarce, particularly for rare conditions [4] [39]. UNI model evaluation spans 34 representative computational pathology tasks of varying diagnostic difficulty, demonstrating capabilities in resolution-agnostic tissue classification, few-shot slide classification using class prototypes, and disease subtyping generalization across up to 108 cancer types in the OncoTree classification system [40].

Quantitative Performance Comparison

Table 1: Pan-Cancer Detection Performance of Foundation Models

Model	Training Data Scale	Architecture	Overall AUC	Rare Cancer AUC	Key Capabilities
Virchow	1.5M WSIs from ~100k patients	632M parameter ViT	0.950	0.937	Pan-cancer detection across 9 common and 7 rare cancers [26]
TITAN	335,645 WSIs + 423K synthetic captions	Multimodal Transformer	Not specified	Superior rare cancer retrieval	Zero-shot classification, cross-modal retrieval, report generation [4] [39]
UNI	100M images from 100K+ WSIs	Self-supervised encoder	0.940 (comparative)	Not specified	Resolution-agnostic classification, few-shot learning, 108 cancer type classification [40]
CTransPath	Not specified in results	Convolutional Transformer	0.907	Not specified	Baseline comparison [26]

Table 2: Rare Cancer Detection Performance of Virchow Model

Cancer Type	Virchow AUC	UNI AUC	Phikon AUC	CTransPath AUC	Incidence Category
Cervical Cancer	0.875	0.830	0.810	0.753	Rare [26]
Bone Cancer	0.841	0.813	0.822	0.728	Rare [26]
All Rare Cancers (Aggregate)	0.937	Not specified	Not specified	Not specified	7 rare types combined [26]

Rare Cancer Diagnosis Capabilities

Technical Approaches to Rare Cancer Diagnosis

Foundation models address the significant challenge of rare cancer diagnosis, where limited training data traditionally hinders AI model development. These models leverage their extensive pretraining on diverse tissue types to recognize subtle morphological patterns indicative of rare malignancies [26] [38]. The Virchow model demonstrates particularly strong performance on rare cancers (aggregate AUC of 0.937), achieving clinically relevant detection rates for cancers with annual incidence below 15 per 100,000 people [26]. This capability stems from the model's exposure to a million-image-scale dataset encompassing both common and rare tissue morphologies [26].

The TITAN model enhances rare cancer diagnosis through its multimodal architecture and synthetic data augmentation [4]. By incorporating 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology, the model learns fine-grained visual-language correspondences that improve its ability to recognize and describe rare histological findings [4] [39]. This approach is particularly valuable in resource-limited clinical scenarios where examples of rare conditions may be insufficient for training traditional models [4]. The model's cross-modal retrieval capabilities enable clinicians to search for similar cases based on either image content or textual descriptions, facilitating diagnosis of challenging rare cases [4].

Cross-Modal Retrieval for Rare Cancers

The following diagram illustrates how multimodal foundation models enable rare cancer diagnosis through cross-modal retrieval:

Diagram: Cross-Modal Retrieval for Rare Cancer Diagnosis

Table 3: Essential Research Reagents and Computational Resources for Pathology Foundation Models

Resource Category	Specific Components	Function/Application	Examples from Literature
Histopathology Data	H&E-stained Whole Slide Images (WSIs)	Foundation model pretraining and evaluation	1.5M WSIs (Virchow) [26]; 335,645 WSIs (TITAN) [4]
Clinical Annotations	Pathology reports, diagnostic labels, molecular biomarkers	Supervised fine-tuning and model validation	182,862 medical reports (TITAN) [4]; OncoTree cancer classification [40]
Synthetic Data	AI-generated captions and morphological descriptions	Data augmentation for rare conditions	423,122 synthetic captions from PathChat copilot (TITAN) [4] [39]
Computational Infrastructure	High-performance GPUs/TPUs, distributed training frameworks	Model pretraining and inference	DINO v2 algorithm (Virchow) [26]; iBOT framework (TITAN) [4]
Evaluation Frameworks	Multiple dataset benchmarks, OOD validation sets	Performance assessment and generalization testing	34 CPath tasks (UNI) [40]; rare cancer retrieval (TITAN) [4]

Foundation models represent a transformative advancement in computational pathology, enabling robust pan-cancer detection and accurate diagnosis of rare malignancies. By learning from massive-scale histopathology datasets, these models capture the complex morphological patterns necessary for generalizable cancer diagnosis across diverse tissue types and disease presentations. The integration of multimodal data, including pathology images, clinical reports, and synthetic captions, further enhances their diagnostic capabilities and facilitates novel applications such as cross-modal retrieval and report generation.

As these models continue to evolve, future research directions include developing more efficient architectures for processing gigapixel whole-slide images, improving model interpretability for clinical adoption, and enhancing multimodal reasoning capabilities for comprehensive pathology analysis. The exceptional performance of foundation models on both common and rare cancers highlights their potential to significantly impact clinical practice and precision oncology research.

Foundation models in computational pathology represent a paradigm shift, moving from task-specific artificial intelligence (AI) tools to versatile models trained on vast datasets of whole slide images (WSIs). These models leverage self-supervised learning to develop a deep understanding of tissue morphology, which can then be adapted to a wide range of downstream clinical and research tasks with minimal additional training. This whitepaper details how these models are enabling advanced capabilities in biomarker prediction, patient prognosis, and the generation of structured pathology reports. We provide a technical examination of the underlying methodologies, present quantitative performance benchmarks, outline key experimental protocols, and catalog the essential reagents and tools that form the scientist's toolkit for this rapidly evolving field.

Computational pathology applies AI to digitized WSIs to support disease diagnosis, characterization, and understanding. Traditional approaches relied on training individual deep learning models for specific tasks, such as cancer grading or cell segmentation, which required large, expensively annotated datasets and often resulted in limited generalizability [38]. Foundation models overcome these limitations by pretraining on extremely large and diverse corpora of WSIs—often hundreds of thousands to millions of slides—using self-supervised learning (SSL) algorithms that do not require manual labels [26] [2]. This process produces powerful, general-purpose feature representations (embeddings) that capture a wide spectrum of morphological patterns, from cellular details to tissue architecture.

Once trained, these foundational encoders can be efficiently adapted (e.g., via linear probing or fine-tuning) to a multitude of downstream tasks with limited labeled data. This versatility is particularly valuable in oncology for applications such as predicting molecular biomarkers directly from routine hematoxylin and eosin (H&E)-stained images, stratifying patient risk, and generating diagnostic reports, thereby accelerating the pace of precision oncology and drug development [38] [2].

Quantitative Performance Benchmarks

The clinical utility of pathology foundation models is demonstrated through rigorous validation on diverse tasks. The tables below summarize their state-of-the-art performance in biomarker prediction and prognosis.

Table 1: Performance of Foundation Models in Biomarker Prediction

Foundation Model	Task (Cancer Type)	Biomarker / Alteration	Performance (AUC)	Data Source
Johnson & Johnson MIA:BLC-FGFR [41]	Bladder Cancer	FGFR Alterations	0.80 - 0.86	H&E WSI
PathOrchestra [5]	Pan-Cancer	Gene Expression Prediction	>0.950 (Accuracy in 47/112 tasks)	Multi-center H&E WSI
TITAN [4]	Pan-Cancer	Biomarker Prediction	Outperformed baseline models	H&E WSI & Reports
Virchow [26]	Pan-Cancer	Biomarker Prediction	Generally outperformed other models	H&E WSI

Table 2: Performance of Foundation Models in Prognosis and Classification

Foundation Model	Task	Cancer Type / Context	Performance	Key Finding
CAPAI Biomarker [41]	Risk Stratification	Stage III Colon Cancer	35% vs. 9% 3-year recurrence risk	Identified high-risk ctDNA-negative patients
Artera MMAI Model [41]	Metastasis Prediction	Prostate Cancer (Post-RP)	18% vs. 3% 10-year risk (High vs. Low)	Combined H&E images & clinical variables
PathOrchestra [5]	Pan-Cancer Classification	17 Cancers (In-house FFPE)	Average AUC: 0.988
PathOrchestra [5]	Lymphoma Subtyping	Lymphoma	Accuracy > 0.950
Virchow [26]	Pan-Cancer Detection	9 Common & 7 Rare Cancers	Specimen-level AUC: 0.950	Achieved 0.937 AUC on rare cancers
PLUTO-4G [42]	Dermatopathology Diagnosis	Skin Cancer	11% improvement (vs. benchmarks)	Macro F1: 67.1%

Methodologies and Experimental Protocols

Multimodal Pretraining of Foundation Models

The power of foundation models stems from their large-scale pretraining. The following workflow illustrates a state-of-the-art approach that integrates visual and linguistic data to create a highly versatile model.

Diagram 1: Multimodal foundation model pretraining.

Protocol Explanation:

• Stage 1 (Vision-Only SSL): Models like Virchow and PLUTO-4 use algorithms such as DINOv2 to train a vision transformer (ViT) on millions of tissue patches extracted from WSIs. This teaches the model to recognize fundamental histopathological features without any labels [26] [42]. For example, Virchow was trained on 1.5 million WSIs using DINOv2 [26].
• Stage 2 (ROI-Text Alignment): To ground the model in language, region-of-interest (ROI) images are paired with detailed morphological captions. TITAN, for instance, used 423,122 synthetic captions generated by a generative AI copilot (PathChat) to align visual features with descriptive text at a local level [4].
• Stage 3 (WSI-Report Alignment): Finally, the model is trained to align entire WSIs with their corresponding pathology reports. TITAN performed this cross-modal alignment using 183,000 pairs of WSIs and clinical reports, enabling slide-level language understanding [4].

This multi-stage process results in a foundation model that can not only extract powerful visual features but also understand and generate language, enabling tasks like structured report generation and zero-shot classification.

Protocol for Developing a Prognostic Biomarker

A common application is developing an AI-based prognostic score from H&E images. The following workflow outlines the key steps, as seen in models like the CAPAI biomarker for colon cancer [41].

Diagram 2: Prognostic model development workflow.

Experimental Steps:

• Dataset Curation: Assemble a cohort of WSIs (e.g., from resection specimens) with associated long-term clinical outcome data, such as overall survival, disease-free survival, or metastasis. The cohort should be split into training, validation, and hold-out test sets [41] [26].
• Feature Extraction: Process each WSI by dividing it into smaller patches (e.g., 256x256 pixels). Pass these patches through a pretrained foundation model (e.g., Virchow, PLUTO-4, TITAN) to extract a feature vector for each patch [26].
• Slide-Level Aggregation: Use a weakly supervised aggregation method, such as Attention-Based Multiple Instance Learning (ABMIL), to combine the thousands of patch-level features from a single slide into a comprehensive slide-level representation. This step identifies which regions of the slide are most predictive of the outcome [5] [26].
• Multimodal Integration (Optional): For enhanced performance, integrate the slide-level features with other clinical variables (e.g., patient age, pathological stage, PSA levels) or genomic data (e.g., ctDNA status) [41]. The Artera model for prostate cancer is a prime example, combining H&E image features with clinical data [41].
• Model Training and Validation: Train a prognostic model (e.g., a Cox proportional hazards model or a classifier) on the training set using the aggregated features. The model is validated and tested on the held-out sets to ensure its generalizability and to establish its performance in stratifying patient risk [41] [26].

The Scientist's Toolkit: Research Reagents & Essential Materials

Table 3: Key Resources for Pathology Foundation Model Research

Category / Reagent	Specific Examples	Function & Application
Foundation Models	Virchow [26], PLUTO-4 [42], TITAN [4], PathOrchestra [5], UNI [26]	Pretrained encoders providing core feature extraction capabilities for diverse downstream tasks.
SSL Algorithms	DINOv2 [26], iBOT [4]	Self-supervised learning frameworks used for vision-only pretraining without labeled data.
Data Resources	The Cancer Genome Atlas (TCGA) [5], Multi-institutional private cohorts [5] [42]	Large-scale, diverse sources of WSIs and associated data for model training and validation.
Model Architectures	Vision Transformer (ViT) [4] [26], Attention-Based MIL (ABMIL) [5]	Neural network backbones and aggregation methods for processing gigapixel WSIs.
Stains	Hematoxylin & Eosin (H&E) [5] [26], Immunohistochemistry (IHC) panels [42]	Standard and special stains for tissue preparation; H&E is the primary input for most models.
Scanner Systems	Aperio, Philips, Ventana, Hamamatsu [42]	Whole-slide scanners for digitizing glass slides into WSIs.

Foundation models are fundamentally reshaping the landscape of computational pathology. By serving as versatile and powerful starting points for a wide array of applications, they are overcoming historical bottlenecks related to data annotation and model generalizability. Their demonstrated success in predicting biomarkers from routine H&E stains, providing robust patient prognostication, and even generating structured reports, underscores their immense potential to augment the capabilities of researchers and pathologists. As these models continue to evolve in scale and sophistication, integrating ever more data modalities, they are poised to become an indispensable engine for discovery and precision in oncology research and drug development.

Implementing Foundation Models: Overcoming Data, Computational, and Workflow Challenges

Addressing Data Scarcity and Annotation Costs in Niche Applications

Computational pathology foundation models (CPathFMs) represent a transformative class of artificial intelligence systems pretrained on extensive histopathology datasets using self-supervised learning (SSL) to extract robust feature representations from unlabeled whole-slide images (WSIs) [25]. These models serve as versatile "foundations" that can be adapted to diverse downstream tasks such as diagnosis, biomarker prediction, and prognosis with minimal task-specific labeling [38]. However, their development and application face significant challenges in niche clinical scenarios, including rare diseases, uncommon cancer subtypes, and specialized molecular alterations, where extensive annotated datasets are practically unavailable.

The fundamental limitation in these niche applications stems from the prohibitive costs and expertise requirements for large-scale data annotation in histopathology. Expert pathologists must manually review gigapixel WSIs to identify subtle morphological patterns, creating a critical bottleneck for traditional supervised learning approaches [25]. This challenge is particularly acute for rare conditions, where multi-institutional data collection is complicated by privacy concerns, tissue availability, and technical variability across institutions [4] [43]. Foundation models address these constraints through novel methodologies that maximize information extraction from limited data resources, thereby enabling robust AI applications even in data-scarce environments.

Technical Strategies to Overcome Data Limitations

Leveraging Unlabeled Data Through Self-Supervised Learning

Self-supervised learning has emerged as the cornerstone paradigm for developing CPathFMs without extensive manual annotations. SSL frameworks enable models to learn rich visual representations by formulating pretext tasks that generate supervisory signals directly from the intrinsic structure of unlabeled histopathology data [25] [44]. The most effective SSL approaches include:

• Masked Image Modeling (MIM): Methods like iBOT randomly mask portions of histology images and train models to reconstruct the missing visual content, enabling learning of contextual relationships in tissue microenvironments [4] [25]. This approach has been successfully implemented in models including Phikon and TITAN, demonstrating exceptional performance in capturing fine-grained morphological patterns essential for rare disease diagnosis [4] [44].

• Knowledge Distillation and Self-Distillation: Frameworks such as DINO employ a teacher-student architecture where both networks process different augmented views of the same image. The student network is trained to match the output distribution of the teacher, which is updated via exponential moving average of the student weights [25] [44]. This self-distillation mechanism encourages the model to learn invariant features across tissue variations, improving generalization to unseen rare conditions.

• Contrastive Learning: Methods including MoCo and SimCLR maximize agreement between differently augmented views of the same image while minimizing similarity with other images in the dataset [44]. CTransPath enhances this approach by selecting positive instances with similar visual information from a memory bank, effectively expanding the learning signal from limited data [44].

Table 1: Representative Self-Supervised Learning Methods in Computational Pathology

Method	Core Mechanism	Representative Models	Handles Data Scarcity Via
iBOT	Masked image modeling with online tokenizer	TITAN, Phikon	Reconstruction of masked tissue regions to learn contextual features
DINO/DINOv2	Self-distillation with different augmentations	UNI, Lunit	Learning invariant features across tissue variations
Contrastive Learning	Positive-negative sample discrimination	CTransPath	Multi-scale similarity learning from unlabeled data
CLIP-style Alignment	Image-text representation alignment	TITAN (multimodal)	Leveraging paired images and medical reports

Multimodal Integration for Enhanced Representation Learning

Multimodal foundation models substantially enhance capabilities in data-scarce scenarios by integrating complementary information sources that provide additional supervisory signals [4] [25]. The TITAN model exemplifies this approach through its three-stage training paradigm:

• Vision-only pretraining on 335,645 WSIs using self-supervised learning to capture fundamental histological patterns [4]
• ROI-level vision-language alignment with 423,122 synthetic captions generated using PathChat, a multimodal generative AI copilot for pathology [4]
• Slide-level cross-modal alignment with 182,862 pathology reports to connect whole-slide visual patterns with diagnostic descriptions [4]

This multimodal approach enables zero-shot capabilities where the model can recognize rare conditions without explicit training examples by leveraging textual descriptions and cross-modal reasoning [4]. For instance, TITAN can retrieve rare cancer cases and generate pathology reports without fine-tuning, demonstrating exceptional utility for niche applications where training data is severely limited [4].

Efficient Adaptation Strategies for Downstream Tasks

When applying foundation models to niche applications with limited annotated data, adaptation strategy selection critically impacts performance. Recent benchmarking studies have systematically evaluated various approaches across multiple pathology tasks [44] [45]:

Table 2: Performance of Adaptation Strategies in Data-Limited Scenarios

Adaptation Method	Mechanism	Data Efficiency	Best-Suited Scenarios
Linear Probing	Trains only final classification layer	High	Rapid prototyping, extreme data scarcity (<50 samples)
Parameter-Efficient Fine-Tuning (PEFT)	Updates small subset of parameters (e.g., adapters, LoRA)	High	Preserving general knowledge while adapting to new tasks
Partial Fine-Tuning	Updates only later layers of network	Medium	Task-specific adaptation with hundreds of samples
Full Fine-Tuning	Updates all model parameters	Low	When sufficient task-specific data exists (>1,000 samples)
Few-Shot Learning	Uses minimal data with specialized algorithms	Very High	Extreme data scarcity (1-20 samples per class)

Benchmarking results demonstrate that parameter-efficient fine-tuning approaches provide the optimal balance between performance and data efficiency for adapting pathology-specific foundation models to diverse datasets within the same classification tasks [44] [45]. In scenarios with extremely limited data (fewer than 20 samples per class), methods that modify only the testing phase (such as certain few-shot learning approaches) typically outperform more extensive fine-tuning [45].

Experimental Protocols and Validation Frameworks

Protocol 1: Cross-Modal Pretraining for Rare Cancer Retrieval

The TITAN model development protocol exemplifies a comprehensive approach to addressing data scarcity in niche applications [4]:

Dataset Curation and Preparation

• Collect 335,645 WSIs across 20 organ types (Mass-340K dataset) to ensure morphological diversity
• Generate 423,122 synthetic fine-grained region-of-interest (ROI) captions using PathChat, a multimodal generative AI copilot for pathology
• Include 182,862 paired pathology reports for slide-level alignment

Model Architecture Specifications

• Implement Vision Transformer (ViT) backbone for slide-level encoding
• Process WSIs by dividing into non-overlapping 512×512 pixel patches at 20× magnification
• Extract 768-dimensional features for each patch using CONCHv1.5 patch encoder
• Employ Attention with Linear Biases (ALiBi) for handling variable WSI sizes and long-range context modeling

Training Procedure

• Stage 1: Vision-only self-supervised pretraining using iBOT framework on ROI crops
• Stage 2: Cross-modal alignment of generated morphological descriptions at ROI-level
• Stage 3: Cross-modal alignment at WSI-level with clinical reports

Validation Metrics

• Zero-shot classification accuracy on rare cancer subtypes
• Cross-modal retrieval performance (slide-to-text and text-to-slide)
• Few-shot learning capability with limited labeled examples (1, 5, 10 shots)

This protocol demonstrates that synthetic data generation combined with multimodal alignment enables robust performance in data-scarce scenarios, with TITAN outperforming both region-of-interest and slide foundation models across multiple machine learning settings including linear probing, few-shot, and zero-shot classification [4].

Protocol 2: Fine-Tuning for Molecular Biomarker Prediction

The EAGLE study provides a validated protocol for adapting foundation models to predict EGFR mutations in lung adenocarcinoma with limited tissue samples [46]:

Data Preparation and Curation

• Assemble international multi-institutional dataset of 8,461 LUAD slides
• Ensure representation of diverse specimen types (primary and metastatic)
• Include samples from various scanner types and institutions to enhance generalization

Foundation Model Adaptation

• Initialize with pretrained pathology foundation model (e.g., Phikon, CTransPath)
• Implement partial fine-tuning strategy, updating only later transformer layers
• Use weak supervision with slide-level labels without detailed region annotations

Validation Framework

• Internal validation on 1,742 slides from MSKCC with rigorous statistical analysis
• External validation on samples from Mount Sinai Health System (294 slides), Sahlgrenska University Hospital (95 slides), Technical University of Munich (76 slides), and TCGA (519 slides)
• Prospective silent trial to simulate real-world clinical implementation

Performance Assessment

• Evaluate using area under the curve (AUC) with comparison to ground truth (MSK-IMPACT NGS)
• Assess performance stratification by tissue amount, metastasis location, and mutation variant
• Compare with clinical standard (Idylla rapid test) for sensitivity, specificity, PPV, and NPV

This protocol achieved an AUC of 0.847 on internal validation and 0.870 on external validation, demonstrating that foundation models fine-tuned on multi-institutional data can generalize effectively to new clinical settings even with limited tissue availability [46].

Diagram 1: TITAN's 3-stage training for data scarcity

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Resources for Foundation Model Development

Resource Category	Specific Examples	Function in Addressing Data Scarcity	Implementation Considerations
Pretrained Foundation Models	TITAN, CTransPath, Phikon, UNI, Virchow2	Provide transferable feature extractors reducing need for task-specific data	Model selection depends on target tissue type and specific clinical question
Public Datasets	TCGA (29,000 WSIs), CPTAC, PAIP	Offer diverse pretraining data across multiple cancer types and tissues	Require careful data curation and preprocessing for specific clinical tasks
Synthetic Data Generation Tools	PathChat, multimodal generative AI copilots	Create artificial training examples and annotations for rare conditions	Quality validation essential through pathologist review of generated content
Adaptation Frameworks	Parameter-efficient fine-tuning (PEFT), few-shot learning libraries	Enable effective model customization with minimal labeled examples	Choice depends on available data volume and computational resources
Benchmarking Platforms	Multi-institutional validation suites, silent trial frameworks	Assess model generalizability across diverse populations and settings	Critical for establishing clinical credibility and identifying failure modes

Validation and Performance Assessment

Rigorous validation is particularly crucial for foundation models applied to data-scarce niche applications, where the risk of overfitting and biased performance is elevated. The following approaches have demonstrated effectiveness:

Comprehensive Multi-Institutional Benchmarking Large-scale benchmarking studies evaluating foundation models across 14-20 datasets from multiple organs provide critical insights into real-world performance [44] [45]. These assessments reveal that pathology-specific foundation models like Virchow2 consistently achieve superior performance across TCGA, CPTAC, and external tasks, highlighting their effectiveness in diverse histopathological evaluations [35]. Interestingly, model size and pretraining data volume do not consistently correlate with improved performance, challenging conventional scaling assumptions in histopathological applications [35].

Prospective Silent Trials for Clinical Validation The EAGLE study implemented a prospective silent trial where the model was deployed in real-time clinical workflows without impacting patient care, achieving an AUC of 0.890 and demonstrating potential to reduce rapid molecular testing by up to 43% while maintaining clinical standard performance [46]. This validation approach provides the most realistic assessment of clinical utility before formal implementation.

Cross-Modal Retrieval and Zero-Shot Evaluation For multimodal foundation models like TITAN, cross-modal retrieval between histology slides and clinical reports provides a powerful validation mechanism in data-scarce scenarios [4]. The model's ability to retrieve relevant WSIs based on textual descriptions of rare conditions, and conversely generate diagnostic reports from unseen WSIs, demonstrates robust understanding without task-specific training.

Diagram 2: Solving data scarcity with foundation models

Foundation models represent a paradigm shift in addressing data scarcity and annotation challenges in computational pathology, particularly for niche applications where traditional supervised approaches face fundamental limitations. Through self-supervised learning on large unlabeled datasets, multimodal integration with clinical reports and synthetic captions, and parameter-efficient adaptation strategies, these models demonstrate remarkable capability to generalize to rare conditions and specialized tasks with minimal labeled examples.

The validated methodologies presented in this technical guide provide researchers and clinicians with a framework for developing and implementing foundation models in data-constrained environments. As the field advances, focus on multimodal integration, synthetic data generation, and rigorous multi-institutional validation will further enhance the accessibility of robust AI tools for rare diseases and specialized clinical applications, ultimately expanding the benefits of computational pathology to broader patient populations.

The development of artificial intelligence (AI) for computational pathology faces a significant constraint: the scarcity of large-scale, expertly annotated histopathology datasets. The process of data collection and annotation of whole-slide images (WSIs) is labor-intensive and not scalable to open-set recognition problems or rare diseases, both of which are common in pathology practice [6]. With thousands of possible diagnoses, training separate, dedicated models for every task in the pathology workflow is impractical [6]. This limitation is particularly pronounced for rare cancers and conditions where only a handful of annotated examples may exist [36].

Foundation models, pretrained on vast amounts of unlabeled or weakly labeled data, represent a paradigm shift. These models learn general-purpose, transferable representations of histopathology images that can be adapted to specific clinical tasks with minimal task-specific data [4] [47]. This capability is foundational to implementing few-shot learning (where models learn from a very limited number of examples) and zero-shot learning (where models perform tasks without any task-specific training examples) [48] [49]. By leveraging strategies such as self-supervised learning, multimodal vision-language alignment, and innovative model architectures, foundation models are overcoming the data bottleneck and paving the way for more scalable and versatile AI tools in pathology [4] [6].

Foundation Models: A Primer for Computational Pathology

Foundation models are large-scale neural networks trained on broad data at scale that can be adapted to a wide range of downstream tasks [4]. In computational pathology, these models are designed to learn robust feature representations from the complex and high-dimensional data contained in gigapixel whole-slide images. The transition from traditional patch-based analysis to slide-level modeling marks a critical evolution, enabling a more holistic understanding of the tissue microenvironment and its clinical implications [4] [50].

There are two primary pretraining paradigms for these models. Visual Self-Supervised Learning (SSL) uses predefined pretext tasks to learn from the intrinsic structure of unlabeled images alone. Techniques like masked image modeling and contrastive learning fall into this category [4] [47]. Multimodal Vision-Language Pretraining aligns visual features from histopathology images with textual descriptions from paired pathology reports or synthetic captions [4] [6]. This approach, exemplified by models like CONCH and TITAN, creates a shared semantic space between images and text, which is the fundamental enabler for zero-shot capabilities [6]. By learning that visual patterns correspond to descriptive phrases (e.g., "invasive ductal carcinoma" or "lymph node metastasis"), the model can later classify images based solely on text prompts, without any explicit training examples for those classes [6] [49].

Technical Strategies for Low-Data Regimes

Zero-Shot Learning: Classification Without Examples

Zero-shot learning allows a model to recognize and classify concepts it has never explicitly seen during training. The core mechanism relies on a shared semantic space that bridges the gap between visual features and semantic attributes or text descriptions [49].

Core Mechanism: A ZSL model is first trained on a set of "seen" classes where image-text pairs are available. It learns to map images and their corresponding text descriptions (e.g., from pathology reports) into a shared embedding space where semantically similar concepts are close together. During inference for an "unseen" class, the model processes an input image and a set of text prompts describing potential classes. The image is classified by matching its embedded representation with the most similar text prompt in this shared space [6] [49]. For gigapixel WSIs, this process is often applied at the tile level, with tile-level scores aggregated into a final slide-level prediction using frameworks like MI-Zero [51] [6].

The following diagram illustrates the generalized workflow for zero-shot classification of a whole-slide image using a vision-language foundation model.

Few-Shot Learning: Maximizing Minimal Data

Few-shot learning aims to learn a new task from a very limited number of labeled examples, often just one to five instances per class [48] [49]. This is particularly valuable in pathology for tasks involving rare disease subtypes or new biomarkers where data is inherently scarce.

Core Methodologies:

• Meta-Learning: Often referred to as "learning to learn," meta-learning algorithms train a model on a variety of related tasks such that it can quickly adapt to a new task with only a few examples. The model learns a generalized initialization or a parameter update rule that is highly efficient for fast adaptation [48].
• Transfer Learning with Foundation Models: A more straightforward approach involves using a large, pretrained foundation model as a feature extractor. For a new few-shot task, the features for the support images (the few labeled examples) and query images are extracted. A simple classifier (e.g., a linear probe or a nearest-neighbor classifier) is then trained on top of these frozen features using the limited support set [4] [6]. This leverages the rich, general-purpose features learned during pretraining, reducing the need for massive task-specific data.

Key Model Architectures and Methodologies

Multimodal Vision-Language Models: Models like CONCH (CONtrastive learning from Captions for Histopathology) and TITAN (Transformer-based pathology Image and Text Alignment Network) are at the forefront of zero-shot pathology. CONCH is trained using a combination of contrastive loss (to align images and text in a shared space) and a captioning loss (to generate textual descriptions from images) [6]. TITAN employs a multi-stage pretraining process, starting with visual self-supervised learning on 335,645 WSIs, followed by vision-language alignment with pathology reports and 423,122 synthetic captions [4].

Weakly Supervised Multiple-Instance Learning (MIL): Frameworks like CLAM (Clustering-constrained Attention Multiple-instance Learning) address data efficiency by using only slide-level labels. CLAM uses an attention mechanism to automatically identify and rank diagnostically relevant regions in a WSI without any pixel-level or region-level annotations. It further refines its feature space by applying instance-level clustering to the attended regions, effectively generating its own pseudo-labels for learning [36].

Handling Long-Range Context: Processing entire WSIs requires handling extremely long sequences of image patches. Prov-GigaPath introduces a novel architecture using Dilated Attention from the LongNet framework. This allows the model to efficiently capture dependencies across the gigapixel image by sparsely sampling the attention computation, countering the quadratic complexity of standard self-attention and making whole-slide modeling computationally feasible [47].

Quantitative Performance of Key Models

The performance of foundation models in few-shot and zero-shot settings has been rigorously evaluated on standard benchmarks. The tables below summarize key quantitative results.

Table 1: Zero-Shot Performance of CONCH on Slide-Level Classification Tasks (Balanced Accuracy, %)

Task (Dataset)	CONCH	PLIP	BiomedCLIP	OpenAI CLIP
NSCLC Subtyping (TCGA)	90.7	78.7	76.1	73.2
RCC Subtyping (TCGA)	90.2	80.4	78.9	75.5
BRCA Subtyping (TCGA)	91.3	50.7	55.3	53.1

Table 2: Zero-Shot Performance of CONCH on Region-of-Interest (ROI) Tasks

Task (Dataset)	Metric	CONCH	PLIP	BiomedCLIP
Gleason Pattern (SICAP)	Quadratic κ	0.690	0.570	0.550
Colorectal Tissue (CRC100k)	Accuracy (%)	79.1	67.4	65.8
LUAD Pattern (WSSS4LUAD)	Accuracy (%)	71.9	62.4	59.1

Table 3: TITAN's Few-Shot Linear Probing Performance (AUROC, %)

Dataset	TITAN (5-shot)	TITAN (10-shot)	Slide Foundation Model A	Slide Foundation Model B
TCGA-BRCA	89.2	92.5	83.1	81.7
TCGA-LUAD	85.7	89.3	79.5	77.8
In-House PD-L1	82.4	86.1	75.9	73.2

The data demonstrates that modern foundation models like CONCH and TITAN significantly outperform previous state-of-the-art models, particularly in challenging zero-shot and few-shot scenarios. CONCH's massive leap in BRCA subtyping accuracy highlights its superior semantic understanding [6]. TITAN's strong performance with minimal shots confirms the generalizability and robustness of the slide-level representations it learns [4].

Experimental Protocols for Key Evaluations

Protocol: Zero-Shot Slide-Level Classification with CONCH

Objective: To evaluate the ability of a visual-language foundation model to classify whole-slide images into diagnostic categories without using any task-specific training labels [6].

• Text Prompt Engineering:

  • For each class in the target task (e.g., "Lung Adenocarcinoma," "Lung Squamous Cell Carcinoma"), design an ensemble of text prompts. Example prompts could be "histopathology image of lung adenocarcinoma," "whole slide image of lung squamous cell carcinoma," "photomicrograph showing features of adenocarcinoma."
  • Use the model's text encoder to generate embeddings for all prompts in the ensemble.

• Whole-Slide Image Processing:

  • For a given WSI, perform tissue segmentation and tile the segmented region into non-overlapping patches (e.g., 256x256 pixels at 20x magnification).
  • Use the model's vision encoder to extract a feature vector for each tile.

• Similarity Calculation and Aggregation:

  • For each tile, compute the cosine similarity between its visual feature vector and the text embeddings of all class prompts.
  • Aggregate the tile-level similarity scores to produce a slide-level prediction. The MI-Zero framework can be used, which may involve averaging the top-K highest similarity scores per class across all tiles or using a more sophisticated attention mechanism [51] [6].
  • The class with the highest aggregate similarity score is assigned as the final slide-level prediction.

Protocol: Few-Shot Linear Probing with TITAN

Objective: To evaluate the quality of a foundation model's features by training a simple linear classifier on top of frozen features with very few labeled examples [4].

• Feature Extraction:

  • Using the pretrained TITAN model with frozen weights, process the entire dataset (including train and test splits) to extract a fixed-dimensional, slide-level feature vector for each WSI.

• Few-Shot Dataset Creation:

  • From the training set, randomly sample K examples per class (e.g., K=5 for 5-shot) to form the support set. This process is often repeated multiple times with different random seeds to obtain robust performance estimates.

• Classifier Training:

  • Train a logistic regression classifier or a single linear layer on the support set's feature vectors and their corresponding labels. No data augmentation is typically applied at this stage to isolate the effect of the feature quality.

• Evaluation:

  • Use the trained linear classifier to make predictions on the frozen features of the held-out test set. Report standard metrics such as Area Under the Receiver Operating Characteristic curve (AUROC) or balanced accuracy.

The Scientist's Toolkit: Essential Research Reagents

The following table details key computational tools and resources that form the essential "reagents" for research in this field.

Table 4: Key Research Reagents for Low-Data Computational Pathology

Reagent / Resource	Type	Primary Function	Example in Use
Pretrained Foundation Models (e.g., CONCH, TITAN, Prov-GigaPath)	Software Model	Provides general-purpose, transferable feature embeddings for histopathology images and/or text.	Used as a frozen feature extractor for few-shot classification or for zero-shot inference via prompt engineering [4] [6].
Large-Scale Slide-Report Datasets	Dataset	Serves as the pretraining corpus for multimodal vision-language models, enabling semantic alignment.	Mass-340K (335k WSIs + reports) for TITAN; 1.17M image-caption pairs for CONCH [4] [6].
Synthetic Caption Generators	Tool / Model	Generates fine-grained textual descriptions for histology image patches to augment pretraining data.	TITAN used PathChat, a generative AI copilot, to create 423k synthetic ROI captions [4].
Weakly-Supervised MIL Frameworks (e.g., CLAM)	Algorithm	Enables model training using only slide-level labels, eliminating need for expensive pixel-wise annotations.	Applied for cancer subtyping and tumor detection, producing interpretable attention heatmaps [36].
Long-Sequence Transformers (e.g., LongNet, Dilated Attention)	Model Architecture	Manages the computational complexity of processing gigapixel WSIs by sparsifying self-attention.	Core component of Prov-GigaPath for encoding entire slides efficiently [47].

The integration of foundation models equipped with few-shot and zero-shot learning capabilities marks a critical advancement in computational pathology. By addressing the fundamental challenge of data scarcity, these strategies extend the reach of AI to rare diseases, novel biomarkers, and clinical settings where curating large annotated datasets is not feasible. The empirical success of models like TITAN, CONCH, and Prov-GigaPath across diverse tasks—from cancer subtyping and prognosis to cross-modal retrieval—demonstrates a tangible path toward more generalizable, scalable, and clinically adaptable AI tools. Future progress will hinge on developing even larger and more diverse pretraining datasets, refining model architectures for efficient whole-slide understanding, and, most importantly, rigorous validation within real-world clinical workflows to fully translate this transformative potential into improved patient care.

Foundation models in computational pathology are large-scale deep neural networks trained on enormous datasets of whole slide images (WSIs) using self-supervised learning algorithms that do not require curated labels [26]. These models generate data representations called embeddings that generalize well to diverse predictive tasks, offering a distinct advantage over diagnostic-specific methods limited to subsets of pathology images [26]. The clinical value of these models has been demonstrated across numerous applications, including pan-cancer detection, biomarker prediction, treatment response assessment, and patient risk stratification [26] [41].

However, this capability comes at a significant computational cost. The development of models such as Virchow (632 million parameters trained on 1.5 million WSIs) and PathOrchestra (trained on 287,424 slides across 21 tissue types) illustrates the massive data and parameter scaling required for state-of-the-art performance [26] [52]. This technical guide examines the computational and resource challenges inherent to these models and provides practical methodologies for managing scale and inference costs in research and clinical settings.

The Computational Landscape: Model Scale and Infrastructure Demands

Table 1: Scale Comparison of Major Pathology Foundation Models

Model Name	Parameters	Training Dataset Size	Architecture	Key Performance Metrics
Virchow	632 million	1.5 million WSIs from ~100,000 patients	Vision Transformer (ViT)	0.950 AUC pan-cancer detection; 0.937 AUC rare cancers [26]
PathOrchestra	Not specified	287,424 WSIs across 21 tissue types	Self-supervised vision encoder	>0.950 accuracy in 47/112 tasks including pan-cancer classification [52]
CONCH	Not specified	1.17 million image-caption pairs	Vision-language model	0.71 average AUROC across 31 tasks [53]
Virchow2	Not specified	3.1 million WSIs	Vision-only	0.71 average AUROC across 31 tasks; close second to CONCH [53]

Computational Resource Requirements

Table 2: Computational Resource Requirements for Pathology Foundation Models

Resource Type	Training Phase	Inference Phase	Storage Requirements
GPU Memory	Extensive (e.g., 2x AMD Radeon Instinct MI210 with 64GB RAM for deployment framework) [31]	Significant for whole slide processing	Massive for raw images and extracted features
Processing Time	Weeks to months on multiple high-end GPUs	Minutes to hours per slide depending on model complexity	Petabyte-scale storage systems for institutional deployments
Network Infrastructure	High-speed interconnects for multi-node training	100Mbit/s+ network connections for slide access [31]	Network-attached storage (NAS) systems for slide management [31]

The resource demands begin with the fundamental challenge of processing whole slide images, which are exceptionally large data objects. As noted in recent research, "WSIs cannot be directly incorporated into CNNs due to their large size and are divided into patches or tiles" [1]. A single WSI can exceed 100,000 × 100,000 pixels, requiring division into thousands of smaller patches for processing [1]. This multi-scale nature necessitates specialized architectures that can capture both local patterns (within small tiles) and global patterns across the entire slide [54].

Technical Methodologies for Efficient Model Development and Deployment

Experimental Protocols for Foundation Model Evaluation

Benchmarking Methodology Comprehensive evaluation of foundation models requires standardized benchmarking across multiple tasks and datasets. A recent large-scale benchmark evaluated 19 foundation models on 31 clinically relevant tasks using data from 6,818 patients and 9,528 slides [53]. The protocol included:

• Task Diversity: Evaluation across morphology (5 tasks), biomarkers (19 tasks), and prognostication (7 tasks)
• Data Sourcing: Use of external cohorts never part of foundation model training to prevent data leakage
• Evaluation Metrics: Primary metrics included Area Under the Receiver Operating Characteristic Curve (AUROC), with additional analysis of Area Under the Precision-Recall Curve (AUPRC), balanced accuracy, and F1 scores
• Aggregation Methods: Comparison of transformer-based aggregation with attention-based multiple instance learning (ABMIL) approaches

This benchmarking revealed that "CONCH, a vision-language model trained on 1.17 million image-caption pairs, performs on par with Virchow2, a vision-only model trained on 3.1 million WSIs" [53], highlighting that data diversity and model architecture significantly impact performance efficiency.

Weakly Supervised Learning for Data Efficiency Multiple instance learning (MIL) coupled with an 'attention' approach, where the algorithm applies a weighted average to each tile, provides a more efficient learning paradigm [1]. This approach is particularly valuable because "not every tile on a slide will reflect the ground truth label; instead, only the plurality of tiles generated from a WSI represents a specific label" [1]. This method reduces annotation requirements while maintaining model accuracy.

Integration Frameworks for Clinical Deployment

Recent work has demonstrated successful integration frameworks that address computational challenges. One proof-of-concept implementation "leverages Health Level 7 (HL7) standard and open-source DP resources" to integrate deep learning models into the clinical workflow [31]. The technical architecture includes:

• Python-based server-client architecture interconnecting the anatomic pathology laboratory information system with an external AI-based decision support system
• Dual deployment modes: Continuous background processing of new slides and on-demand analysis initiated by pathologists
• Open-source toolboxes for model deployment, including WSInfer for patch-level classification models
• Visualization integration through QuPath for intuitive display of model predictions as colored heatmaps

This framework successfully processed "11,805 hematoxylin and eosin slides digitized as part of routine diagnostic activity" from 3,157 patients [31], demonstrating scalability for clinical implementation.

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Reagents and Computational Tools for Pathology Foundation Models

Tool/Resource	Function	Implementation Example
WSInfer	Open-source toolbox for patch-level model deployment	"Deployment of pre-trained patch-level classification models was performed relying on the WSInfer command-line tool v0.6.1" [31]
QuPath	Open-source digital pathology platform for visualization	"QuPath was used to provide an intuitive visualization of model predictions as colored heatmaps" [31]
ABMIL	Attention-Based Multiple Instance Learning for WSI classification	"MIL can learn to associate individual tiles that contain tumour even though only the slide is labelled as tumour" [1]
DINO v2	Self-supervised learning algorithm for representation learning	"Virchow, a 632 million parameter ViT model, is trained using the DINO v.2 algorithm" [26]
HL7 Standards	Protocol for healthcare system interoperability	"HL7 is used to interconnect the DL models to the AP-LIS" [31]
Vision Transformers	Neural network architecture for image recognition	"Modern foundation models use vision transformers (ViTs) with hundreds of millions to billions of parameters" [26]

Cost Management Strategies for Model Training and Inference

Data Efficiency Techniques

The scaling laws in computational pathology demonstrate interesting properties that can be leveraged for cost optimization. While foundation model performance generally improves with dataset size, research has shown that "data diversity outweighs data volume for foundation models" [53]. This principle is exemplified by the performance of CONCH, which "outperformed BiomedCLIP despite seeing far fewer image-caption pairs (1.1 million versus 15 million)" [53].

Strategies for data-efficient training include:

• Leveraging Pre-trained Models: Building on existing foundation models rather than training from scratch
• Targeted Data Curation: Prioritizing diverse tissue representation over sheer volume
• Cross-domain Transfer: Utilizing models trained on related tissue types or cancer domains

Inference Optimization

For deployment settings, several techniques can reduce computational costs:

• Model Distillation: Training smaller, specialized models from larger foundation models
• Selective Processing: Implementing attention mechanisms to focus computation on diagnostically relevant regions
• Hardware-aware Optimization: Utilizing optimized inference servers and GPU acceleration

The field of computational pathology is rapidly evolving to address scale and cost challenges. Promising directions include:

• Modular Foundation Models: Developing composable architectures that allow efficient fine-tuning for specific clinical tasks
• Federated Learning: Enabling multi-institution collaboration without centralized data aggregation
• Hardware-Software Co-design: Creating specialized processors optimized for pathology workflow patterns

The integration frameworks and methodologies presented in this guide demonstrate that despite the significant computational hurdles, strategic approaches to model development, evaluation, and deployment can enable the clinical adoption of pathology foundation models. As the field matures, continued focus on computational efficiency will be essential for realizing the potential of these transformative technologies in routine patient care.

The future adoption pathway will likely depend on "addressing barriers through education, regulatory approval, and collaboration with pathologists and biopharma" [55], with successful implementation requiring both technical solutions and stakeholder engagement.

Foundation models in computational pathology are transformative AI systems pretrained on vast datasets of histopathological images, capable of adapting to a wide range of downstream diagnostic, prognostic, and predictive tasks. These models, trained via self-supervised learning on millions of histology image patches, capture fundamental morphological patterns in tissues that can be leveraged for various clinical applications with minimal fine-tuning. However, deploying these powerful models in real-world clinical settings presents significant challenges, including model overconfidence, performance degradation on rare cancer types, and handling of noisy medical data. Ensemble methods and multi-model strategies have emerged as crucial optimization techniques to address these limitations, enhancing the reliability, accuracy, and generalizability of computational pathology systems in high-stakes healthcare environments where diagnostic errors can have severe consequences.

The clinical implementation of foundation models requires not only exceptional performance but also well-calibrated uncertainty estimation and robust operation across diverse patient populations and imaging protocols. Ensemble approaches provide a principled framework for quantifying model uncertainty, reducing overconfident predictions on ambiguous cases, and improving out-of-distribution detection. This technical guide examines cutting-edge ensemble methodologies and multi-model frameworks that are advancing the clinical readiness of computational pathology systems, with detailed experimental protocols and performance benchmarks to guide research and implementation.

Ensemble Methodologies for Enhanced Model Performance

Uncertainty-Aware Deep Ensembles for Diagnostic Reliability

The PICTURE (Pathology Image Characterization Tool with Uncertainty-aware Rapid Evaluations) system exemplifies a sophisticated ensemble approach specifically designed for challenging neuropathological diagnostics. This system addresses the critical clinical problem of distinguishing histologically similar but therapeutically distinct central nervous system tumors, particularly glioblastoma and primary central nervous system lymphoma (PCNSL), which have overlapping morphological features but require dramatically different treatment regimens. The methodological framework integrates multiple uncertainty quantification techniques to enhance diagnostic reliability and flag rare cancer types not represented in the training data [56].

Experimental Protocol: The PICTURE development involved collecting 2,141 pathology slides from international medical centers, including both formalin-fixed paraffin-embedded (FFPE) permanent slides and frozen section whole-slide images. The ensemble architecture incorporates three complementary uncertainty quantification methods: Bayesian inference, deep ensemble modeling, and normalizing flow techniques. This multi-faceted approach accounts for both epistemic uncertainty (related to model knowledge gaps) and aleatoric uncertainty (inherent data noise). During inference, the system aggregates predictions across multiple specialized foundation models while simultaneously evaluating confidence metrics to identify cases where the ensemble lacks sufficient knowledge, thereby reducing dangerously overconfident misclassifications [56].

Performance Benchmarks: The PICTURE system achieved exceptional performance in distinguishing glioblastoma from PCNSL, with an area under the receiver operating characteristic curve (AUROC) of 0.989 in internal validation. More importantly, it maintained robust performance across five independent international cohorts (AUROC = 0.924-0.996), demonstrating superior generalizability compared to conventional single-model approaches. The uncertainty quantification module successfully identified 67 types of rare central nervous system cancers that were neither gliomas nor lymphomas, enabling appropriate referral of these cases for specialized expert review rather than generating potentially misleading confident predictions [56].

Ensemble Distillation for Computational Efficiency

While ensembles significantly enhance performance and reliability, their computational demands during inference often preclude deployment in resource-constrained clinical environments. Ensemble distillation addresses this challenge by transferring the collective knowledge of multiple models into a single, efficient architecture without substantial performance degradation. This approach is particularly valuable for processing high-volume pathology report data where computational efficiency is essential for practical implementation [57] [58].

Experimental Protocol: The distillation process follows a teacher-student framework wherein an ensemble of 1,000 multitask convolutional neural networks (teacher models) generates aggregated predictions on the training dataset. These aggregated predictions produce "soft labels" that capture the uncertainty and class relationships within the data. A single multitask convolutional neural network (student model) is then trained using these soft labels rather than the original hard labels, enabling it to learn the nuanced decision boundaries discovered by the full ensemble. This approach is particularly beneficial for problems with extreme class imbalance and noisy datasets, common challenges in cancer pathology reporting where some cancer subtypes have very few examples while others are predominant [57].

Table 1: Performance Comparison of Baseline, Ensemble, and Distilled Models on Cancer Pathology Report Classification

Model Type	Accuracy	Abstention Rate	Additional Reports Classified	Computational Demand
Baseline Single Model	Baseline	Baseline	Reference	1x
Full Ensemble (1000 models)	Highest	Lowest	N/A (reference)	1000x
Distilled Model	Intermediate	Intermediate	+1.81% (subsite), +3.33% (histology)	1x

Results and Analysis: The distilled student model outperformed the baseline single model in both accuracy and abstention rates, allowing deployment with a larger effective volume of documents while maintaining required accuracy thresholds. The most significant improvements were observed for the most challenging classification tasks: cancer subsite and histology determination, where the distilled model correctly classified an additional 1.81% of reports for subsite and 3.33% for histology without compromising accuracy. Crucially, the distilled model substantially reduced overconfident incorrect predictions, particularly for difficult-to-classify documents where the ensemble exhibited disagreement, demonstrating that the distillation process effectively preserved the uncertainty calibration of the full ensemble while maintaining the computational efficiency of a single model [57].

Multi-Model Frameworks in Pathology Foundation Models

Vision-Language Multimodal Integration

Multimodal foundation models represent a paradigm shift in computational pathology by integrating visual processing with language understanding capabilities. TITAN (Transformer-based pathology Image and Text Alignment Network) exemplifies this approach, combining histopathological image analysis with corresponding pathology reports and synthetic captions to create a unified representation space that supports diverse clinical tasks without requiring task-specific fine-tuning [4] [39].

Experimental Protocol: TITAN employs a three-stage pretraining strategy to develop general-purpose slide representations. The first stage involves vision-only self-supervised pretraining on 335,645 whole-slide images using masked image modeling and knowledge distillation objectives. The second stage incorporates cross-modal alignment at the region-of-interest level, utilizing 423,122 synthetic fine-grained morphological descriptions generated by PathChat, a multimodal generative AI copilot for pathology. The third stage performs cross-modal alignment at the whole-slide level using 183,000 pairs of WSIs and clinical reports. This progressive training approach enables the model to learn both visual representations and their corresponding semantic descriptions, supporting zero-shot classification and cross-modal retrieval capabilities [4].

Table 2: TITAN Model Architecture Specifications and Training Data

Component	Specification	Data Scale	Modality
Visual Encoder	Vision Transformer (ViT)	335,645 WSIs	Image
Patch Feature Extractor	CONCHv1.5 (768-dimensional features)	141M+ patches	Image
Text Encoder	Pretrained Language Model	182,862 reports	Text
Synthetic Data	PathChat-generated captions	423,122 ROI-caption pairs	Image-Text
Pretraining Objective	Masked Image Modeling + Knowledge Distillation	340K dataset	Multimodal

Performance Analysis: The multimodal TITAN framework demonstrated superior performance compared to both region-of-interest and slide-level foundation models across multiple machine learning settings, including linear probing, few-shot learning, and zero-shot classification. Particularly notable was its performance in resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis, where it outperformed specialized models trained specifically for these tasks. The cross-modal alignment enabled clinically valuable capabilities such as pathology report generation from whole-slide images and semantic search of histology images using natural language queries, significantly expanding the potential applications of AI in pathology practice [4].

Large-Scale Multi-Model Architectures for Pan-Cancer Analysis

PathOrchestra represents a comprehensive foundation model approach trained on an unprecedented scale of 287,424 slides across 21 tissue types from three independent centers. This massive multi-model framework enables state-of-the-art performance across 112 diverse clinical tasks, establishing new benchmarks in computational pathology [5].

Experimental Protocol: PathOrchestra was evaluated on 27,755 whole-slide images and 9,415,729 region-of-interest images across multiple task categories: digital slide preprocessing, pan-cancer classification, lesion identification, multi-cancer subtype classification, biomarker assessment, gene expression prediction, and structured report generation. The model architecture employs a self-supervised vision encoder pretrained on 141,471,591 patches sampled at 256×256 pixels under 20× magnification. For whole-slide classification tasks, the model utilizes attention-based multiple instance learning (ABMIL) to aggregate patch-level predictions into slide-level diagnoses, effectively handling the gigapixel-scale of whole-slide images [5].

Performance Benchmarks: PathOrchestra achieved remarkable accuracy exceeding 0.950 in 47 of the 112 evaluation tasks, demonstrating exceptional generalization across cancer types and diagnostic challenges. In pan-cancer classification, it attained an average AUC of 0.988 for 17-class tissue classification, with perfect classification (ACC, AUC, and F1 = 1.0) for prostate cancer biopsies. The model established the first framework for generating structured pathology reports for colorectal cancer and lymphoma, representing a significant advancement toward automated pathology reporting systems. Performance variation between FFPE and frozen sections highlighted the importance of specimen preparation, with FFPE sections showing 1.4% higher AUC, 8.9% higher accuracy, and 9% higher F1 scores compared to frozen sections, attributed to better preservation of tissue morphology in FFPE samples [5].

Visualization of Ensemble and Multi-Model Frameworks

Uncertainty-Aware Ensemble Framework for CNS Tumor Diagnosis

Uncertainty-Aware Ensemble Diagnostic Pipeline

Multimodal Foundation Model Architecture

Multimodal Vision-Language Architecture

Essential Research Reagents and Computational Tools

Table 3: Key Research Reagent Solutions for Implementing Pathology Ensemble Methods

Resource Category	Specific Tool/Model	Primary Function	Implementation Role
Foundation Models	PathOrchestra	Large-scale pathology vision encoder	Base feature extraction for ensemble systems
Multimodal Models	TITAN	Vision-language whole-slide encoding	Cross-modal alignment and zero-shot tasks
Uncertainty Methods	Bayesian Deep Ensembles	Epistemic uncertainty quantification	Confidence calibration in PICTURE
Distillation Framework	Teacher-Student Protocol	Model compression	Transfer ensemble knowledge to deployable models
Data Resources	TCGA, Mass-340K	Large-scale pretraining datasets	Model development and validation
Synthetic Data Tools	PathChat	Caption generation for ROIs	Multimodal pretraining data augmentation

Ensemble methods and multi-model strategies represent essential optimization techniques for enhancing the clinical utility and reliability of foundation models in computational pathology. The approaches detailed in this technical guide—including uncertainty-aware deep ensembles, ensemble distillation for efficient deployment, and multimodal integration—address critical challenges in real-world clinical implementation. These methodologies enable more accurate diagnostics, better uncertainty quantification, improved generalization across diverse populations and imaging protocols, and computationally efficient deployment in resource-constrained healthcare environments. As foundation models continue to evolve in scale and capability, sophisticated ensemble and multi-model frameworks will play an increasingly vital role in translating their potential into clinically validated tools that enhance pathological diagnosis, prognosis, and therapeutic decision-making while maintaining appropriate safeguards against overconfident predictions on challenging or out-of-distribution cases.

Benchmarking Foundation Models: Performance Validation and Model Selection

Independent Benchmarking Frameworks and Performance Metrics

Independent benchmarking is a critical methodology for objectively evaluating the performance, robustness, and generalizability of foundation models in computational pathology. As the number of proposed models grows, comprehensive benchmarking on diverse, external datasets using standardized metrics and protocols is essential to uncover true capabilities, prevent data leakage, and guide clinical translation. This guide details the established frameworks, key performance indicators, and experimental methodologies that constitute rigorous, independent evaluation, providing researchers with the tools to validate model performance against clinically relevant tasks.

Benchmarking Frameworks and Performance Landscape

Independent benchmarking in computational pathology involves systematically evaluating foundation models—often trained on massive, proprietary datasets—on a battery of external, clinically-focused tasks they have not encountered during training. This process mitigates the risks of selective reporting and data leakage, providing a realistic assessment of model utility in real-world scenarios [53].

A seminal large-scale benchmark evaluated 19 histopathology foundation models on 13 patient cohorts comprising 6,818 patients and 9,528 whole-slide images (WSIs) across lung, colorectal, gastric, and breast cancers. The evaluation spanned 31 weakly supervised downstream tasks categorized into three critical domains: morphological properties (5 tasks), biomarkers (19 tasks), and prognostic outcomes (7 tasks) [53]. This study established that the vision-language model CONCH and the vision-only model Virchow2 achieved the highest overall performance, with an ensemble of both outperforming individual models in 55% of tasks [53] [59].

A subsequent, broader benchmark expanded this analysis to 31 foundation models (including general vision, general vision-language, pathology-specific vision, and pathology-specific vision-language models) across 41 tasks from TCGA, CPTAC, and external out-of-domain datasets. It confirmed Virchow2 as a top performer and highlighted that model size and pretraining data volume do not consistently correlate with performance, challenging assumptions about scaling in histopathology [22].

The table below summarizes the quantitative findings from these major benchmarking studies.

Table 1: Performance Summary of Top-Tier Pathology Foundation Models from Independent Benchmarks

Foundation Model	Model Type	Key Pretraining Data	Average AUROC (31 Tasks, Neidlinger et al.)	Performance Highlights
CONCH	Vision-Language (Path-VLM)	1.17M image-caption pairs [53]	0.71 [53]	Highest mean AUROC for morphology (0.77) and prognosis (0.63) [53]
Virchow2	Vision-Only (Path-VM)	3.1M WSIs [53]	0.71 [53]	Top performer in 41-task benchmark; led in 8/17 tasks with n=300 training samples [53] [22]
Prov-GigaPath	Vision-Only (Path-VM)	Large-scale WSI cohort [53]	0.69 [53]	Close third place overall; mean AUROC of 0.72 for biomarker tasks [53]
DinoSSLPath	Vision-Only (Path-VM)	Not Specified	0.69 [53]	Tied for third overall; mean AUROC of 0.76 for morphology [53]
TITAN	Multimodal WSI	335,645 WSIs [4] [39]	Not Quantified	Outperformed other slide and ROI models in few-shot/zero-shot classification and retrieval [4]
PathOrchestra	Vision-Only (Path-VM)	287,424 WSIs, 21 tissues [52]	Not Quantified	Achieved >0.950 accuracy in 47 of 112 clinical tasks, including pan-cancer classification [52]

Core Performance Metrics and Evaluation Tasks

The evaluation of computational pathology foundation models relies on a suite of established metrics applied to tasks that reflect real-world clinical and research needs.

Table 2: Standard Performance Metrics and Clinical Task Definitions for Benchmarking

Metric Category	Specific Metrics	Definition and Clinical Relevance
Primary Classification Performance	Area Under the Receiver Operating Characteristic Curve (AUROC) [53]	Measures the model's ability to discriminate between classes across all classification thresholds. The primary metric for most benchmarks.
	Area Under the Precision-Recall Curve (AUPRC) [53]	Particularly informative for imbalanced datasets, as it focuses on performance on the positive (often minority) class.
	Balanced Accuracy, F1 Score [53] [52]	Provide a single-threshold measure of performance, with balanced accuracy accounting for class imbalance.
Core Evaluation Tasks	Biomarker Prediction (e.g., MSI, HRD, BRAF) [53]	Predicts molecular alterations directly from H&E morphology, crucial for targeted therapy.
	Morphological Property Assessment [53]	Evaluates tasks like tumor grading and cell type identification based on tissue structure.
	Prognostic Outcome Prediction [53]	Predicts patient outcomes such as survival from WSIs.
	Pan-Cancer and Subtype Classification [52]	Identifies the cancer type and its histological subtypes from a wide range of organs.
	Slide Preprocessing & Quality Control [52]	Detects artifacts, identifies stain types, and determines sample type.

Experimental Protocols for Independent Benchmarking

A robust benchmarking protocol requires meticulous attention to dataset curation, feature extraction, model training, and statistical validation. The following workflow delineates the standard methodology.

Standard Benchmarking Workflow diagram outlines the four-phase protocol for independent model evaluation, from dataset curation to final reporting.

Dataset Curation and Preparation

• External Cohort Selection: Benchmarks must use multiple proprietary cohorts from different countries and institutions that were not part of any foundation model's pretraining data to ensure external validation and prevent data leakage [53]. This involves collecting thousands of WSIs from diverse cancer types [53] [22].
• Task Definition: Define a comprehensive set of weakly supervised tasks based on slide-level or patient-level labels, including biomarker status, morphological classifications, and prognostic endpoints [53].
• WSI Preprocessing: Tessellate each gigapixel WSI into small, non-overlapping patches (typically 256×256 to 1024×1024 pixels at 20× magnification). This transforms each WSI into a "bag" of patches for multiple instance learning (MIL) [53] [52] [25].

Feature Extraction and Model Training

• Feature Extraction with Foundation Models: Pass each image patch through the foundation model under evaluation to extract a feature embedding. Models like CONCH and Virchow2 generate tile-level embeddings that serve as powerful inputs for downstream models [53].
• Feature Aggregation: Use an aggregation model to combine the hundreds or thousands of tile-level embeddings from a single WSI into a slide-level representation. Common aggregators include Transformer-based models and Attention-Based Multiple Instance Learning (ABMIL), with benchmarks showing Transformer-based aggregation slightly outperforms ABMIL [53].
• Task-Specific Head Training: Train a task-specific prediction head (e.g., a classifier) on the aggregated slide-level features using the weakly supervised labels. This is typically done with cross-validation on the training set [53].

Evaluation and Statistical Analysis

• Performance Calculation: Evaluate the trained model on a held-out test set, calculating AUROC, AUPRC, and other metrics as described in Section 2 [53].
• Statistical Significance Testing: Perform statistical tests, such as the Wilcoxon signed-rank test, to determine if performance differences between models are statistically significant (p < 0.05) across multiple tasks [53].
• Low-Data Scenario Analysis: Evaluate model performance in data-scarce settings by training on randomly sampled subsets of the data (e.g., 75, 150, and 300 patients) to simulate real-world scenarios with limited labeled data [53].

The Scientist's Toolkit: Key Research Reagents

Successful benchmarking requires a suite of "research reagents"—curated datasets, software tools, and computational models.

Table 3: Essential Resources for Benchmarking Computational Pathology Models

Resource Category	Specific Resource	Description and Function in Benchmarking
Public Data Repositories	The Cancer Genome Atlas (TCGA) [53] [52]	A primary source of public WSIs and associated molecular data for training and validation.
	Clinical Proteomic Tumor Analysis Consortium (CPTAC) [22]	Provides additional proteogenomically characterized tumor samples for validation.
	Image Data Resource (IDR) [60]	Public repository of bioimaging data, including histopathology images.
Software & Management Tools	OMERO [60]	Open-source image data management platform for organizing, visualizing, and analyzing WSIs.
	QuPath [60]	Digital pathology platform for whole-slide image analysis and annotation.
	Comparative Pathology Workbench (CPW) [60]	A web-based "spreadsheet" interface for visual comparison of pathology images and analysis results across cases.
Computational Models	CONCH [53]	A leading vision-language foundation model for extracting tile embeddings.
	Virchow2 [53] [22]	A leading vision-only foundation model for extracting tile embeddings.
	TITAN [4] [39]	A multimodal whole-slide foundation model that generates slide-level representations.
Evaluation Frameworks	Attention-Based MIL (ABMIL) [53]	A standard multiple instance learning architecture for aggregating tile-level features.
	Transformer Aggregator [53]	A transformer-based architecture for aggregating tile-level features, often outperforming ABMIL.

Critical Considerations and Future Directions

The field is moving towards benchmarking whole-slide foundation models like TITAN, which are pretrained to encode entire WSIs into a single slide-level embedding, simplifying the clinical workflow by eliminating the need for a separate aggregation step [4]. Furthermore, multimodal evaluation is becoming crucial, assessing capabilities like cross-modal retrieval (e.g., finding WSIs based on text descriptions) and pathology report generation, which are hallmarks of general-purpose foundation models [4] [25].

Future benchmarking efforts must also prioritize low-prevalence tasks and rare diseases to truly test model generalizability in the most challenging and clinically critical scenarios [53] [4]. Finally, the consistent finding that data diversity often outweighs data volume for foundation model performance suggests that future model development and evaluation should focus more on the breadth and quality of pretraining data rather than simply its scale [53].

Foundation models are transforming computational pathology by leveraging large-scale, self-supervised learning on vast collections of histopathology images. These models learn universal feature representations from unlabeled whole slide images (WSIs), capturing diverse morphological patterns across tissues and diseases [50] [12]. This paradigm shift addresses critical limitations in traditional computational pathology approaches, including dependency on large annotated datasets, poor generalization across domains, and difficulty analyzing rare diseases with limited training data [52] [26]. By pre-training on millions of image patches from hundreds of thousands of WSIs, pathology foundation models create versatile embeddings that can be adapted to numerous downstream diagnostic tasks with minimal fine-tuning, thereby accelerating the development of robust AI tools for clinical decision support in cancer diagnosis, subtyping, biomarker prediction, and prognosis assessment [4] [26].

Model Architectures and Technical Specifications

The leading foundation models in computational pathology employ sophisticated transformer-based architectures and self-supervised learning objectives trained on massive, diverse datasets of histopathology images.

Table 1: Architectural and Training Specifications of Leading Pathology Foundation Models

Model	Architecture	Parameters	Training Data (WSIs)	Training Objective	Key Innovation
Virchow	Vision Transformer (ViT)	632 million	1.5 million (MSKCC)	DINOv2	Million-scale training dataset [26]
PathOrchestra	Vision Transformer (ViT)	Not specified	~300,000 (multi-center)	DINOv2	Evaluation across 112 clinical tasks [52] [61]
TITAN	Transformer-based WSI encoder	Not specified	335,645 (Mass-340K)	Multi-stage: iBOT + VLM alignment	Whole-slide representation learning [4]
CONCH	Vision-Language Model	Not specified	Not specified	Contrastive learning	Cross-modal alignment for histopathology [4]
UNI	Transformer-based	Not specified	~100,000	Self-supervised learning	Cross-instructional learning [26]

Virchow: Million-Scale Foundation Model

Virchow represents a significant scaling achievement in computational pathology, trained on approximately 1.5 million H&E-stained WSIs from Memorial Sloan Kettering Cancer Center (MSKCC) [26]. The model employs a 632-million parameter Vision Transformer architecture trained using the DINOv2 self-supervised framework, which leverages both global and local regions of tissue tiles to learn rich feature representations. The training data encompasses 17 different tissue types from both biopsy (63%) and resection (37%) specimens, providing extensive morphological diversity [26].

PathOrchestra: Comprehensive Multi-Task Performer

PathOrchestra is designed as a versatile pathology foundation model trained on approximately 300,000 pathological slides (262.5 TB) spanning 20-21 tissue types across multiple medical centers [52] [61]. Implemented as a self-supervised vision encoder based on the DINOv2 architecture, the model employs a teacher-student framework with multi-scale, multi-view data augmentation techniques. A key innovation is its comprehensive evaluation across 112 diverse clinical tasks, establishing benchmarks in digital slide preprocessing, pan-cancer classification, lesion identification, multi-cancer subtype classification, biomarker assessment, gene expression prediction, and structured report generation [52].

TITAN: Multimodal Whole-Slide Foundation Model

TITAN (Transformer-based pathology Image and Text Alignment Network) introduces a multimodal approach to whole-slide representation learning [4]. The model undergoes a three-stage pretraining strategy: (1) vision-only unimodal pretraining on ROI crops using iBOT framework; (2) cross-modal alignment of generated morphological descriptions at ROI-level; and (3) cross-modal alignment at WSI-level with clinical reports [4]. TITAN processes non-overlapping patches of 512×512 pixels at 20× magnification, extracting 768-dimensional features for each patch using CONCHv1.5. To handle computational complexity from gigapixel WSIs, TITAN employs attention with linear bias (ALiBi) for long-context extrapolation [4].

CONCH: Cross-Modal Histopathology Representation

CONCH (Cross-Modal Histopathology) is a vision-language model designed to learn meaningful visual representations by aligning image patches with corresponding text in pathology reports [4]. While specific architectural details are limited in the search results, CONCH serves as a foundational component in the TITAN pipeline, providing patch-level feature extraction that enables cross-modal retrieval and zero-shot capabilities. The model demonstrates the value of incorporating textual context from pathology reports to enhance visual representation learning in histopathology.

Performance Benchmarks and Clinical Applications

Recent comprehensive benchmarking studies evaluating 31 AI foundation models for computational pathology reveal that pathology-specific vision models (Path-VMs) generally outperform both pathology-specific vision-language models (Path-VLMs) and general vision models across diverse tasks [22]. The evaluation covered 41 tasks sourced from TCGA, CPTAC, external benchmarking datasets, and out-of-domain datasets.

Table 2: Performance Comparison Across Clinical Tasks

Model	Pan-Cancer Detection (AUC)	Rare Cancer Detection (AUC)	Structured Report Generation	Key Clinical Strengths
Virchow	0.950 (across 9 common cancers)	0.937 (across 7 rare cancers)	Not specified	Excellent rare cancer detection, robust OOD performance [26]
PathOrchestra	0.988 (17-class), 0.964 (32-class TCGA)	>0.950 accuracy in 47/112 tasks	Yes (colorectal cancer, lymphoma)	Multi-task capability, high accuracy across diverse tasks [52]
TITAN	Outperforms other slide foundation models	Excels in rare disease retrieval	Yes (via vision-language alignment)	Zero-shot classification, cross-modal retrieval [4]
UNI	0.940 (pan-cancer)	Comparable to Virchow for 8/9 common cancers	Not specified	Strong generalizability, competitive with larger models [26]

Pan-Cancer Detection Capabilities

Pan-cancer detection represents a critical benchmark for evaluating the generalization capability of pathology foundation models. Virchow demonstrates exceptional performance with an overall AUC of 0.950 across nine common cancer types, outperforming UNI (0.940), Phikon (0.932), and CTransPath (0.907) [26]. PathOrchestra achieves even higher performance in specific pan-cancer classification tasks, reaching an AUC of 0.988 in 17-class tissue classification and 0.964 in 32-class classification using TCGA FFPE data [52]. Notably, PathOrchestra achieved perfect scores (ACC, AUC, and F1 = 1.0) for prostate cancer classification, attributed to the consistent and visually distinctive features of needle biopsy samples [52].

Rare Cancer Detection and Out-of-Distribution Generalization

Rare cancer detection poses significant challenges due to limited training data. Virchow demonstrates robust performance on rare cancers (NCI definition: <15 annual cases per 100,000 people) with an AUC of 0.937 [26]. Performance varies across rare cancer types, with cervical (0.875 AUC) and bone cancers (0.841 AUC) presenting greater challenges. When evaluated on out-of-distribution data from institutions other than MSKCC, Virchow maintains consistent performance, demonstrating effective generalization to new populations and tissue types not observed during training [26].

Structured Report Generation and Multimodal Capabilities

PathOrchestra is the first foundation model to generate structured reports for high-incidence colorectal cancer and diagnostically complex lymphoma [52] [61]. TITAN also demonstrates strong language capabilities, generating pathology reports and enabling cross-modal retrieval between histology slides and clinical reports without fine-tuning [4]. This functionality is particularly valuable for resource-limited clinical scenarios and rare disease retrieval. TITAN's training incorporates 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology, enhancing its language understanding capabilities [4].

Experimental Methodologies and Workflows

Whole Slide Image Preprocessing Pipeline

Computational pathology workflows begin with comprehensive WSI preprocessing and quality control. PathOrchestra demonstrates robust performance across 12 preprocessing tasks, achieving accuracy and F1 scores exceeding 0.950 in 7 subtasks [52]. These tasks include:

• Quality Control: Wrinkle detection, bubble and adhesive identification, contamination detection, and blur detection
• General Analysis: Pathology image identification, H&E and IHC staining recognition, IHC marker type identification, magnification recognition, frozen vs. FFPE section differentiation, and biopsy versus large specimen identification

Self-Supervised Training Methodology

The DINOv2 framework, employed by both Virchow and PathOrchestra, utilizes a knowledge distillation approach with a teacher-student architecture [26] [61]. The training process involves:

• Multi-crop Strategy: Generating multiple views of the same image through global and local crops with diverse augmentations
• Feature Alignment: The student network processes both global and local crops while the teacher network processes only global crops
• Loss Computation: Combining DINO loss (aligning global features between student and teacher) and iBOT loss (masked image modeling)
• Exponential Moving Average: Slowly updating teacher parameters from the student network to maintain training stability

Multimodal Alignment Training (TITAN)

TITAN employs a sophisticated three-stage training approach for vision-language alignment [4]:

• Vision-Only Pretraining: Application of iBOT framework on ROI crops from 335,645 WSIs
• ROI-Level Cross-Modal Alignment: Contrastive learning with 423,122 synthetic fine-grained ROI captions
• WSI-Level Cross-Modal Alignment: Alignment with 182,862 medical reports at the whole-slide level

Research Reagent Solutions and Essential Materials

Table 3: Essential Research Reagents and Computational Resources for Pathology Foundation Models

Resource Category	Specific Solution	Function in Workflow	Example Specifications
Whole Slide Imaging Scanners	Philips IntelliSite, SQS-600P, KF-PRO-005, Aperio ScanScope GT 450, Pannoramic MIDI II	Digitizes glass slides into high-resolution WSIs for computational analysis	20×-40× objectives, formats: .svs, .sdpc, .kfb, .mdsx [52]
Pathology Datasets	TCGA, CAMELYON, CPTAC, In-house institutional datasets	Provides diverse, multi-organ data for model training and validation	PathOrchestra: 300K WSIs; Virchow: 1.5M WSIs [52] [26]
Self-Supervised Learning Frameworks	DINOv2, iBOT, MAE, MoCo	Enables pre-training on unlabeled data through contrastive learning objectives	Teacher-student architecture, multi-crop strategies [26] [61]
Computational Infrastructure	GPU clusters, High-performance computing	Handles massive computational requirements for training on gigapixel images	PathOrchestra: 262.5 TB training data [61]
Annotation Platforms	Digital pathology annotation tools	Enables region-of-interest labeling for supervised fine-tuning	ROI, patch, and pixel-level annotations [50]

The development of foundation models in computational pathology represents a paradigm shift from task-specific models to versatile, general-purpose frameworks capable of addressing diverse clinical challenges. Virchow demonstrates the value of scale, with its 1.5-million WSI training corpus enabling robust pan-cancer detection, particularly for rare cancers [26]. PathOrchestra establishes new benchmarks through comprehensive evaluation across 112 clinical tasks, showing exceptional performance in structured report generation for complex diagnostic scenarios [52] [61]. TITAN advances multimodal capabilities through vision-language alignment, enabling zero-shot classification and cross-modal retrieval without fine-tuning [4].

Future research directions include developing more efficient architectures to reduce computational demands, improving explainability for clinical trust, enhancing multimodal integration with genomic and clinical data, and establishing standardized evaluation frameworks across diverse populations and healthcare settings [50] [12]. As these models continue to evolve, they hold significant promise for transforming cancer diagnosis, biomarker discovery, and personalized treatment planning through more accessible, accurate, and efficient computational pathology solutions.

Foundation models (FMs) in computational pathology are large-scale artificial intelligence models pre-trained on vast datasets of histopathology images, often integrated with other data modalities like text reports or genomic information [38]. These models learn universal feature representations from digitized whole-slide images (WSIs) without the need for task-specific labels, thereby mitigating critical challenges such as data imbalance and heavy annotation dependency that have long constrained traditional AI approaches [38] [12]. Trained on hundreds of thousands of WSIs, foundation models capture fundamental morphological patterns in tissue architecture and cellular structure, serving as a versatile "starting point" for a wide range of downstream clinical and research applications with minimal fine-tuning [4] [5].

The transition from task-specific models to foundation models represents a paradigm shift in computational pathology research and clinical application. Traditionally, research in this field depended on the collection and labeling of large datasets for specific tasks, followed by the development of task-specific computational pathology models [38]. However, this approach is labor-intensive and does not scale efficiently for open-set identification or rare diseases [38]. Foundation models address these limitations by leveraging self-supervised learning on massive, diverse datasets, enabling unprecedented generalization capabilities across diverse diagnostic tasks [4] [5].

This technical guide examines the performance of pathological foundation models across three critical domains: morphological analysis, biomarker prediction, and clinical prognostication. Through comprehensive evaluation of state-of-the-art models and methodologies, we provide researchers and drug development professionals with experimental protocols, performance benchmarks, and implementation frameworks to advance precision oncology through computational pathology.

Foundation Model Architectures and Training Paradigms

Architectural Foundations

Pathology foundation models are predominantly built upon vision transformer (ViT) architectures adapted to handle the gigapixel-scale dimensions of whole-slide images. The computational challenge posed by WSIs, which can exceed 100,000 × 100,000 pixels, necessitates a hierarchical processing approach [38]. Modern implementations, including Transformer-based pathology Image and Text Alignment Network (TITAN), process WSIs by first dividing them into non-overlapping patches of 512 × 512 pixels at 20× magnification [4]. Each patch is encoded into a 768-dimensional feature vector using a pre-trained patch encoder, spatially arranged in a two-dimensional feature grid that replicates the original tissue architecture [4].

To manage the long and variable input sequences inherent to WSI analysis (>10^4 tokens versus 196-256 tokens at patch-level), innovative solutions have been developed. TITAN employs attention with linear bias (ALiBi) extended to 2D, where the linear bias is based on the relative Euclidean distance between features in the feature grid, enabling long-context extrapolation during inference [4]. This approach preserves spatial relationships while managing computational complexity, allowing the model to capture both local cellular features and global tissue architecture.

Multimodal Integration

The most advanced foundation models incorporate multimodal capabilities, aligning visual features with corresponding pathology reports and other clinical data. TITAN undergoes a three-stage pretraining strategy: (1) vision-only unimodal pretraining on region crops, (2) cross-modal alignment of generated morphological descriptions at ROI-level using 423k pairs of ROIs and synthetic captions, and (3) cross-modal alignment at WSI-level using 183k pairs of WSIs and clinical reports [4]. This multimodal approach enables capabilities such as pathology report generation, cross-modal retrieval between histology slides and clinical reports, and zero-shot classification without requiring fine-tuning or clinical labels [4] [39].

Figure 1: Multimodal training pipeline for pathology foundation models, integrating visual and textual data across multiple stages.

Scaling Laws and Optimization

Current evidence challenges assumptions about scaling in histopathological applications. A comprehensive benchmarking study of 31 AI foundation models revealed that model size and data size did not consistently correlate with improved performance [22]. This finding suggests that factors beyond mere scale—such as data diversity, training methodology, and architectural optimizations—play critical roles in determining model capability. Virchow2, a pathology foundation model, delivered the highest performance across TCGA, CPTAC, and external tasks in comparative evaluations, highlighting the importance of specialized architectural considerations rather than pure scaling [22].

Performance Across Morphological Tasks

Pan-Cancer Classification

Foundation models demonstrate exceptional performance in pan-cancer classification, accurately distinguishing between multiple cancer types from histology images. PathOrchestra, trained on 287,424 slides from 21 tissue types, achieved an average AUC of 0.988 in a 17-class pan-cancer tissue classification task using an in-house FFPE dataset [5]. Notably, the model achieved perfect scores (ACC, AUC, and F1 = 1.0) in prostate cancer classification, attributed to the consistent and visually distinctive features of needle biopsy samples compared to other organ types that are mostly large surgical specimens [5].

Performance varies between tissue preparation methods, with frozen sections generally showing lower classification metrics compared to FFPE sections. In 32-class classification tasks using TCGA data, PathOrchestra attained an AUC of 0.964 for FFPE samples versus 1.4% lower for frozen sections, with ACC and F1 scores approximately 9% lower for frozen tissue [5]. This discrepancy likely reflects the superior preservation of tissue structure and morphology in FFPE sections, which facilitates more effective feature extraction.

Table 1: Pan-Cancer Classification Performance of PathOrchestra Across Tissue Types and Preparation Methods

Classification Task	Tissue Types	Sample Type	AUC	Accuracy	F1 Score
17-class pan-cancer	17 organs	FFPE (in-house)	0.988	0.879	0.863
32-class classification	32 cancer types	FFPE (TCGA)	0.964	0.666	0.667
32-class classification	32 cancer types	Frozen (TCGA)	0.950	0.577	0.577
Prostate classification	Prostate	FFPE (needle biopsy)	1.0	1.0	1.0

Disease Subtyping and Histologic Grading

Comprehensive benchmarking across diverse cancer types reveals consistent performance advantages for foundation model-based approaches. The nnMIL framework, which connects patch-level foundation models to robust slide-level clinical inference, has been evaluated across 40,000 WSIs encompassing 35 clinical tasks [62]. In disease subtyping tasks, nnMIL achieved an average balanced accuracy of 80.7-82.0% across eight challenging classification tasks including skin cancer subtyping (2-class, 3-class, and 5-class), breast cancer subtyping (7-class), brain tumour subtyping (12-class and 30-class), colorectal cancer diagnosis, and Gleason grading of prostate cancer [62].

Compared to conventional multiple instance learning (MIL) methods, nnMIL demonstrated significant improvements, outperforming the second-best method (ABMIL) by 2.6-3.8% across four different pathology foundation models [62]. The performance advantage was most pronounced on complex fine-classification tasks, with a 10.5% relative improvement in balanced accuracy on the EBRAINS Fine classification task (0.724 vs 0.656, p < 0.001) [62]. These results highlight the particular value of foundation models in distinguishing subtle histological patterns across disease subtypes.

Experimental Protocol: Disease Subtyping

Data Preparation:

• Collect whole-slide images from formalin-fixed paraffin-embedded (FFPE) tissue blocks stained with hematoxylin and eosin (H&E)
• Perform quality control to exclude slides with significant artifacts, bubbles, adhesive, or blur
• Segment tissue regions using automated algorithms, excluding background areas
• Extract patch-level features using a pre-trained foundation model (e.g., Virchow2, UNI, or GigaPath) with a patch size of 256×256 or 512×512 pixels at 20× magnification

Model Training:

• Implement the nnMIL framework with random sampling at both patch and feature levels
• Use a task-specific batch sampler to improve data utilization across different classification tasks
• Train with large-batch optimization enabled by fixed-length sub-bags converted from variable-length bags
• Employ a sliding-window inference scheme to generate ensemble predictions and uncertainty estimates

Evaluation Metrics:

• Calculate balanced accuracy, F1 score, and area under the receiver operating characteristic curve (AUC)
• Perform cross-validation with official or widely adopted data splits for fair comparison
• Conduct statistical significance testing using paired t-tests or appropriate non-parametric alternatives

Biomarker Prediction Capabilities

Molecular Alteration Prediction

Foundation models demonstrate remarkable capability in predicting molecular biomarkers directly from H&E-stained whole-slide images, potentially reducing reliance on costly ancillary tests. Johnson & Johnson Innovative Medicine's MIA:BLC-FGFR algorithm predicts Fibroblast Growth Factor Receptor (FGFR) alterations in non-muscle invasive bladder cancer (NMIBC) patients directly from H&E-stained slides with 80-86% AUC, showing strong concordance with traditional testing [41]. This approach is particularly valuable for NMIBC, where scarce tissue samples often struggle to meet the high nucleic acid requirements of traditional molecular approaches.

Spatial transcriptomic prediction represents another advanced application, with HE2RNA demonstrating capability to predict transcriptome profiles from histology slides, providing virtual spatialization of gene expression and transferable predictions for molecular phenotypes [38]. Deep residual learning models can predict microsatellite instability directly from H&E-stained images, potentially broadening access to immunotherapy for gastrointestinal cancer patients by eliminating the need for additional genetic or immunohistochemical tests [38].

Table 2: Biomarker Prediction Performance from H&E-Stained Whole Slide Images

Biomarker Type	Cancer Type	Model	Performance	Clinical Utility
FGFR alterations	Non-muscle invasive bladder cancer	MIA:BLC-FGFR	AUC: 80-86%	Identifies candidates for FGFR-targeted therapies
Microsatellite instability	Gastrointestinal cancer	Deep residual learning	High concordance with standard testing	Broadens immunotherapy access
Transcriptome profiles	Pan-cancer	HE2RNA	Accurate gene expression prediction	Virtual spatialization of gene expression
Inflammatory activation pathways	Lung cancer	DPCT	Accurate pathway activity assessment	Distinguishes adenocarcinoma vs squamous cell carcinoma
p53abn endometrial cancer	Endometrial cancer	AI-driven histopathological analysis	Identifies distinct prognostic subgroups	Detects "p53abn-like NSMP" group with worse survival

Immunotherapy Biomarkers

The tumor microenvironment contains critical spatial biomarkers for immunotherapy response that foundation models can quantify with high precision. Stanford University researchers developed a five-feature model analyzing interactions between tumor cells, fibroblasts, T-cells, and neutrophils that achieved a hazard ratio of 5.46 for progression-free survival in advanced non-small cell lung cancer (NSCLC) patients treated with immune checkpoint inhibitors—significantly outperforming PD-L1 tumor proportion scoring alone (HR=1.67) [41]. This spatial analysis capability represents a paradigm shift, moving beyond protein expression levels to quantify complex cellular interactions within the tumor microenvironment.

Quantitative Continuous Scoring (QCS), a computational pathology solution developed by AstraZeneca, has shown significant promise in enriching patient populations for targeted therapies. In a retrospective analysis of the NSCLC trial TROPION-Lung02 evaluating Dato-DXd with pembrolizumab ± chemotherapy, QCS-positive patients in both dual therapy and triple therapy cohorts showed a trend toward prolonged progression-free survival compared to QCS-negative patients [41]. The clinical relevance of this AI-derived biomarker is now being prospectively validated in ongoing pivotal studies (TROPION-Lung07/08) with stratification by QCS status [41].

Experimental Protocol: Biomarker Prediction

Data Requirements:

• Whole-slide images with matched molecular profiling data (genomic, transcriptomic, or protein expression)
• Clinical annotation including treatment history and response outcomes
• Sample partitioning into discovery, validation, and hold-out test sets

Feature Extraction:

• Process WSIs using a pathology foundation model to generate tile-level embeddings
• Apply spatial aggregation methods to preserve tissue architecture information
• Integrate clinical variables (age, stage, grade) with image-derived features

Model Development:

• Implement cross-modal alignment between histology features and biomarker status
• Utilize attention mechanisms to identify predictive morphological regions
• Apply regularization techniques to prevent overfitting, given the high-dimensional feature space
• Incorporate uncertainty estimation to flag low-confidence predictions

Validation Framework:

• Assess performance using area under the curve (AUC), sensitivity, specificity
• Evaluate clinical utility through decision curve analysis
• Perform external validation on independent cohorts from different institutions
• Compare against standard-of-care biomarker testing methods

Prognostication and Survival Analysis

Outcome Prediction

Foundation models excel at extracting prognostically relevant features from tumor histology that may not be apparent through human assessment. In stage III colon cancer, the CAPAI (Combined Analysis of Pathologists and Artificial Intelligence) biomarker, an AI-driven score using H&E slides and pathological stage data, effectively stratified recurrence risk even in ctDNA-negative patients [41]. Among ctDNA-negative patients, CAPAI high-risk individuals showed 35% three-year recurrence rates versus 9% for low/intermediate-risk patients, identifying a substantial patient group with very low recurrence risk who might be candidates for therapy de-escalation [41].

Multimodal AI approaches that integrate histology with clinical variables demonstrate particularly robust prognostic performance. Researchers from UCSF and Artera externally validated a pathology-based multimodal AI (MMAI) biomarker for predicting prostate cancer outcomes after radical prostatectomy [41]. Using H&E images from RP specimens alongside clinical variables, the model independently predicted metastasis and bone metastasis, with high-risk patients showing significantly higher 10-year risk of metastasis (18% vs. 3% for low-risk) [41]. This approach combines the rich information content of histology with established clinical prognostic factors.

Figure 2: Multimodal prognostication framework integrating histology, clinical, and molecular data for comprehensive outcome prediction.

Treatment Response Prediction

Foundation models show increasing utility in predicting response to specific therapeutic interventions, potentially guiding treatment selection in precision oncology. The QCS computational pathology solution has been granted Breakthrough Device Designation by the U.S. FDA as a cancer companion test—representing the first time an AI-based computational pathology device has received this status [41]. This regulatory milestone underscores the growing acceptance of AI-derived histopathological biomarkers in clinical decision-making, particularly for therapy selection.

Spatial biomarkers derived from foundation model analysis provide unique insights into treatment mechanisms and resistance patterns. By quantifying complex cellular interactions within the tumor microenvironment, these models can identify features predictive of response to immunotherapies, targeted therapies, and conventional chemotherapy regimens [41] [63]. The ability to predict treatment response from standard H&E slides could significantly reduce costs and turnaround times compared to current biomarker testing approaches.

Experimental Protocol: Survival Analysis

Data Curation:

• Collect whole-slide images with linked survival data, including time-to-event and censoring information
• Ensure adequate follow-up duration and event rates for robust statistical analysis
• Preprocess images to standardize staining variations using normalization algorithms

Feature Engineering:

• Extract tile-level features using pre-trained foundation models
• Apply multiple instance learning frameworks to aggregate tile-level predictions
• Integrate clinical covariates (age, stage, treatment history) with deep learning features

Model Training:

• Implement Cox proportional hazards models with regularization
• Utilize deep survival approaches that directly model hazard functions from image data
• Incorporate ensemble methods to improve robustness and calibration
• Apply time-dependent accuracy metrics to evaluate discriminative performance

Validation Strategy:

• Assess performance using time-dependent AUC and concordance index (C-index)
• Evaluate calibration using observed vs. predicted survival curves
• Perform bootstrapping to obtain confidence intervals for performance metrics
• Conduct external validation across multiple institutions and patient populations

Implementation Frameworks and Methodologies

Multiple Instance Learning Frameworks

The nnMIL framework represents a significant advancement in connecting patch-level foundation model features to slide-level clinical predictions. This framework introduces random sampling at both the patch and feature levels, enabling large-batch optimization—a fundamental limitation of conventional MIL approaches that are constrained to batch size of one due to varying patch counts across WSIs [62]. By partitioning variable-length bags into fixed-length sub-bags, nnMIL supports substantially larger and more balanced batch sizes, improving training efficiency, stability, and overall performance [62].

A key innovation in modern MIL implementations is the incorporation of uncertainty quantification. nnMIL employs a sliding-window inference scheme that integrates predictions from multiple overlapping sub-sampled embeddings, functioning as an ensemble and providing principled uncertainty estimates for model outputs [62]. This capability is particularly valuable in clinical settings, where understanding model confidence directly impacts decision-making. In selective prediction experiments, nnMIL's performance on retained slides increased steadily as slides with the highest uncertainty scores were excluded, demonstrating well-calibrated uncertainty estimates [62].

The Scientist's Toolkit: Essential Research Reagents

Table 3: Essential Research Reagents and Computational Resources for Foundation Model Research

Resource Category	Specific Tools/Solutions	Function/Purpose	Key Characteristics
Foundation Models	TITAN, PathOrchestra, Virchow2, GigaPath, UNI	Extract transferable features from WSIs	Pre-trained on 250K+ WSIs, multimodal capabilities
MIL Frameworks	nnMIL, ABMIL, TransMIL, DTFD-MIL	Aggregate patch-level features to slide-level predictions	Enable large-batch training, uncertainty quantification
Computational Infrastructure	High-performance GPUs (e.g., NVIDIA H100, A100), Argonne Leadership Computing Facility	Process large-scale WSI datasets	Distributed training capabilities, large memory capacity
Data Resources	TCGA, CPTAC, CAMELYON, PANDA, in-house institutional datasets	Model training and validation	Diverse cancer types, linked clinical and molecular data
Validation Frameworks	DAPPER, benchmark tasks from TCGA and external datasets	Standardized performance assessment	Multiple tissue types, cross-institutional validation
Pathology Image Databases	Concentriq platform, Aperio ScanScope, 3DHISTECH Pannoramic	Image storage, management, and analysis	Whole-slide image support, integration with AI tools

Foundation models in computational pathology demonstrate robust performance across diverse tasks including morphological classification, biomarker prediction, and clinical prognostication. The quantitative evidence presented in this technical guide reveals consistent performance advantages over traditional approaches, with particularly notable capabilities in predicting molecular alterations from standard H&E stains and stratifying patient risk with precision that complements or exceeds conventional biomarkers.

As the field advances, key opportunities and challenges emerge. The integration of multimodal data—combining histology with clinical, genomic, and transcriptomic information—represents a promising direction for enhancing predictive accuracy and clinical utility [41] [22]. Additionally, the development of standardized benchmarking frameworks and rigorous external validation protocols will be essential for translating these technologies from research to clinical practice [22] [64]. With ongoing advances in model architectures, training methodologies, and implementation frameworks, foundation models are poised to fundamentally transform pathology practice and precision oncology.

Foundation models in computational pathology are large-scale deep neural networks, such as Virchow and TITAN, pretrained on massive datasets of histopathology whole-slide images (WSIs) using self-supervised learning algorithms [4] [26]. These models generate versatile feature representations (embeddings) that capture fundamental morphological patterns in tissue, including cellular morphology, tissue architecture, staining characteristics, and nuclear features, enabling them to serve as a "base" for various downstream diagnostic tasks without requiring task-specific training from scratch [26]. The transformative potential of these models lies in their ability to generalize—to maintain high performance when applied to data from new healthcare institutions (cross-institutional generalization) or to data with statistical distributions different from the training set (out-of-distribution generalization) [65] [66].

Robust generalizability assessment is paramount for clinical deployment, as models trained on data from a single institution often face performance degradation when applied externally due to variations in patient populations, clinical practices, laboratory preparations, imaging equipment, and data collection protocols [65] [67]. Furthermore, the accurate detection of rare cancers and conditions depends on a model's ability to handle "out-of-distribution" scenarios where training data is inherently limited [4] [26]. This technical guide provides a comprehensive framework for assessing the cross-institutional and out-of-distribution generalizability of computational pathology foundation models, featuring detailed protocols, quantitative benchmarks, and essential research tools.

Established Validation Frameworks and Performance Metrics

Core Validation Paradigms

• Internal-External Cross-Validation: This method involves iteratively training a model on data from multiple sites and validating it on data from a held-out site that was not used in training. This approach helps evaluate the need for complex modeling strategies and assesses performance heterogeneity across different clinical practices [68]. For instance, one study developed Cox regression models for heart failure risk prediction across 225 general practices, using this method to reveal that simpler models often generalized better than complex ones with minimal between-practice heterogeneity [68].

• Federated Learning and OHDSI/OMOP CDM Framework: Leveraging the Observational Health Data Sciences and Informatics (OHDSI) tools and the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) standardizes and harmonizes electronic health records (EHRs) from multiple institutions into a unified format [65]. This enables federated analysis where data remains with the data owners, and only aggregated results or model parameters are shared. Research has demonstrated that models trained with cross-site feature selection—creating feature supersets from the union or intersection of significant features across multiple databases—significantly outperform models using only site-specific features (P < 0.05) in external validation [65].

• Leave-One-Group-Out OOD Validation: This approach systematically creates OOD test sets by excluding entire categories of data during training, such as samples from specific institutions, elements in materials science, or rare cancer types in pathology [66]. For example, studies may evaluate generalization by leaving out all samples containing a specific chemical element or all WSIs of a particular cancer type, then testing performance exclusively on these excluded categories [66].

Key Performance Metrics for Generalizability Assessment

The table below summarizes the essential metrics for evaluating model generalizability across different clinical tasks and data distributions.

Table 1: Key Performance Metrics for Generalizability Assessment

Metric	Interpretation	Use Case in Generalizability
Area Under Receiver Operating Characteristic Curve (AUC/AUROC)	Overall discrimination ability between classes. Values closer to 1.0 indicate better performance.	Primary metric for cancer detection [26] and biomarker prediction tasks across institutions.
Coefficient of Determination (R²)	Proportion of variance in the outcome explained by the model. Dimensionless with range to negative infinity.	Used in regression tasks to assess OOD prediction accuracy, especially with systematic biases [66].
Calibration Slope and Observed/Expected Ratio	Agreement between predicted probabilities and actual outcomes. Slope of 1 indicates perfect calibration.	Measures reliability of probabilistic predictions across different patient populations and clinical sites [68].
Between-Site Heterogeneity in Performance	Variance in metrics (e.g., AUC, calibration) across different validation sites.	Quantifies consistency of model performance, where lower heterogeneity indicates better generalizability [68].
Specificity at Fixed Sensitivity	Model's ability to correctly identify negatives when sensitivity is constrained (e.g., 95%).	Critical for clinical applications where false positive rates must be controlled across diverse populations [26].

Quantitative Benchmarks: Current Foundation Models and Their Generalizability

Recent comprehensive benchmarking studies have evaluated numerous AI foundation models for computational pathology, including general vision models (VM), vision-language models (VLM), pathology-specific vision models (Path-VM), and pathology-specific vision-language models (Path-VLM) across diverse tasks and datasets [22]. The following table synthesizes performance data for leading foundation models in computational pathology, highlighting their generalization capabilities.

Table 2: Performance Benchmarks of Pathology Foundation Models on Generalization Tasks

Foundation Model	Pretraining Data Scale	Key Architecture	Pan-Cancer Detection AUC (Overall)	Rare Cancer Detection AUC	Cross-Institutional Generalization Performance
Virchow [26]	1.5M WSIs from 100k patients	632M parameter ViT, DINO v2 algorithm	0.950	0.937 (across 7 rare cancers)	Maintains stable AUC on external institution data without performance degradation
TITAN [4] [39]	335,645 WSIs + 423k synthetic captions	Transformer-based multimodal architecture	Outperforms other slide foundation models across multiple tasks	Excels in rare disease retrieval and cancer prognosis	Generates pathology reports and enables cross-modal retrieval in resource-limited scenarios
UNI [26]	Not specified in results	Not specified in results	0.940	Comparable to Virchow for 5/7 rare cancers	Statistically similar performance to Virchow on external data for most cancer types
Virchow 2 [22]	Not specified in results	Not specified in results	Highest performance across TCGA, CPTAC, and external tasks	Top rankings across diverse tissue types	Superior generalization in external benchmarks and fusion models

Key findings from recent benchmarks indicate that pathology-specific vision models (Path-VMs) generally outperform both general vision models and vision-language models, with Virchow2 achieving the highest overall performance across multiple tasks and datasets [22]. Notably, model size and pretraining data size do not consistently correlate with improved performance, challenging conventional scaling assumptions in histopathological applications [22].

Experimental Protocols for Generalizability Assessment

Protocol 1: Cross-Institutional Validation for Clinical Endpoint Prediction

This protocol outlines the methodology used in studies evaluating models for predicting post-surgery prolonged opioid use (POU) across multiple countries [65].

• Data Harmonization: Map electronic health records (EHRs) from multiple institutions to the OMOP Common Data Model to standardize structure and content. This includes uniform representation of demographics, clinical conditions, procedures, measurements, and drug exposures [65].

• Cohort Definition: Apply consistent inclusion/exclusion criteria across sites. For POU prediction, include adult patients who underwent surgery (2008-2019) with opioid prescriptions 30 days before/after surgery. Exclude patients with additional surgeries within 2-7 months post-index surgery or who died within one year [65].

• Cross-Site Feature Selection:

  • Divide the target cohort at each site into train/test sets (80:20) using stratified random sampling based on the outcome.
  • Calculate chi-square metric for all features in the training set based on their distribution between positive and negative outcomes.
  • Select features significant at P < 0.01 for each site.
  • Create two feature supersets:
    • Superset-union: Union of significant features from all development sites.
    • Superset-intersect: Intersection of site-specific features, excluding features unique to single databases, with additional filtering using Positive-Negative Frequency (PNF > 1.5) metric [65].

• Model Training and Validation: Train multiple machine learning algorithms (e.g., Lasso logistic regression, random forest) using the different feature sets. Validate models internally on held-out test sets and externally on 100% of target cohorts from completely separate institutions [65].

Protocol 2: OOD Generalization Assessment for Rare Cancer Detection

This protocol is derived from methodologies used to evaluate foundation models like Virchow and TITAN on challenging detection tasks [4] [26].

• Dataset Stratification: Split whole-slide image datasets by cancer type, specifically creating evaluation sets for rare cancers (defined by NCI as annual incidence <15/100,000). Ensure these rare cancer types are excluded from foundation model pretraining when assessing OOD capabilities [26].

• Slide-Level Embedding Extraction: Process WSIs through the foundation model to generate tile-level embeddings. For Virchow, this uses a 632M parameter Vision Transformer processed via DINO v2, which learns from both global and local tissue regions [26].

• Weakly Supervised Aggregation: Train a pan-cancer detection aggregator model using attention-based multiple-instance learning (MIL) to combine tile embeddings into slide-level predictions. Use only slide-level labels without pixel-level annotations [26] [36].

• OOD Performance Evaluation: Evaluate model performance exclusively on rare cancer slides and external institution data that were not part of the training set. Compare AUC, sensitivity, and specificity metrics against in-distribution performance and performance on common cancers [26].

Protocol 3: Zero-Shot and Few-Shot Evaluation

For multimodal foundation models like TITAN, additional language-enabled capabilities can be assessed [4] [39].

• Zero-Shot Classification: Evaluate the model's ability to classify histopathology images without task-specific fine-tuning by leveraging its vision-language alignment. Use text prompts describing diagnostic categories and assess cross-modal retrieval accuracy [4].

• Few-Shot Learning: Fine-tune the foundation model with limited labeled examples (e.g., 1-10 samples per class) from new institutions or rare diseases. Compare performance against models trained from scratch on the same limited data [4].

• Cross-Modal Retrieval: Test the model's ability to retrieve relevant histopathology images based on textual descriptions from pathology reports, and vice versa, across institutional boundaries [4] [39].

Visualization of Validation Workflows

Generalizability Assessment Workflow: This diagram illustrates the comprehensive validation pipeline for pathology foundation models, spanning data harmonization, model development, and rigorous multi-faceted performance assessment.

OOD Generalization Assessment Framework: This diagram outlines methodologies for creating challenging out-of-distribution tasks and analyzing model performance on true extrapolation versus interpolation scenarios.

Table 3: Essential Research Reagents and Computational Tools for Generalizability Assessment

Resource Category	Specific Tools/Platforms	Function in Generalizability Research
Data Harmonization Standards	OMOP Common Data Model (CDM) [65]	Standardizes EHR structure and content across institutions, enabling cross-site feature selection and federated analysis.
Federated Analysis Platforms	OHDSI (Observational Health Data Sciences and Informatics) [65]	Enables model development and validation across multiple institutions without sharing raw patient data.
Pathology Foundation Models	Virchow [26], TITAN [4], UNI [26]	Pretrained models that generate transferable feature representations for diverse downstream tasks across institutions.
Weakly Supervised Learning Algorithms	CLAM (Clustering-constrained Attention Multiple-instance Learning) [36]	Enables whole-slide classification using only slide-level labels, critical for data-efficient adaptation to new institutions.
Benchmarking Datasets	TCGA, CPTAC, JARVIS, Materials Project [66] [22]	Standardized datasets with multiple cancer types and materials for systematic OOD generalization assessment.
Model Interpretation Frameworks	SHAP (SHapley Additive exPlanations) [66]	Identifies sources of prediction bias by quantifying feature contributions, especially for poor OOD performance.

Robust generalizability assessment through cross-institutional and out-of-distribution validation is fundamental for translating computational pathology foundation models from research tools to clinical decision support systems. The methodologies outlined in this guide—including internal-external cross-validation, cross-site feature selection, and rigorous OOD benchmarking—provide a framework for evaluating model performance across diverse healthcare settings and patient populations.

Future research directions should focus on developing more challenging OOD benchmarks that truly test extrapolation capabilities rather than interpolation [66], improving model robustness to domain shifts through advanced regularization and domain adaptation techniques, and establishing standardized reporting guidelines for generalizability metrics in computational pathology. Fusion models that integrate multiple top-performing foundation models show particular promise for achieving superior generalization across external tasks and diverse tissue types [22]. As foundation models continue to evolve in scale and capability, rigorous and standardized generalizability assessment will remain essential for ensuring their reliability, equity, and effectiveness in real-world clinical practice.

Conclusion

Foundation models represent a paradigm shift in computational pathology, offering unprecedented generalization across diverse diagnostic and predictive tasks. Key takeaways indicate that no single model dominates all scenarios; instead, ensembles and strategic model selection based on specific tasks yield optimal performance. Future directions point toward larger multimodal models integrating histology with genomic and clinical data, increased focus on clinical validation for regulatory approval, and the development of more efficient architectures to facilitate widespread clinical adoption. For researchers and drug developers, these models are poised to accelerate biomarker discovery, enhance therapeutic R&D, and ultimately power the next generation of precision medicine tools by unlocking the rich morphological information embedded in pathology images.

References

• 1. Computational pathology in the age of artificial intelligence [https://pmc.ncbi.nlm.nih.gov/articles/PMC12451823/]
• 2. Pathology Foundation Models - PMC - PubMed Central [https://pmc.ncbi.nlm.nih.gov/articles/PMC11799676/]
• 3. Artificial intelligence and computational pathology [https://www.nature.com/articles/s41374-020-00514-0]
• 4. A multimodal whole-slide foundation model for pathology [https://www.nature.com/articles/s41591-025-03982-3]
• 5. PathOrchestra: a comprehensive foundation model for ... [https://www.nature.com/articles/s41746-025-02027-w]
• 6. A visual-language foundation model for computational ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11384335/]
• 7. Tissue concepts: Supervised foundation models in ... [https://www.sciencedirect.com/science/article/pii/S0010482524017062]
• 8. Foundation model [https://en.wikipedia.org/wiki/Foundation_model]
• 9. A clinical benchmark of public self-supervised pathology ... [https://www.nature.com/articles/s41467-025-58796-1]
• 10. Challenges and Limitations of Supervised Learning in AI ... [https://medium.com/@adibostoninstitute123/challenges-and-limitations-of-supervised-learning-in-ai-models-de627a06c1a6]
• 11. Computational pathology: A comprehensive review of ... [https://www.sciencedirect.com/science/article/pii/S2950261625000214]
• 12. Foundation Models for Computational Pathology [https://www.sciencedirect.com/special-issue/324717/foundation-models-for-computational-pathology]
• 13. A clinical benchmark of public self-supervised pathology ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12003829/]
• 14. Adapting Self-Supervised Learning for Computational ... [https://arxiv.org/html/2405.01688v1]
• 15. Self-supervised learning for data augmentation in ... [https://www.nature.com/articles/s41598-025-23280-9]
• 16. Reducing the workload of medical diagnosis through ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11813001/]
• 17. Foundation Models in Digital Pathology: The Good, the Bad ... [https://beyondtheslide.substack.com/p/foundation-models-in-digital-pathology]
• 18. A Survey on Computational Pathology Foundation Models [https://arxiv.org/html/2501.15724v2]
• 19. Deep learning for digital pathology: A critical overview of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12593639/]
• 20. Comparing Computational Pathology Foundation Models ... [https://arxiv.org/html/2509.15482v1]
• 21. Deep learning quantifies pathologists' visual patterns for ... [https://www.nature.com/articles/s41467-025-60307-1]
• 22. Evaluating Vision and Pathology Foundation Models for ... [https://www.medrxiv.org/content/10.1101/2025.05.08.25327250v1]
• 23. Deep learning-based image classification for AI-assisted ... [https://www.frontiersin.org/journals/medicine/articles/10.3389/fmed.2025.1574514/full]
• 24. A Survey on Computational Pathology Foundation Models [https://arxiv.org/abs/2501.15724]
• 25. A Survey on Computational Pathology Foundation Models [https://arxiv.org/html/2501.15724v1]
• 26. A foundation model for clinical-grade computational ... [https://www.nature.com/articles/s41591-024-03141-0]
• 27. Towards a General-Purpose Foundation Model for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11403354/]
• 28. Contrastive learning-based histopathological features infer ... [https://pubmed.ncbi.nlm.nih.gov/38271839/]
• 29. Generalized deep learning for histopathology image ... [https://www.sciencedirect.com/science/article/pii/S2090123224005320]
• 30. Large-vocabulary forensic pathological analyses via ... [https://www.nature.com/articles/s41467-025-62060-x]
• 31. Closing the gap in the clinical adoption of computational ... [https://genomemedicine.biomedcentral.com/articles/10.1186/s13073-025-01484-y]
• 32. A Survey of Pathology Foundation Model [https://www.ijcai.org/proceedings/2025/1193]
• 33. A Survey on Foundation and Vision-Language Models [https://arxiv.org/abs/2408.14496]
• 34. A multimodal generative AI copilot for human pathology [https://www.nature.com/articles/s41586-024-07618-3]
• 35. Evaluating Vision and Pathology Foundation Models for ... [https://pubmed.ncbi.nlm.nih.gov/40463538/]
• 36. Data-efficient and weakly supervised computational pathology ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8711640/]
• 37. Foundation Models in Computational Pathology: A Review ... [https://arxiv.org/abs/2502.08333]
• 38. Computational pathology in precision oncology [https://pmc.ncbi.nlm.nih.gov/articles/PMC12634262/]
• 39. A multimodal whole-slide foundation model for pathology [https://pubmed.ncbi.nlm.nih.gov/41193692/]
• 40. Towards a general-purpose foundation model for ... - PubMed [https://pubmed.ncbi.nlm.nih.gov/38504018/]
• 41. Digital Pathology and AI Highlights from ASCO 2025 [https://proscia.com/digital-pathology-and-ai-highlights-from-asco-2025/]
• 42. PLUTO-4: Frontier Pathology Foundation Models [https://arxiv.org/html/2511.02826v1]
• 43. 数字化和综合病理学在易位检测中的作用：叙述性回顾 [https://pcm.amegroups.org/article/view/6825/10710]
• 44. Benchmarking Pathology Foundation Models: Adaptation ... [https://arxiv.org/html/2410.16038v1]
• 45. Benchmarking pathology foundation models: Adaptation ... [https://pubmed.ncbi.nlm.nih.gov/40179809/]
• 46. Real-world deployment of a fine-tuned pathology ... [https://www.nature.com/articles/s41591-025-03780-x]
• 47. 病理切片数据不够，预测效果不佳？快来看看Nature发表的 ... [https://blog.csdn.net/m0_59235245/article/details/143028154]
• 48. A systematic review of few-shot learning in medical imaging [https://www.sciencedirect.com/science/article/pii/S093336572400191X]
• 49. 零点学习（ZSL）详解 [https://www.ultralytics.com/zh/glossary/zero-shot-learning]
• 50. Computational pathology: A survey review and the way forward [https://pmc.ncbi.nlm.nih.gov/articles/PMC10900832/]
• 51. 【阅读笔记】MI-Zero：视觉语言模型+ 多实例学习，零样本迁移 ... [https://blog.csdn.net/qq_43538018/article/details/143957558]
• 52. PathOrchestra: a comprehensive foundation model for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12630941/]
• 53. Benchmarking foundation models as feature extractors for ... [https://www.nature.com/articles/s41551-025-01516-3]
• 54. What AI can tell us about microscope slides – fast.ai [https://www.fast.ai/posts/2025-02-03-cpath-foundation/]
• 55. Clinician Perspectives on Digital and Computational ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12523993/]
• 56. Uncertainty-aware ensemble of foundation models ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12480093/]
• 57. Using ensembles and distillation to optimize the ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9469924/]
• 58. Using Ensembles and Distillation to Optimize the Deployment ... [https://computational.cancer.gov/publication/using-ensembles-and-distillation-optimize-deployment-deep-learning-models]
• 59. Benchmarking foundation models as feature extractors for ... [https://pubmed.ncbi.nlm.nih.gov/41034516/]
• 60. The Comparative Pathology Workbench: Interactive visual ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10491844/]
• 61. PathOrchestra: A Comprehensive Foundation Model for ... [https://arxiv.org/html/2503.24345v1]
• 62. nnMIL: A generalizable multiple instance learning ... [https://arxiv.org/html/2511.14907v1]
• 63. Challenges in Computational Pathology of Biomarker ... [https://link.springer.com/rwe/10.1007/978-3-032-00775-9_334]
• 64. Computational pathology for breast cancer: Where do we ... [https://www.sciencedirect.com/science/article/pii/S0960977625004813]
• 65. Towards global model generalizability: independent cross ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11031239/]
• 66. Probing out-of-distribution generalization in machine ... [https://www.nature.com/articles/s43246-024-00731-w]
• 67. Generalizability of SuperAlarm via Cross-Institutional ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7971165/]
• 68. Internal-external cross-validation helped to evaluate the ... [https://www.sciencedirect.com/science/article/pii/S0895435621001074]