Mass-100K & Mass-340K: The Pathology Foundation Model Datasets Powering a New Era in AI-Driven Diagnostics

Abstract

This article provides a comprehensive analysis of the Mass-100K and Mass-340K datasets, foundational resources revolutionizing computational pathology. Tailored for researchers and drug development professionals, it explores the scale, composition, and origins of these datasets. The content details their critical role in training versatile models like UNI and TITAN for tasks ranging from cancer subtyping to biomarker prediction. It further examines the methodologies for leveraging these datasets, addresses key computational challenges, and validates their performance against established benchmarks. Finally, the discussion synthesizes how these datasets are accelerating the development of robust, general-purpose AI tools for clinical and research applications.

Unpacking Mass-100K and Mass-340K: The Foundational Data Powering Next-Gen Pathology AI

In the rapidly evolving field of computational pathology (CPath), the development of robust foundation models is critically dependent on large-scale, diverse, and well-curated datasets. Among the most significant resources enabling recent advancements are the Mass-100K and Mass-340K datasets, which have served as the foundational pretraining corpora for pioneering models such as UNI and TITAN [1] [2]. These datasets have pushed the boundaries of scale and diversity in histopathology data, moving the field beyond the constraints of earlier collections like The Cancer Genome Atlas (TCGA). This technical guide provides a comprehensive analysis of the scale, composition, and origin of these two pivotal datasets, framing them within the broader context of pathology foundation model research. Understanding their precise characteristics is essential for researchers, scientists, and drug development professionals aiming to leverage, evaluate, or build upon these foundational resources.

The Mass-100K and Mass-340K datasets represent consecutive generations of scale and complexity in histopathology data collection. Mass-100K, introduced with the UNI model, marked a significant step up from previous benchmarks [1]. Its successor, Mass-340K, expanded this paradigm further in both volume and multimodal richness for the development of TITAN, a whole-slide foundation model [2]. The table below provides a detailed quantitative comparison of their core characteristics.

Table 1: Core Characteristics of Mass-100K and Mass-340K Datasets

Characteristic	Mass-100K Dataset	Mass-340K Dataset
Total Number of Whole Slide Images (WSIs)	100,426 diagnostic H&E-stained WSIs [1]	335,645 WSIs [2]
Total Number of Image Patches	>100 million tissue patches [1]	Information Not Specified
Data Volume	>77 TB of data [1]	Information Not Specified
Major Tissue Types	20 major tissue types [1]	20 organs [2]
Data Sources	Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), Genotype-Tissue Expression (GTEx) consortium [1]	Internal dataset (implied from MGH/BWH), includes 182,862 medical reports [2]
Associated Foundation Model	UNI [1] [3]	TITAN (Transformer-based pathology Image and Text Alignment Network) [2]
Key Innovation	Scale and diversity for self-supervised patch-encoder pretraining [1]	Scale combined with multimodal alignment (images + reports + synthetic captions) for whole-slide representation learning [2]

Detailed Profile of Mass-100K

Composition and Curation

The Mass-100K dataset was explicitly designed to overcome the limitations of previous datasets like TCGA, which primarily contained primary cancer histology slides [1]. Its composition of over 100 million image patches from more than 100,000 diagnostic hematoxylin and eosin (H&E)-stained whole-slide images was curated to provide a rich source of information for learning objective characterizations of histopathologic biomarkers [1]. The dataset's massive scale and diversity across 20 major tissue types were instrumental in training UNI, a general-purpose self-supervised vision encoder based on a Vision Transformer Large (ViT-L) architecture [1] [4].

Experimental Validation and Scaling Laws

The utility of Mass-100K was demonstrated through rigorous experiments establishing scaling laws in computational pathology. Researchers systematically evaluated the impact of data scale by creating subsets of the full dataset: Mass-1K (1 million images, 1,404 WSIs) and Mass-22K (16 million images, 21,444 WSIs) [1]. When used to pretrain the UNI model for a large-scale, hierarchical cancer classification task based on the OncoTree system (covering 108 cancer types), a clear positive correlation between pretraining data volume and downstream task performance was observed [1]. The model pretrained on the full Mass-100K dataset outperformed those trained on the smaller subsets, demonstrating a critical characteristic of a foundation model: improved performance on various tasks when trained on larger datasets [1].

Table 2: Key Experiments Demonstrating Mass-100K's Utility

Experiment Purpose	Experimental Setup	Key Findings
Establishing Scaling Laws	Pretraining UNI on Mass-1K, Mass-22K, and Mass-100K subsets. Evaluation on OncoTree cancer classification (OT-43 and OT-108 tasks) using an Attention-Based Multiple Instance Learning (ABMIL) classifier [1].	Performance increased significantly with data scale. From Mass-22K to Mass-100K, top-1 accuracy increased by +3.7% on OT-43 and +3.0% on OT-108 (P < 0.001) [1].
Benchmarking Against Other Models	Comparing UNI (pretrained on Mass-100K) to other encoders like CTransPath (TCGA, PAIP) and REMEDIS (TCGA) on the same OncoTree classification tasks [1].	UNI outperformed all baseline models by a wide margin, demonstrating the advantage of its large-scale and diverse pretraining dataset [1].

Detailed Profile of Mass-340K

Composition and Multimodal Expansion

The Mass-340K dataset represents a generational leap, not only in the number of WSIs but also in its multimodal nature. It was assembled to train TITAN, a multimodal whole-slide foundation model [2]. Beyond the 335,645 WSIs, the dataset incorporates 182,862 medical reports and 423,122 synthetic fine-grained captions generated using a multimodal generative AI copilot for pathology [2]. This structure enables a three-stage pretraining strategy: 1) vision-only unimodal pretraining, 2) cross-modal alignment with generated morphological descriptions at the region-of-interest (ROI) level, and 3) cross-modal alignment at the WSI level with clinical reports [2].

Advancements in Whole-Slide Representation

A pivotal innovation facilitated by Mass-340K is the shift from patch-level to whole-slide image representation learning. While patch-based models like UNI require an additional aggregation model (e.g., an ABMIL) for slide-level tasks, TITAN is designed to directly produce a general-purpose slide-level representation [2]. The dataset's scale and multimodal annotations were crucial for this advancement. The pretraining process involves dividing WSIs into non-overlapping patches at 20x magnification, extracting features using a powerful patch encoder, and then processing the spatially arranged 2D feature grid with a Vision Transformer to model long-range dependencies across the entire slide [2].

Experimental Workflows and Validation Protocols

Workflow for Patch-Based Foundation Models (e.g., UNI)

The typical experimental workflow for building and validating a patch-based foundation model like UNI using Mass-100K involves a self-supervised learning approach, followed by transfer learning on downstream tasks. The following diagram illustrates this multi-stage process.

Figure 1: Workflow for training and applying a patch-based foundation model like UNI.

Validation via Whole-Slide Image Retrieval

A critical methodology for validating the quality of embeddings learned by foundation models like UNI is zero-shot whole-slide image (WSI) retrieval. This protocol tests the model's ability to find semantically similar cases in a large database without task-specific fine-tuning, directly assessing the generalizability and semantic richness of the features [4]. A standard protocol is outlined below:

• Data: The Cancer Genome Atlas (TCGA) diagnostic slides, comprising 11,444 WSIs from 9,339 patients across 23 organs and 117 cancer subtypes, serve as a standard benchmark [4].
• Search Framework: The Yottixel search engine is often used due to its flexible topology, which allows for the integration of various deep learning models. It uses an unsupervised "mosaic" patching method to create a compact, representative set of patches from each WSI [4].
• Patching: WSIs are segmented into distinct regions via color composition clustering (k-means). A small percentage (e.g., 2%) of representative 224x224 pixel patches are then selected from each region to form the WSI's mosaic [4].
• Embedding and Indexing: Each patch is passed through the foundation model (e.g., UNI) to generate an embedding. These patch embeddings are used to build a search index for the entire database [4].
• Evaluation Metric: Performance is measured using the macro-averaged F1-score for top-1, top-3, and top-5 WSI retrievals. The macro-average ensures balanced evaluation across all cancer subtypes, regardless of class prevalence [4].

Key Validation Result: In a comprehensive benchmark, the UNI model (Yottixel-UNI) achieved a top-5 retrieval F1 score of 42% ± 14%, outperforming the baseline DenseNet model (27% ± 13%) and demonstrating competitive performance with other contemporary foundation models like Virchow and GigaPath [4].

The Researcher's Toolkit

The following table details key computational tools and resources essential for working with and evaluating large-scale pathology datasets and foundation models.

Table 3: Essential Research Reagents and Computational Tools

Tool / Resource	Type	Primary Function	Relevance to Mass-100K/340K
UNI Model Weights	Foundation Model	Provides pretrained patch encoder for feature extraction from histology patches [3].	Direct output of Mass-100K pretraining; used as a feature extractor for downstream tasks [1] [3].
TITAN Model	Multimodal Whole-Slide Foundation Model	Generates general-purpose slide-level representations and enables cross-modal tasks like report generation [2].	Direct output of Mass-340K pretraining; represents the next generation of slide-level models [2].
Yottixel	Search Engine / Framework	Enables efficient whole-slide image search and retrieval using patch-based embeddings [4].	Key framework for the zero-shot evaluation of foundation model embeddings on retrieval tasks [4].
ABMIL (Attention-Based MIL)	Algorithm	Aggregates patch-level features into a slide-level representation for prediction tasks [1].	Standard algorithm used to evaluate patch-based models like UNI on slide-level classification tasks [1].
DINOv2	Self-Supervised Learning Algorithm	Framework for self-supervised pretraining combining knowledge distillation and masked image modeling [1].	The SSL algorithm used to pretrain the UNI model on the Mass-100K dataset [1].
Vision Transformer (ViT)	Model Architecture	Neural network architecture that uses self-attention to process sequences of image patches [1] [2].	Core architecture for both UNI (ViT-L) and TITAN [1] [2].
TCGA (The Cancer Genome Atlas)	Public Dataset	A large public repository of cancer-related WSIs and molecular data [1].	Serves as the primary benchmark dataset for evaluating models pretrained on Mass-100K/340K [1] [4].

The Mass-100K and Mass-340K datasets are cornerstone resources that have fundamentally shaped the landscape of computational pathology. Mass-100K established the critical importance of scale and diversity for training general-purpose patch encoders, while Mass-340K has further advanced the field by enabling multimodal, whole-slide foundation models. The rigorous experimental protocols established for their validation, particularly in challenging zero-shot retrieval settings, provide a robust framework for evaluating future models. As the field progresses, these datasets and the models they spawned serve as both a foundation and a benchmark, guiding ongoing research toward more generalizable, robust, and clinically applicable AI tools in pathology and drug development.

The development of powerful computational pathology foundation models (CPathFMs) is intrinsically linked to the scale, diversity, and quality of the histopathology data used for their training [5]. These models, which learn rich feature representations from unlabeled whole-slide images (WSIs) via self-supervised learning, have demonstrated remarkable potential in automating complex pathology tasks such as diagnosis, prognosis, and biomarker discovery [5]. However, their performance and generalizability are critically dependent on the data they are trained on. The "target population of images" an AI solution may encounter in its intended use is vast, distributed across multiple dimensions of variability including patient demographics, specimen sampling, slide processing, and imaging protocols [6]. To create models that are robust to this biological and technical heterogeneity, training datasets must be correspondingly diverse and representative. This technical guide delves into the core aspects of data compilation for CPathFMs, with a specific focus on the Mass-340K dataset, analyzing its composition, sourcing, and the methodologies it enables.

The Mass-340K dataset represents a significant scaling of its predecessor, Mass-100K, and stands as a cornerstone for training large-scale pathology foundation models. The following table summarizes the core quantitative attributes of the Mass-340K dataset as used in the development of the TITAN (Transformer-based pathology Image and Text Alignment Network) model [2].

Table 1: Composition of the Mass-340K Dataset

Attribute	Description	Scale/Value
Total WSIs	Number of whole-slide images	335,645
Medical Reports	Accompanying pathology reports	182,862
Synthetic Captions	Fine-grained ROI captions generated via AI copilot (PathChat)	423,122
Organ Diversity	Number of different organ types represented	20
Stain Types	Includes H&E and other staining protocols	Multiple
Scanner Types	Various scanner models used for digitization	Multiple

The Mass-340K dataset was designed with diversity as a key principle, distributed across 20 organ types, different stains, diverse tissue types, and scanned with various scanner types [2]. This diversity has proven to be a critical factor in developing patch encoders that generalize well, a principle that was successfully translated to the slide level with TITAN. The dataset is used for multi-stage pretraining, involving vision-only self-supervised learning on region-of-interest (ROI) crops, followed by cross-modal alignment using both synthetic captions and original pathology reports [2].

Data Sourcing and Institutional Partnerships

Large-scale pathology datasets are often compiled through collaborations with multiple medical and research institutions. These partnerships are essential for accessing a wide variety of cases that reflect real-world clinical practice.

The Mass-340K dataset is an internal dataset, and while its specific institutional sources are not exhaustively detailed in the provided context, major academic medical centers like Massachusetts General Hospital (MGH) and Brigham and Women's Hospital (BWH) are consistently featured as key contributors in the computational pathology research ecosystem [5]. Furthermore, public data sources play an indispensable role in benchmarking and model development.

Table 2: Key Data Sources in Computational Pathology

Data Source	Type	Role and Relevance
MGH, BWH	Academic Medical Centers	Often sources of large, diverse, real-world clinical pathology data for model training and validation [5].
GTEx (Genotype-Tissue Expression)	Public Research Program	Provides a rich resource of normal, non-diseased tissue samples, crucial for understanding baseline biology and changes in disease [7].
TCGA (The Cancer Genome Atlas)	Public Database	A foundational source for cancer genomics and associated histopathology images across multiple cancer types [5].
Camelyon Series	Public Benchmark Dataset	Widely used for evaluating metastasis detection in breast cancer; recently refined into the "Camelyon+" dataset with cleaned labels and expanded annotations [8].
HuBMAP (Human BioMolecular Atlas Program)	Public Research Consortium	Aims to construct a 3D reference atlas of the healthy human body, providing multiscale data from organs down to cells and biomarkers [7].

Initiatives like HuBMAP involve experts from over 20 consortia and are critical for establishing a Common Coordinate Framework (CCF) that helps harmonize multimodal data, including 3D organ models, histology images, and single-cell omics data [7]. Mapping new experimental data into such a reference atlas enables powerful comparisons between healthy and diseased tissue.

Experimental Protocols and Workflow Methodologies

The utility of a large-scale dataset like Mass-340K is realized through sophisticated experimental protocols. The pretraining of the TITAN model exemplifies a modern, multi-stage methodology for building a multimodal whole-slide foundation model.

TITAN's Multi-Stage Pretraining Workflow

The pretraining strategy for TITAN consists of three distinct stages to ensure that the resulting slide-level representations capture histomorphological semantics at both the region and whole-slide levels [2].

• Stage 1: Vision-Only Unimodal Pretraining. This stage uses the 335,645 WSIs from Mass-340K for visual self-supervised learning. The core technique adapts the iBOT framework, which combines masked image modeling and knowledge distillation, to the slide level. The input to the model is not raw pixels but a 2D grid of pre-extracted patch features (768-dimensional features from CONCHv1.5). The model is trained by creating multiple views of a WSI through random cropping of this feature grid and applying augmentations like flipping and posterization [2].
• Stage 2: Cross-Modal Alignment at ROI-Level. In this stage, the vision model is aligned with language. The training uses 423,122 pairs of high-resolution ROIs (8,192 x 8,192 pixels) and corresponding synthetic captions generated by PathChat, a multimodal generative AI copilot for pathology. This step teaches the model to associate fine-grained visual patterns with descriptive text [2].
• Stage 3: Cross-Modal Alignment at WSI-Level. The final stage aligns entire WSIs with their corresponding pathology reports. This training uses 182,862 WSI-report pairs from Mass-340K, enabling the model to understand slide-level clinical context and summaries [2].

The following diagram illustrates this integrated workflow, from data input to final model capabilities.

Handling Gigapixel WSIs and Long-Range Context

A significant technical challenge in slide-level modeling is handling the gigapixel size of WSIs. TITAN addresses this by:

• Feature Grid Construction: Dividing each WSI into non-overlapping 512x512 pixel patches at 20x magnification and extracting 768-dimensional features for each patch using a pre-trained patch encoder (CONCHv1.5). These features are spatially arranged in a 2D grid [2].
• Context Modeling with Transformers: Using a Vision Transformer (ViT) to process the feature grid. To handle long and variable input sequences, TITAN uses Attention with Linear Biases (ALiBi), extended to 2D. This allows the model to extrapolate to longer contexts during inference than seen in training, based on the relative Euclidean distance between features in the grid [2].

The Scientist's Toolkit: Essential Research Reagents and Materials

To replicate or build upon research involving datasets like Mass-340K, scientists rely on a suite of computational tools, models, and benchmark datasets. The table below catalogues key resources referenced in the context of modern computational pathology research.

Table 3: Key Research Reagents and Solutions for Computational Pathology

Resource Name	Type	Function and Description
CONCH / CONCHv1.5	Patch Encoder Model	A foundational model trained via contrastive learning on image-caption pairs. Used to extract foundational feature representations from histology image patches [2] [8].
TITAN	Whole-Slide Foundation Model	A Transformer-based multimodal model that produces general-purpose slide representations from a grid of patch features, enabling tasks like classification, retrieval, and report generation [2] [8].
DINOv2 / iBOT	Self-Supervised Learning Algorithm	A self-supervised training framework that uses knowledge distillation and masked image modeling to learn powerful visual representations without labeled data [2] [5].
Camelyon+	Benchmark Dataset	A cleaned and re-annotated version of the Camelyon-16 and -17 datasets for breast cancer metastasis detection, providing reliable labels for model evaluation [8].
Protege Evaluation Datasets	Evaluation Benchmark	A set of multi-modal datasets (e.g., combining EMR, pathology slides, imaging) specifically curated for unbiased evaluation of healthcare AI models, independent of training data [9].
HuBMAP CCF (Common Coordinate Framework)	Spatial Reference Framework	A 3D open-source atlas that enables registration and integration of multimodal tissue data (histology, omics) within a standardized spatial context of the human body [7].
PLUTO	Pathology Foundation Model	PathAI's foundation model, used to extract biologically-relevant features from WSIs for downstream tasks like toxicology assessment [10].

The Mass-340K dataset exemplifies the critical trend towards large-scale, diverse, and multimodal data collection in computational pathology. Its composition—spanning hundreds of thousands of WSIs from multiple organs, stains, and scanners, and augmented with both real and synthetic textual descriptions—provides the essential fuel for training transformative foundation models like TITAN. The experimental protocols that leverage this data, including multi-stage pretraining and sophisticated context modeling, are as important as the data itself. For researchers and drug development professionals, understanding the provenance, structure, and application of these data resources is paramount. The future of robust, clinically applicable AI in pathology hinges on continued efforts to compile representative datasets, develop standardized benchmarks like Camelyon+ and Protege's offerings, and build upon the foundational tools and methodologies that this deep dive has outlined.

Addressing the Limitations of Previous Datasets like TCGA for Foundation Model Pretraining

The development of powerful foundation models in computational pathology has been historically constrained by the limited scale and diversity of available training data. Prior to the creation of recent large-scale datasets, models were primarily trained on resources like The Cancer Genome Atlas (TCGA), which contains approximately 29,000 whole-slide images (WSIs) spanning 32 cancer types [1]. While valuable, TCGA and similar collections present significant limitations for foundation model pretraining, including restricted sample sizes that inhibit the scaling laws crucial for robust feature learning, a predominant focus on primary cancer histology that limits morphological diversity, and insufficient representation of rare diseases and varied tissue types [1]. These constraints have fundamentally limited the generalizability and clinical applicability of pathology AI models across real-world diagnostic scenarios. To overcome these challenges, researchers have pioneered the creation of massively scaled, diversified histology datasets specifically designed for foundation model pretraining, notably Mass-100K and its expanded successor Mass-340K, which have enabled unprecedented advances in self-supervised learning for computational pathology.

Dataset Architectures: Mass-100K and Mass-340K

Core Specifications and Composition

The Mass-100K and Mass-340K datasets represent foundational resources specifically engineered to overcome the scaling limitations of previous pathology data collections. The table below summarizes their core architectural specifications:

Table 1: Core Specifications of Mass-100K and Mass-340K Datasets

Specification	Mass-100K Dataset	Mass-340K Dataset
Total Whole-Slide Images (WSIs)	100,426+ diagnostic H&E-stained WSIs [1]	335,645 WSIs [2]
Tissue Patches/ROIs	>100 million images [1] [11]	Not explicitly quantified (builds upon Mass-100K)
Organ/Tissue Types	20 major tissue types [1]	20 organ types [2]
Data Sources	Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), Genotype-Tissue Expression (GTEx) consortium [1]	Expanded institutional collection (assumed similar sources as Mass-100K)
Primary Application	Pretraining of UNI foundation model [1] [11]	Pretraining of TITAN multimodal foundation model [2]
Multimodal Pairing	Not specified	182,862 medical reports [2]

Methodological Advancements Over Previous Datasets

These datasets incorporate several methodological innovations that directly address TCGA's limitations. Mass-340K specifically enables multimodal vision-language pretraining by incorporating paired pathology reports and synthetic captions, facilitating cross-modal learning between histology images and clinical text [2]. The datasets employ diversified sampling strategies across multiple organ systems and tissue types, contrasting with TCGA's cancer-dominated profile [1]. They also establish scaling laws for computational pathology, demonstrating that increasing pretraining data size consistently improves downstream performance on complex diagnostic tasks [1]. Furthermore, they support rare disease representation through inclusion of diverse cancer subtypes and morphological patterns essential for robust generalizability [11].

Experimental Frameworks and Pretraining Methodologies

Foundation Model Pretraining workflows

The Mass-100K and Mass-340K datasets have enabled the development of sophisticated pretraining methodologies that leverage self-supervised learning (SSL) at unprecedented scales. The following diagram illustrates the core pretraining workflow for models trained on these datasets:

Technical Implementation Details

The pretraining of foundation models on these datasets involves several technically sophisticated components. For visual feature extraction, WSIs are divided into non-overlapping patches of 512×512 pixels at 20× magnification, with 768-dimensional features extracted for each patch using specialized encoders like CONCH [2]. The Transformer architecture employs attention with linear bias (ALiBi) to handle long sequences of patch features while preserving spatial relationships across gigapixel WSIs [2]. For multimodal alignment, contrastive learning objectives align image features with corresponding pathology reports and synthetically generated fine-grained morphological descriptions [2]. The self-supervised objectives utilize masked image modeling and knowledge distillation (iBOT framework) to learn morphological representations without manual annotations [2].

Performance Benchmarking and Validation

Experimental Protocols and Evaluation Metrics

Rigorous benchmarking against existing pathology foundation models demonstrates the performance advantages enabled by Mass-100K and Mass-340K. The evaluation framework encompasses multiple clinically relevant domains:

Table 2: Performance Benchmarking Across Clinical Tasks

Evaluation Domain	Specific Tasks	Superior Performing Models	Key Performance Metrics
Cancer Subtyping	43-class and 108-class OncoTree classification [1]	UNI (trained on Mass-100K) [1]	Top-1 accuracy: +7.2% over baselines [1]
Rare Disease Retrieval	Cross-modal retrieval and zero-shot classification [2]	TITAN (trained on Mass-340K) [2]	Outperforms existing slide foundation models [2]
Multi-task Benchmarking	41 tasks across TCGA, CPTAC, and external datasets [12]	Virchow2 ranks first (0.706 mean performance) [12]	Balanced accuracy, precision, recall, F1 score [12]
Biomarker Prediction	Molecular alteration prediction from histology [1]	UNI and other Mass-100K trained models [1]	AUROC, F1 scores across multiple cancer types [1]

Scaling Law Validation

Experimental validation on the Mass-100K dataset demonstrates clear scaling laws in computational pathology. When evaluating the UNI model on the 108-class OncoTree classification task, performance increased by +3.5% in top-1 accuracy when scaling from Mass-1K (1,404 WSIs) to Mass-22K (21,444 WSIs), with further gains of +3.0% when scaling to the full Mass-100K dataset (100,426 WSIs) [1]. This scaling relationship demonstrates that increased pretraining data volume directly enhances model capability on complex, clinically relevant classification tasks, validating the core hypothesis behind creating these large-scale datasets.

Essential Research Infrastructure

The development and application of foundation models pretrained on Mass-100K/Mass-340K requires specialized computational resources and methodological components:

Table 3: Essential Research Reagents for Pathology Foundation Model Development

Resource Category	Specific Tools/Components	Function/Purpose
Foundation Models	UNI, TITAN, CONCH [2] [11]	Pretrained encoders providing transferable feature representations for diverse downstream tasks
SSL Algorithms	DINOv2, iBOT, masked autoencoders [2] [1]	Self-supervised learning frameworks for unsupervised representation learning from unlabeled images
Model Architectures	Vision Transformers (ViT-Large, ViT-Huge) [2] [1]	Neural network backbones capable of processing sequences of patch embeddings from WSIs
Multimodal Alignment	Contrastive language-image pretraining [2]	Learning joint embeddings between histology images and textual reports/captions
Benchmarking Frameworks	PathoROB, clinical task collections [12] [13]	Standardized evaluation pipelines to assess model robustness and clinical utility

The creation of Mass-100K and Mass-340K datasets represents a paradigm shift in computational pathology, directly addressing the scaling limitations of previous resources like TCGA. By providing orders of magnitude more diverse histology images across multiple tissue types and pairing them with clinical reports, these datasets have enabled the development of foundation models with significantly enhanced capabilities for cancer subtyping, rare disease identification, and multimodal reasoning. The experimental protocols and scaling laws established through their use provide a roadmap for future dataset development in medical AI. As the field progresses, increasing focus on multi-institutional data collection to reduce site-specific bias [13], incorporation of additional multimodal data sources such as genomics and proteomics [14], and development of more efficient pretraining methodologies [15] will further advance the clinical applicability of pathology foundation models. These resources collectively establish a new foundation for data-driven discovery in diagnostic pathology and precision medicine.

The field of computational pathology is undergoing a fundamental transformation, moving from specialized task-specific models toward general-purpose foundation models capable of addressing diverse clinical challenges. This paradigm shift is largely driven by the creation of massive histopathology datasets and advances in self-supervised learning techniques. Central to this transition are the Mass-100K and Mass-340K datasets—comprehensive collections of whole-slide images that have enabled the development of foundational models like UNI and TITAN. These models demonstrate unprecedented capabilities across a wide spectrum of pathology tasks, from cancer subtyping and rare disease identification to prognostic prediction and report generation. This technical review examines the architectural innovations, training methodologies, and evaluation frameworks underpinning this transformative shift, with particular focus on how large-scale datasets are redefining the boundaries of computational pathology.

Computational pathology (CPath) has traditionally relied on task-specific models trained for specialized applications such as tumor detection, cancer grading, or biomarker prediction. These conventional approaches typically utilized supervised learning on limited annotated datasets, constraining their generalizability and requiring extensive labeling efforts for each new clinical task. The emergence of foundation models represents a pivotal shift toward unified architectures pretrained on massive unlabeled datasets that can be adapted to numerous downstream tasks with minimal fine-tuning.

The limitations of task-specific models become particularly apparent when facing real-world diagnostic challenges. Pathologists routinely navigate thousands of possible diagnoses across diverse tissue types and disease categories, requiring models with broad rather than narrow expertise [1]. Early transfer learning approaches using models pretrained on natural images (e.g., ImageNet) struggled with the unique characteristics of histopathology data, including minimal color variation, rotation-agnosticism, and hierarchical tissue organization [16]. This gap prompted the development of pathology-specific foundation models trained on extensive histopathology datasets.

Two landmark datasets have catalyzed this paradigm shift: Mass-100K and Mass-340K. These datasets provide the scale and diversity necessary for training general-purpose models that capture the complex morphological patterns present in human tissues across health and disease states. The Mass-100K dataset comprises over 100,000 diagnostic H&E-stained whole-slide images (WSIs) from 20 major tissue types, while the expanded Mass-340K dataset contains 335,645 WSIs with corresponding pathology reports and synthetic captions [2] [1]. The creation of these datasets has enabled the development of foundation models that demonstrate remarkable versatility across diverse machine learning settings, including zero-shot learning, few-shot adaptation, and multimodal reasoning.

The Foundation Dataset Ecosystem: Mass-100K and Mass-340K

Dataset Composition and Scale

The Mass-100K and Mass-340K datasets represent unprecedented collections of histopathology data that have enabled the training of general-purpose foundation models. The table below summarizes the key characteristics of these datasets:

Table 1: Composition of Mass-100K and Mass-340K Datasets

Characteristic	Mass-100K Dataset	Mass-340K Dataset
Total WSIs	100,426+	335,645
Tissue patches	>100 million	Not specified
Organ types	20	20
Data volume	>77 TB	Not specified
Additional data	-	182,862 medical reports + 423,122 synthetic captions
Sources	MGH, BWH, GTEx consortium	Not specified
Stain types	H&E	Multiple stains
Scanner types	Various	Various

The Mass-100K dataset was specifically designed to address the limitations of previous datasets like The Cancer Genome Atlas (TCGA), which primarily contained oncology-focused slides from a limited number of cancer types [1]. By incorporating diverse tissue types from both cancerous and non-cancerous sources, including the Genotype-Tissue Expression (GTEx) consortium, Mass-100K provides a more comprehensive representation of histopathological morphology [1]. This diversity has proven essential for developing models that generalize across various clinical scenarios and tissue types.

The Mass-340K dataset extends this concept further by incorporating not only additional WSIs but also multimodal data in the form of pathology reports and synthetically generated captions [2]. The inclusion of 423,122 synthetic captions generated using PathChat (a multimodal generative AI copilot for pathology) provides fine-grained morphological descriptions at the region-of-interest level, enabling more sophisticated vision-language pretraining [2]. This combination of visual and textual data creates a rich training environment for models learning to associate histological patterns with clinical descriptions.

Data Diversity and Clinical Representativeness

Both datasets explicitly address the critical need for diversity in foundation model pretraining. The 20 organ types encompass major tissue systems, ensuring broad coverage of human anatomy. Additionally, the inclusion of various stain types (beyond standard H&E) and scanner manufacturers enhances model robustness to technical variations commonly encountered in clinical practice [2]. This diversity is particularly valuable for rare diseases and conditions where limited data would otherwise constrain model development.

The scale of these datasets aligns with emerging principles of foundation model development, where increased data volume and diversity consistently lead to improved downstream performance [1]. In ablation studies, researchers observed performance improvements of +3.5% to +4.2% in top-1 accuracy when scaling from smaller datasets (Mass-1K) to the full Mass-100K collection for cancer classification tasks [1]. Similar scaling benefits likely extend to the even larger Mass-340K dataset, though comprehensive ablation studies have not been reported for this expanded collection.

Architectural Foundations: From Patch-Level to Slide-Level Modeling

The Multiple Instance Learning Framework

Whole-slide images in computational pathology present unique computational challenges due to their gigapixel resolution (often exceeding 100,000 × 100,000 pixels). The standard approach for handling these massive images employs a multiple instance learning framework, where WSIs are treated as "bags" of smaller patches (instances) [16]. Formally, this relationship can be expressed as:

Table 2: Multiple Instance Learning Formulation

Component	Mathematical Representation	Description
WSI patches	( \boldsymbol{X}={\boldsymbol{x}i}{i=1}^N \in \mathbb{R}^{N \times h \times w \times 3} )	N non-overlapping patches from tessellated WSI
Feature extraction	( \boldsymbol{z}i = \mathcal{M}e(\boldsymbol{x}_i) )	Extractor ( \mathcal{M}_e ) generates patch features
Feature aggregation	( \boldsymbol{h} = \mathcal{M}_g(\boldsymbol{Z}) )	Aggregator ( \mathcal{M}_g ) produces slide-level features
Bag label assignment	( Y = \begin{cases} 1 & \exists i, yi = 1 \ 0 & \forall i, yi = 0 \end{cases} )	Slide-level label determined by patch labels

In conventional MIL pipelines, feature extraction typically relies on models pretrained on natural images (e.g., ImageNet-pretrained ResNet-50). However, these models struggle with pathology-specific characteristics, prompting the development of specialized pathology foundation models that serve as more effective feature extractors [16].

Model Architectures and Design Innovations

Foundation models in pathology have embraced transformer-based architectures, which have demonstrated remarkable success in both natural language processing and computer vision. The table below compares key architectural characteristics of prominent pathology foundation models:

Table 3: Architecture Comparison of Pathology Foundation Models

Model	Architecture	Parameters	Base Method	Input Modality	Scale
UNI	ViT-Large	Not specified	DINOv2	Histology patches	Large
CONCH	ViT-B/16	86.3M	iBOT/CoCa	Whole-slide, Text	Base
TITAN	ViT with ALiBi	Not specified	iBOT distillation	Whole-slide, Text	Large
CTransPath	Swin-T/14	28.3M	MoCov3	Histology patches	Small
PLIP	ViT-B/32	87M	CLIP	Pathology, Text	Base
Phikon	ViT-S/B/L/16	21.7M/85.8M/307M	iBOT	Histology patches	Small/Base/Large

UNI utilizes a vision transformer (ViT-Large) architecture pretrained using DINOv2 self-supervised learning on the Mass-100K dataset [1] [17]. This approach enables the model to learn powerful, transferable representations without requiring labeled data during pretraining. UNI's design focuses on creating a general-purpose visual encoder that can be applied to various tasks, from region-of-interest classification to whole-slide analysis.

TITAN (Transformer-based pathology Image and Text Alignment Network) introduces several architectural innovations to address the challenges of whole-slide modeling [2]. The model employs a vision transformer that operates on pre-extracted patch features rather than raw pixels, effectively using patch encoders as "patch embedding layers" in a conventional ViT. To handle variable-length WSI sequences, TITAN incorporates Attention with Linear Biases (ALiBi), originally developed for long-context inference in large language models, extended to 2D for preserving spatial relationships in tissue sections [2].

CONCH represents a multimodal approach that aligns visual and textual representations through contrastive learning [17] [11]. Trained on over 1.17 million histopathology image-text pairs, CONCH demonstrates strong performance on tasks including rare disease identification, tumor segmentation, and cross-modal retrieval. The model's architecture enables natural language interaction, allowing pathologists to search for morphologies of interest using descriptive text [11].

Training Methodologies: Self-Supervised and Multimodal Learning

Self-Supervised Learning Paradigms

Foundation models in pathology predominantly utilize self-supervised learning (SSL) to leverage large-scale unlabeled datasets. SSL generates supervisory signals automatically through pretext tasks, allowing models to learn meaningful representations without manual annotation [16]. Given an input image ( \boldsymbol{x} ), a transformation function ( \mathcal{T}(\cdot) ) generates a modified version ( \tilde{\boldsymbol{x}} = \mathcal{T}(\boldsymbol{x}) ) with corresponding pseudo-label ( \tilde{y} ). The model ( \mathcal{M}e(\cdot) ) then extracts features and predicts ( \hat{y} = \mathcal{M}e(\tilde{\boldsymbol{x}}) ), with the learning objective minimizing the difference between ( \hat{y} ) and ( \tilde{y} ).

Different foundation models employ distinct SSL approaches:

• UNI utilizes DINOv2, a self-distillation method that learns robust representations by matching feature distributions between different augmented views of the same image [1] [17]. This approach has demonstrated remarkable transferability to downstream tasks without task-specific fine-tuning.

• TITAN employs iBOT framework, which combines masked image modeling with online tokenizer distillation [2]. This approach allows the model to learn both local and global visual contexts by reconstructing masked portions of the input while maintaining consistency between teacher and student networks.

• CONCH adapts the CLIP (Contrastive Language-Image Pre-training) framework to pathology, aligning visual and textual representations through contrastive learning [17] [11]. This enables cross-modal retrieval and zero-shot classification capabilities.

Multimodal Pretraining Strategies

TITAN introduces a sophisticated three-stage pretraining approach that progressively builds capabilities from visual to multimodal understanding:

Stage 1: Vision-Only Unimodal Pretraining TITAN first undergoes self-supervised pretraining on region-of-interest (ROI) crops using the iBOT framework [2]. The model learns to encode histomorphological patterns by processing 8,192 × 8,192 pixel regions at 20× magnification, with data augmentation including random cropping, flipping, and posterization feature augmentation [2].

Stage 2: Cross-Modal Alignment with Synthetic Captions The vision encoder is aligned with textual descriptions using 423,122 synthetically generated ROI captions created through PathChat [2]. This stage enables fine-grained understanding of morphological patterns and their semantic descriptions.

Stage 3: Cross-Modal Alignment with Pathology Reports Finally, the model learns slide-level vision-language correspondence using 182,862 pairs of WSIs and clinical reports [2]. This stage bridges whole-slide visual patterns with diagnostic terminology and clinical observations.

The Scientist's Toolkit: Essential Research Reagents

Table 4: Essential Research Reagents for Pathology Foundation Model Development

Resource	Type	Function	Representative Examples
Pretraining Algorithms	Software	Self-supervised learning methods	DINOv2, iBOT, MoCoV3, CLIP
Model Architectures	Software	Neural network backbones	Vision Transformer (ViT), Swin Transformer
Whole-Slide Processing	Software	WSI handling and patch extraction	HistomicsML, CLAM, HIPT
Evaluation Frameworks	Software	Benchmarking and assessment	Multiple Instance Learning (MIL), Linear Probing
Public Datasets	Data	Pretraining and evaluation	TCGA, GTEx, CAMELYON16
Computational Resources	Hardware	Model training and inference	High-memory GPUs, Distributed training systems

Experimental Evaluation and Performance Benchmarking

Comprehensive Evaluation Frameworks

Foundation models in pathology undergo rigorous evaluation across diverse tasks to assess their generalizability and clinical utility. The experimental protocols typically encompass multiple machine learning settings:

• Linear Probing: Frozen features are used to train linear classifiers for specific tasks, testing feature quality without fine-tuning [2] [1]
• Few-Shot Learning: Models are adapted with very limited labeled examples (e.g., 1-10 samples per class) [1]
• Zero-Shot Evaluation: Models perform tasks without any task-specific training, particularly for multimodal models [2]
• Weakly Supervised Learning: Slide-level labels are used without patch-level annotations via multiple instance learning [1] [16]

UNI was evaluated on 34 distinct clinical tasks spanning various difficulty levels and clinical scenarios [1]. These included nuclear segmentation, primary and metastatic cancer detection, cancer grading and subtyping, biomarker screening, molecular subtyping, organ transplant assessment, and large-scale pan-cancer classification with up to 108 cancer types in the OncoTree system [1].

TITAN was assessed across diverse clinical tasks including cancer subtyping, biomarker prediction, outcome prognosis, rare cancer retrieval, cross-modal retrieval, and pathology report generation [2]. The model's performance was measured in both resource-rich and resource-limited scenarios to test its robustness in practical clinical settings.

Performance Comparison and Scaling Laws

Experimental results demonstrate the superior performance of foundation models compared to previous approaches. The table below summarizes key performance comparisons:

Table 5: Performance Comparison of Pathology Foundation Models

Model	Evaluation Tasks	Key Results	Comparative Advantage
UNI	34 tasks including OT-43 and OT-108 cancer classification	Outperformed CTransPath and REMEDIS by wide margin; +3.5-4.2% improvement with data scaling	Demonstrates scaling laws; effective in few-shot settings
TITAN	Cancer prognosis, rare disease retrieval, report generation	Outperforms ROI and slide foundation models in zero-shot and few-shot settings	Strong multimodal capabilities; effective cross-modal retrieval
CONCH	14 tasks including rare disease identification, segmentation	State-of-the-art in zero-shot learning and cross-modal retrieval	Excellent vision-language alignment

UNI demonstrates clear scaling laws, with performance improvements of +3.5% to +4.2% when increasing pretraining data from Mass-1K to Mass-100K [1]. This scaling behavior aligns with observations in natural image foundation models and underscores the importance of dataset size in developing capable pathology models.

TITAN shows particular strength in low-data regimes, outperforming both region-of-interest and slide-level foundation models across machine learning settings including linear probing, few-shot, and zero-shot classification [2]. The model also demonstrates impressive capabilities in rare cancer retrieval, successfully identifying matching cases even for uncommon cancer types with limited training examples.

The paradigm shift from task-specific models to general-purpose foundation models represents a transformative development in computational pathology. The creation of massive datasets like Mass-100K and Mass-340K has enabled the training of models with unprecedented versatility and clinical applicability. These foundation models, including UNI, TITAN, and CONCH, demonstrate strong performance across diverse tasks while reducing the need for extensive labeled data through zero-shot and few-shot learning capabilities.

Looking forward, several research directions promise to further advance the field. Federated learning approaches may enable training on even larger datasets while preserving patient privacy [16]. Multimodal integration beyond vision and text—including genomic, proteomic, and clinical data—could create more comprehensive patient representations [2]. Efficient adaptation methods like prompt tuning and adapter layers may make foundation models more accessible for clinical deployment [16]. Finally, rigorous clinical validation through prospective trials remains essential to translate these technical advances into improved patient care.

The emergence of pathology foundation models marks a significant milestone in the integration of artificial intelligence into diagnostic medicine. By capturing the complex morphological patterns present in human tissues across health and disease states, these models have the potential to augment pathological diagnosis, enhance diagnostic accuracy, and ultimately improve patient outcomes across a broad spectrum of medical conditions.

From Data to Diagnosis: Methodologies and Real-World Applications of Models Trained on Mass-Scale Datasets

The development of powerful foundation models in computational pathology has been constrained by the limited scale and diversity of available histopathology data. To address this challenge, researchers have introduced large-scale datasets such as Mass-100K and Mass-340K, which serve as critical resources for pretraining general-purpose models. These datasets enable the application of advanced self-supervised learning (SSL) methodologies like DINOv2 and vision-language alignment, moving beyond the limitations of previous approaches that relied predominantly on public datasets like The Cancer Genome Atlas (TCGA) [1] [18].

Mass-100K represents a pivotal scaling effort in histopathology pretraining, comprising over 100 million images from more than 100,000 diagnostic H&E-stained whole slide images (WSIs) across 20 major tissue types [1]. This dataset forms the foundation for UNI, a general-purpose self-supervised model that demonstrates remarkable transfer learning capabilities across diverse clinical tasks. Building upon this effort, Mass-340K expands significantly in scale with 335,645 WSIs, enabling the development of TITAN (Transformer-based pathology Image and Text Alignment Network) - a multimodal whole-slide foundation model that incorporates both visual self-supervised learning and vision-language alignment with corresponding pathology reports and synthetic captions [2]. These datasets provide the extensive and diverse pretraining data necessary for developing pathology foundation models that can generalize across a wide spectrum of diagnostic scenarios, including rare diseases and complex clinical conditions.

Table 1: Core Dataset Specifications for Pathology Foundation Model Pretraining

Dataset	Whole Slide Images (WSIs)	Image Patches/ROIs	Tissue Types	Key Characteristics	Primary Models
Mass-100K	100,402+ H&E WSIs [18]	100,130,900 images (75.8M @256×256, 24.3M @512×512) [18]	20 major tissue types [1]	Sourced from MGH, BWH, and GTEx; excludes public benchmarks to prevent data contamination [18]	UNI [1]
Mass-340K	335,645 WSIs [2]	Not explicitly stated	20 organ types [2]	Includes 182,862 medical reports and 423,122 synthetic captions; diverse stains and scanner types [2]	TITAN, TITANV [2]

Core Technical Methodologies

Self-Supervised Learning with DINOv2 Framework

The DINOv2 (self-DIstillation with NO labels) framework represents a breakthrough in self-supervised learning for computer vision, enabling the pretraining of models without extensive labeled datasets [19] [20]. This approach is particularly valuable in computational pathology, where expert annotations are scarce and costly to obtain. DINOv2 employs a knowledge distillation technique where a larger "teacher" model trains a smaller "student" model to mimic its output, effectively transferring knowledge without manual labels [19].

The technical implementation of DINOv2 incorporates several key components that contribute to its effectiveness. The framework utilizes an image-level objective through self-distillation with multi-crop strategies, where different augmented views of the same image are processed by both teacher and student networks [19] [18]. Additionally, it employs a patch-level objective through masked image modeling, randomly masking portions of the input patches during training [19]. The approach also includes KoLeo regularization on [CLS] tokens to prevent dimensional collapse and encourage uniform distribution of features in the embedding space [18]. For model scaling, DINOv2 uses a functional distillation pipeline that compresses large models into smaller variants with minimal performance loss, enabling efficient inference [19].

In the context of pathology foundation models, UNI adapts the DINOv2 framework specifically for histopathology data by training on the Mass-100K dataset. The implementation utilizes a Vision Transformer Large (ViT-L/16) architecture with patch size of 16, embedding dimension of 1024, 16 attention heads, and MLP feed-forward networks, totaling approximately 300 million parameters [18]. The training regimen employs fp16 mixed precision using PyTorch-FSDP for 125,000 iterations with a substantial batch size of 3072, requiring approximately 1024 GPU hours on Nvidia A100 hardware [18].

Diagram 1: DINOv2 Training Workflow for Pathology

Vision-Language Alignment in Histopathology

Vision-language alignment represents a sophisticated multimodal learning approach that connects histopathological visual patterns with clinical and morphological descriptions. This methodology addresses a significant limitation in vision-only models by incorporating rich supervisory signals found in pathology reports, enabling capabilities such as zero-shot visual-language understanding and cross-modal retrieval [2].

The TITAN model implements vision-language alignment through a structured three-stage pretraining strategy. Stage 1 involves vision-only unimodal pretraining on Mass-340K using region-of-interest (ROI) crops, building foundational visual representations [2]. Stage 2 performs cross-modal alignment of generated morphological descriptions at the ROI-level, utilizing 423,122 pairs of high-resolution ROIs (8,192×8,192 pixels) and synthetic captions generated from PathChat, a multimodal generative AI copilot for pathology [2]. Stage 3 conducts cross-modal alignment at the whole-slide level with 182,862 pairs of WSIs and clinical reports, enabling slide-level multimodal understanding [2].

This multimodal approach requires specialized architectures to handle the unique challenges of gigapixel WSIs. TITAN employs a Vision Transformer architecture that processes sequences of patch features encoded by powerful histology patch encoders rather than raw pixels [2]. To manage computational complexity from long input sequences, the model uses attention with linear bias (ALiBi) for long-context extrapolation, where the linear bias is based on the relative Euclidean distance between features in the feature grid [2]. The model creates multiple views of a WSI by randomly cropping 2D feature grids and sampling both global (14×14) and local (6×6) crops for iBOT pretraining, with additional feature augmentation through vertical/horizontal flipping and posterization [2].

Diagram 2: Vision-Language Alignment Architecture

Experimental Protocols and Evaluation Metrics

Benchmarking Frameworks and Tasks

The evaluation of pathology foundation models pretrained on Mass-100K and Mass-340K datasets involves comprehensive benchmarking across diverse clinical tasks to assess their generalization capabilities. For UNI, researchers conducted extensive evaluations across 34 representative computational pathology tasks of varying diagnostic difficulty [1]. These tasks include ROI-level classification for basic tissue characterization, nuclear segmentation for cellular-level analysis, primary and metastatic cancer detection for diagnostic applications, cancer grading and subtyping for prognostic assessment, biomarker screening and molecular subtyping for predictive purposes, and organ transplant assessment for specialized clinical scenarios [1].

A particularly rigorous evaluation involves large-scale, hierarchical cancer classification based on the OncoTree cancer classification system. This benchmark includes two tasks that vary in diagnostic difficulty: OT-43 (43-class OncoTree cancer type classification) and OT-108 (108-class OncoTree code classification) [1]. Notably, 90 out of the 108 cancer types are designated as rare cancers, providing a challenging test for model generalization on underrepresented conditions [1].

For TITAN, evaluation encompasses diverse clinical tasks including linear probing for transfer learning assessment, few-shot and zero-shot classification for data-efficient learning scenarios, rare cancer retrieval for specialized diagnostic applications, cross-modal retrieval for vision-language integration, and pathology report generation for generative capabilities [2].

Table 2: Performance Evaluation of Pathology Foundation Models on Key Benchmarks

Model	Pretraining Data	OncoTree-43 (Top-1 Accuracy)	OncoTree-108 (Top-1 Accuracy)	Zero-Shot Classification	Cross-Modal Retrieval
UNI	Mass-100K (100K+ WSIs) [1]	Significant improvements over previous SOTA (exact metrics not specified in sources) [1]	+3.5-4.2% performance increase with data scaling [1]	Not primary focus	Not primary focus
TITAN	Mass-340K (335K+ WSIs) [2]	Outperforms both ROI and slide foundation models [2]	Superior performance in rare cancer retrieval [2]	Enabled via vision-language alignment [2]	Enabled via shared embedding space [2]
CTransPath	TCGA + PAIP [21]	Lower performance compared to UNI [1]	Lower performance compared to UNI [1]	Not supported	Not supported

Adaptation Strategies and Data Efficiency

A critical aspect of foundation model evaluation involves assessing their adaptability to various downstream tasks under different data constraints. Recent benchmarking studies have examined four pathology-specific foundation models (CTransPath, Lunit, Phikon, and UNI) across 14 datasets through two primary scenarios: consistency assessment and flexibility assessment [21].

In the consistency assessment scenario, which evaluates how well foundation models adapt to different datasets within the same task, researchers found that parameter-efficient fine-tuning (PEFT) approaches were both efficient and effective for adapting pathology-specific foundation models to diverse datasets [21]. In the flexibility assessment scenario under data-limited environments, foundation models benefited more from few-shot learning methods that involve modification only during the testing phase rather than during training [21].

These findings highlight the practical utility of models like UNI and TITAN in real-world clinical settings, where labeled data may be scarce for specific tasks or rare conditions. The ability to perform well in few-shot and zero-shot settings is particularly valuable for clinical applications involving rare diseases or novel biomarkers where large annotated datasets are unavailable [2] [21].

Implementation and Practical Applications

The Scientist's Toolkit: Research Reagent Solutions

Implementing SSL with DINOv2 and vision-language alignment for pathology foundation models requires specific computational tools and frameworks. The following table summarizes essential "research reagents" for this domain.

Table 3: Essential Research Reagents for Pathology Foundation Model Development

Tool/Resource	Type	Function	Example Usage
DINOv2 Framework	Software Library	Self-supervised learning with knowledge distillation	Pretraining visual encoders on unlabeled histopathology images [22] [20]
UNI Model Weights	Pretrained Model	Feature extraction from histopathology images	Downloadable via Hugging Face for research use [18]
Timm Library	Software Library	Vision model architecture and training utilities	Loading UNI model architecture and transforms [18]
PyTorch-FSDP	Training Framework	Fully Sharded Data Parallel for distributed training	Efficient mixed-precision training of large models [18]
ViT-L/16 Architecture	Model Architecture	Vision Transformer with large configuration	Backbone network for UNI and related models [18]
Mass-100K/Mass-340K	Pretraining Dataset	Large-scale histopathology image collections	Training data for foundation models (access restricted) [2] [1]
PathChat	Generative AI Tool	Synthetic caption generation for pathology images	Creating fine-grained ROI captions for vision-language alignment [2]

Code Implementation and Feature Extraction

For researchers seeking to utilize existing pathology foundation models, UNI provides accessible implementation pathways through the Hugging Face ecosystem. The model can be loaded using the timm library after proper authentication:

Feature extraction from histopathology regions of interest (ROIs) follows a straightforward process:

These pre-extracted features can then be utilized for various downstream tasks including ROI classification (via linear probing or k-nearest neighbors), slide classification (using multiple instance learning frameworks), and content-based image retrieval [18].

The development of pathology foundation models using SSL with DINOv2 and vision-language alignment on datasets like Mass-100K and Mass-340K represents a transformative advancement in computational pathology. These approaches enable the creation of general-purpose visual representations that transfer effectively across diverse clinical tasks, particularly in challenging low-data regimes and for rare disease conditions.

The integration of vision-language capabilities through models like TITAN opens new possibilities for AI-assisted pathology, including cross-modal retrieval, automated report generation, and zero-shot diagnostic inference. As these methodologies continue to evolve, we anticipate further scaling of pretraining data, refinement of multimodal alignment techniques, and expanded clinical validation across diverse healthcare settings.

Future research directions likely include the incorporation of additional modalities such as genomic data, development of more efficient adaptation techniques for clinical deployment, and creation of standardized benchmarking frameworks to ensure rigorous evaluation of model capabilities and limitations. The ongoing release of foundation models like UNI and TITAN to the research community promises to accelerate innovation in AI-driven histopathology and potentially transform diagnostic workflows in clinical practice.

The field of computational pathology stands at the precipice of a revolution driven by artificial intelligence and digital transformation. Traditional pathology practice has relied on manual microscopic examination of tissue specimens, a process that is both time-consuming and subject to inter-observer variability [23]. The advent of whole-slide scanners in the 1990s enabled the creation of high-resolution digital images of entire specimens, paving the way for quantitative analysis of histopathological images using computational methods [23]. However, the development of specialized AI models for each diagnostic task proved impractical due to the immense annotation burden on pathologists, whose expertise is both costly and limited in availability [23].

Foundation models represent a paradigm shift in medical artificial intelligence by enabling models that can be adapted to many downstream, clinically relevant tasks without task-specific training from scratch [11]. These models are trained on broad data using self-supervision at scale and can be adapted to a wide range of downstream tasks [23]. In histopathology, where a single whole-slide image (WSI) contains a staggering 100,000 × 100,000 pixels—an immense wealth of biological information—the application of foundation models is particularly promising [24]. The development of Vision Transformers (ViTs) has been instrumental in this transformation, as their architecture is particularly well-suited to handling the gigapixel-scale dimensions of WSIs while capturing both local and global tissue contexts [2] [1].

This technical guide explores the architectural backbones of ViTs for whole-slide image analysis, framed within the context of the Mass-100K and Mass-340K datasets developed by Mass General Brigham researchers. These datasets represent two of the largest collections of histopathology data created for self-supervised learning in computational pathology and have served as the foundation for pioneering models like UNI and TITAN that are pushing the boundaries of what's possible in diagnostic medicine [2] [1] [11].

The Foundation: Mass-100K and Mass-340K Datasets

Dataset Composition and Scaling Laws

The Mass-100K and Mass-340K datasets represent monumental achievements in data collection for computational pathology research. The Mass-100K dataset consists of more than 100 million tissue patches from 100,426 diagnostic H&E-stained whole-slide images across 20 major tissue types collected from Massachusetts General Hospital (MGH) and Brigham and Women's Hospital (BWH), as well as the Genotype-Tissue Expression (GTEx) consortium [1]. This dataset provides a rich source of information for learning objective characterizations of histopathologic biomarkers and has been instrumental in establishing scaling laws for foundation models in computational pathology [1].

The Mass-340K dataset represents an even more ambitious expansion, comprising 335,645 whole-slide images with corresponding pathology reports and 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology [2]. The dataset is distributed across 20 organs, different stains, diverse tissue types, and various scanner types, ensuring remarkable diversity that has proven to be a key factor in successful model development [2]. This extensive collection addresses a critical challenge in computational pathology: limited clinical data in disease-specific cohorts, especially for rare clinical conditions [2].

Table 1: Composition of Mass-100K and Mass-340K Datasets

Dataset Metric	Mass-100K	Mass-340K
Total Whole-Slide Images	100,402 WSIs	335,645 WSIs
Tissue Patches/Images	>100 million	>100 million (estimated)
Organ Types	20	20
Additional Data	-	182,862 medical reports; 423,122 synthetic captions
Primary Use Cases	UNI foundation model	TITAN multimodal foundation model
Data Sources	MGH, BWH, GTEx	MGH, BWH, and other Mass General Brigham sources

Research has demonstrated clear scaling laws for foundation models in computational pathology. When scaling UNI from Mass-1K (1 million images, 1,404 WSIs) to Mass-22K (16 million images, 21,444 WSIs) to Mass-100K, performance increased by +4.2% and +3.7% respectively on challenging 43-class OncoTree cancer type classification tasks [1]. Similar improvements were observed on even more complex 108-class OncoTree code classification tasks, confirming that increasing dataset size and diversity directly enhances model performance on diagnostically relevant tasks [1].

Dataset Curation and Ethical Considerations

The curation of these massive datasets followed rigorous ethical standards. All experiments were conducted in accordance with the Declaration of Helsinki, the International Ethical Guidelines for Biomedical Research Involving Human Subjects (CIOMS), the Belmont Report and the U.S. Common Rule [25]. Anonymized archival tissue samples were retrieved from tissue banks in accordance with regulations and with approval from relevant ethics committees, with informed consent obtained from all patients as part of tissue bank protocols [25].

The datasets were designed to include diverse tissue types beyond just cancerous specimens, incorporating inflammatory, infectious, and normal tissue to enhance model generalizability [18]. This diversity is crucial for developing models that can operate effectively in real-world clinical settings where the range of specimens encompasses the full spectrum of pathological conditions.

ViT Architectures for Whole-Slide Image Analysis

Hierarchical Feature Extraction Approaches

Vision Transformers have emerged as the dominant architectural backbone for whole-slide image analysis in computational pathology due to their ability to capture long-range dependencies and multi-scale features. The fundamental challenge in applying ViTs to WSIs lies in the gigapixel resolution of the images, which makes direct processing computationally infeasible. To address this, researchers have developed hierarchical approaches that extract features at multiple levels.

The UNI model employs a Vision Transformer (ViT-Large) architecture pretrained using the DINOv2 self-supervised learning framework on the Mass-100K dataset [1] [18]. The model processes individual tissue patches at 20× magnification, typically sized 256×256 or 512×512 pixels, and learns representations through a combination of DINO self-distillation loss with multi-crop, iBOT masked-image modeling loss, and KoLeo regularization on [CLS] tokens [18]. This approach enables the model to learn powerful, transferable representations without requiring labeled data during pretraining.

Table 2: Vision Transformer Architectures for Whole-Slide Image Analysis

Architectural Component	UNI Model	TITAN Model
Base Architecture	ViT-L/16 (ViT-Large)	Vision Transformer (ViT)
Patch Size	16×16	Processes 512×512 patches at 20×
Input Resolution	224×224 for patches	8,192×8,192 region crops
Embedding Dimension	1024	768 (from CONCH v1.5 patch encoder)
Attention Heads	16	Variable
Parameters	0.3B (300 million)	Not specified
Pretraining Framework	DINOv2	iBOT knowledge distillation + multimodal alignment

The TITAN model introduces a more sophisticated approach specifically designed for whole-slide analysis. Instead of using tokens from partitioned image patches directly, the slide encoder takes a sequence of patch features encoded by powerful histology patch encoders like CONCH v1.5 [2] [26]. This means TITAN's pretraining occurs in the embedding space based on pre-extracted patch features, with the patch encoder functioning as the 'patch embedding layer' in a conventional ViT [2]. To preserve spatial context, patch features are arranged in a two-dimensional feature grid replicating the positions of corresponding patches within the tissue [2].

Handling Gigapixel Images and Long-Range Dependencies

A significant innovation in TITAN is its approach to handling the computational complexity of gigapixel whole-slide images. The model constructs input embedding space by dividing each WSI into non-overlapping patches of 512×512 pixels at 20× magnification, followed by extraction of 768-dimensional features for each patch with CONCH v1.5 [2]. To address large and irregularly shaped WSIs, TITAN creates views by randomly cropping the 2D feature grid, sampling region crops of 16×16 features covering a region of 8,192×8,192 pixels [2].

From these region crops, TITAN samples two random global (14×14) and ten local (6×6) crops for iBOT pretraining, applying augmentations including vertical and horizontal flipping followed by posterization feature augmentation [2]. Perhaps most innovatively, TITAN uses attention with linear bias (ALiBi) for long-context extrapolation at inference time, extending this technique—originally proposed for large language models—to 2D by basing linear bias on the relative Euclidean distance between features in the feature grid [2]. This approach reflects the actual distances between patches in the tissue and enables more effective modeling of long-range dependencies in whole-slide images.

Experimental Protocols and Methodologies

Pretraining Strategies and Self-Supervised Learning

The development of foundation models for computational pathology relies heavily on self-supervised learning techniques that leverage unlabeled data. The UNI model employs the DINOv2 self-supervised learning framework, which has been shown to yield strong, off-the-shelf representations for downstream tasks without need for further fine-tuning with labeled data [1]. The training regimen consists of 125,000 iterations with a batch size of 3072, using fp16 mixed-precision training via PyTorch-FSDP, totaling approximately 1024 GPU hours on 4×8 Nvidia A100 80GB hardware [18].

TITAN employs a more complex three-stage pretraining strategy to ensure that slide-level representations capture histomorphological semantics at both the region-of-interest (ROI) and whole-slide levels [2]:

• Vision-only unimodal pretraining: Using the Mass-340K dataset on ROI crops with iBOT framework, which combines masked image modeling and knowledge distillation [2].
• Cross-modal alignment at ROI-level: Incorporating 423,000 pairs of 8k×8k ROIs and captions generated using PathChat, a multimodal generative AI copilot for pathology [2].
• Cross-modal alignment at WSI-level: Utilizing 183,000 pairs of WSIs and clinical reports to enable slide-level vision-language understanding [2].

This multi-stage approach allows TITAN to develop both visual and linguistic understanding of histopathological features, enabling sophisticated capabilities like pathology report generation and cross-modal retrieval between images and text [2].

Evaluation Frameworks and Downstream Tasks

Comprehensive evaluation across diverse clinical tasks is essential for validating foundation models in pathology. UNI was assessed on 34 distinct clinical tasks of varying diagnostic difficulty, including nuclear segmentation, primary and metastatic cancer detection, cancer grading and subtyping, biomarker screening and molecular subtyping, organ transplant assessment, and several pan-cancer classification tasks that include subtyping to 108 cancer types in the OncoTree cancer classification system [1].

For weakly supervised slide classification, researchers followed the conventional paradigm of first pre-extracting patch-level features from tissue-containing patches in the WSI using a pretrained encoder, followed by training an attention-based multiple instance learning (ABMIL) algorithm [1]. Performance was measured using top-K accuracy (K = 1, 3, 5) as well as weighted F1 score and area under the receiver operating characteristic curve (AUROC) to reflect the label complexity challenges of these tasks [1].

TITAN was evaluated across even more diverse machine learning settings, including linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval, and pathology report generation [2]. The model's performance was assessed on tasks specifically designed to test generalization to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis [2]. Without any fine-tuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports, demonstrating remarkable versatility for clinical applications [2].

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Reagents for ViT Development in Computational Pathology

Research Reagent	Function	Implementation Examples
CONCH v1.5 Patch Encoder	Extracts visual features from histology patches at 512×512 resolution	Used in TITAN to create patch feature embeddings; provides 768-dimensional features [2] [26]
DINOv2 Framework	Self-supervised learning for vision transformers	Used in UNI pretraining; combines distillation with no labels, iBOT masked modeling, and KoLeo regularization [1] [18]
iBOT Framework	Joint image modeling and self-distillation with online tokenizer	Used in TITAN vision-only pretraining; enables masked image modeling and knowledge distillation [2]
ALiBi Position Encoding	Extrapolates to longer sequences than seen during training	Extended to 2D in TITAN; uses relative Euclidean distance between patches for attention bias [2]
ABMIL (Attention-Based Multiple Instance Learning)	Weakly supervised slide classification from patch features	Standard approach for WSI classification; used in evaluating UNI and other foundation models [1]
PathChat	Multimodal generative AI for pathology caption generation	Used to create 423k synthetic ROI-caption pairs for TITAN vision-language alignment [2]
Hugging Face Transformers Library	Model deployment and sharing	Hosting platform for UNI and TITAN models; provides accessible interface for researchers [18] [26]

Performance Benchmarks and Clinical Applications

Quantitative Performance Across Diagnostic Tasks

The UNI and TITAN foundation models have established new state-of-the-art performance benchmarks across a wide spectrum of computational pathology tasks. UNI demonstrates superior performance compared to previous state-of-the-art models such as CTransPath and REMEDIS, particularly on challenging large multi-class classification tasks like the 108-class OncoTree code classification [1]. The model achieves these results while maintaining robustness across tissue types and disease categories, including rare and underrepresented cancer types [1] [18].

TITAN represents further advancement, outperforming both region-of-interest (ROI) and slide foundation models across diverse machine learning settings [2]. The model exhibits exceptional capability in few-shot and zero-shot learning scenarios, demonstrating particular strength in rare cancer retrieval and cross-modal retrieval between histology slides and clinical reports [2]. Perhaps most impressively, TITAN can generate pathology reports without any fine-tuning or requiring clinical labels, showcasing the power of its multimodal pretraining approach [2].

Table 4: Performance Benchmarks of Pathology Foundation Models

Model	Pretraining Data	Key Performance Metrics	Clinical Applications
UNI	Mass-100K (100M images, 100K WSIs)	SOTA on 34 tasks; +4.2% improvement when scaling from Mass-1K to Mass-22K on OncoTree-43 classification [1]	Cancer subtyping (108 classes), organ transplant assessment, rare cancer diagnosis [1] [18]
TITAN	Mass-340K (335K WSIs + 182K reports + 423K captions)	Outperforms ROI and slide foundation models in linear probing, few-shot/zero-shot classification, rare cancer retrieval [2]	Pathology report generation, cross-modal retrieval, rare disease identification, cancer prognosis [2]
Previous SOTA (CTransPath, REMEDIS)	TCGA (~29K WSIs) and other public datasets	Competitive but lower performance on large multi-class tasks, especially rare cancers [1]	General cancer detection and classification with limitations on rare diseases

Emerging Capabilities and Clinical Implementation

Beyond traditional classification tasks, these foundation models enable previously impossible capabilities in computational pathology. UNI demonstrates novel functionalities such as resolution-agnostic tissue classification and slide classification using few-shot class prototypes for prompt-based slide classification [1]. This enables more flexible deployment in clinical settings where image acquisition parameters may vary.

TITAN's multimodal capabilities represent an even more significant advancement, allowing natural language queries of histopathological images and cross-modal retrieval between image features and textual descriptions [2] [26]. A pathologist could potentially search for similar cases by describing morphological features in text, or generate preliminary reports based on visual analysis of whole-slide images [2]. These capabilities significantly enhance pathologist workflow rather than simply automating discrete tasks.

Implementation of these models in clinical practice is facilitated through platforms like Proscia's Concentriq Embeddings, which integrates foundation models including Bioptimus's H-optimus-0 directly into pathology workflow systems [24]. Research has shown that ensemble approaches combining multiple foundation models can outperform individual models in approximately two-thirds of tasks, highlighting the importance of flexible multi-model strategies for clinical deployment [24].

The development of Vision Transformer architectures for whole-slide image analysis represents a transformative advancement in computational pathology. The Mass-100K and Mass-340K datasets provide the foundational resources necessary to train these models at unprecedented scale, while models like UNI and TITAN demonstrate the remarkable capabilities that can emerge from such large-scale pretraining. These foundation models excel not only on conventional diagnostic tasks but also enable novel capabilities like zero-shot classification, cross-modal retrieval, and report generation.

Looking forward, the integration of pathology foundation models with other medical AI systems—including those for radiology, genomics, and clinical data—will enable the development of generalist medical AI that can provide comprehensive diagnostic support [23]. Such systems will leverage the complementary strengths of different data modalities to enhance diagnostic accuracy and clinical decision-making. Additionally, continued scaling of model and dataset sizes, coupled with refinement of self-supervised learning techniques, will further improve model performance, particularly for rare diseases and underrepresented populations.

The architectural innovations in ViTs for whole-slide image analysis—including hierarchical feature extraction, multimodal alignment, and long-range context modeling—have established a robust foundation for the next generation of computational pathology tools. As these technologies continue to mature and undergo clinical validation, they hold tremendous promise for enhancing diagnostic precision, reducing pathologist workload, and ultimately improving patient outcomes through more accurate and timely diagnosis.

Computational pathology has been transformed by foundation models that learn transferable feature representations from vast collections of histopathology images without extensive manual labeling [5]. These models address critical challenges in the field, including the gigapixel size of whole-slide images (WSIs), variability in morphological features, and the high cost of expert annotations [23]. Among the most significant advancements are UNI and TITAN, developed by the Mahmood Lab, which leverage massive internal datasets—Mass-100K and Mass-340K—to achieve unprecedented performance across diverse clinical tasks [2] [1]. UNI establishes a new paradigm as a general-purpose self-supervised visual encoder for histopathology, while TITAN extends these capabilities through multimodal vision-language alignment, enabling novel applications such as zero-shot classification and pathology report generation [2] [26]. This technical guide examines their core architectures, training methodologies, and output capabilities, providing researchers with the experimental protocols and implementation details necessary to leverage these models in therapeutic R&D and diagnostic applications.

The Foundation Datasets: Mass-100K and Mass-340K

The performance of UNI and TITAN is fundamentally enabled by the scale and diversity of their pretraining datasets. These datasets provide the comprehensive histopathological representation necessary for developing robust foundation models.

Table 1: Composition of Mass-100K and Mass-340K Pretraining Datasets

Dataset	Number of WSIs	Number of Images/Tiles	Data Sources	Tissue Types	Staining Types
Mass-100K	100,402	>100 million	BWH, MGH, GTEx [1]	20 major tissue types [1]	H&E [1]
Mass-340K	335,645	Not specified	BWH, MGH, GTEx [2]	20 organ types [2]	Diverse stains [2]

Mass-100K serves as the pretraining dataset for UNI, consisting of diagnostic H&E-stained WSIs from Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), and the Genotype-Tissue Expression (GTEx) consortium [1]. This dataset provides a rich source of information for learning objective characterizations of histopathologic biomarkers across diverse tissue types and disease categories. The scale of Mass-100K—over 100 million tissue patches—enables UNI to learn generalizable representations without using publicly available datasets like The Cancer Genome Atlas (TCGA), preventing data contamination when evaluating on public benchmarks [18].

Mass-340K represents an expanded dataset used for pretraining TITAN, comprising 335,645 WSIs across 20 organ types with different stains, diverse tissue types, and various scanner types [2]. This dataset's increased scale and diversity are crucial for TITAN's multimodal capabilities, as it also includes 182,862 medical reports and 423,122 synthetic captions generated using PathChat, a multimodal generative AI copilot for pathology [2] [26]. The inclusion of both real clinical reports and synthetically generated fine-grained descriptions enables TITAN to align visual patterns with textual descriptions at both the region-of-interest and whole-slide levels.

UNI: General-Purpose Slide Encoding

Architecture and Pretraining Methodology

UNI implements a general-purpose self-supervised vision encoder based on a Vision Transformer (ViT-Large/16) architecture pretrained using the DINOv2 framework [1] [18]. The model was trained on the Mass-100K dataset using a self-supervised learning approach that combines several objectives: DINO self-distillation loss with multi-crop, iBOT masked-image modeling loss, and KoLeo regularization on [CLS] tokens [18]. This multi-objective pretraining strategy enables the model to learn rich, contextual representations without requiring labeled data.

The technical implementation details include training for 125,000 iterations with a batch size of 3072 using fp16 mixed-precision training via PyTorch-FSDP [18]. The ViT-Large architecture contains approximately 300 million parameters, with a patch size of 16, embedding dimension of 1024, 16 attention heads, and MLP feed-forward networks [18]. This substantial model capacity enables UNI to capture both fine-grained cellular structures and broader tissue architecture patterns essential for pathological assessment.

Key Output Capabilities and Experimental Validation

UNI produces versatile slide representations that demonstrate state-of-the-art performance across 34 clinical tasks of varying diagnostic difficulty [1]. The model's key capabilities include resolution-agnostic tissue classification, few-shot class prototypes for prompt-based slide classification, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree classification system [1].

Table 2: UNI Performance on Representative Clinical Tasks

Task Type	Dataset/Evaluation	Key Metric	Performance	Competitive Baseline
Rare Cancer Classification	OncoTree-108 (108 cancer types)	Top-1 Accuracy	Significantly outperforms baselines [1]	CTransPath, REMEDIS [1]
Metastasis Detection	CAMELYON16	AUROC	State-of-the-art [1]	Previous patch-based methods [1]
Cancer Subtyping	NSCLC Subtyping	Accuracy	Superior generalization [1]	ROI-based foundation models [1]
Few-Shot Learning	Various tissue types	5-shot accuracy	Competitive with fully supervised models [1]	Traditional supervised learning [1]

In large-scale evaluations, UNI demonstrated remarkable scaling properties, with performance monotonically improving as pretraining data increased from Mass-1K to Mass-100K [1]. On the challenging OncoTree-43 and OncoTree-108 tasks, which include many rare cancer types, UNI showed performance increases of +3.7% and +3.0% respectively when scaling from Mass-22K to Mass-100K [1]. This demonstrates that both model and data scaling are pivotal for achieving strong performance on diagnostically challenging and rare cancer classification tasks.

TITAN: Multimodal Whole-Slide Foundation Model

Multimodal Architecture and Training Strategy

TITAN represents a significant advancement beyond unimodal approaches through its multimodal architecture that aligns whole-slide images with textual descriptions. The model is built upon a Vision Transformer framework specifically designed to handle long sequences of patch features extracted from gigapixel WSIs [2]. Unlike traditional patch-based models, TITAN operates on pre-extracted patch features from CONCHv1.5, arranging them in a 2D feature grid that preserves spatial relationships between tissue regions [2] [26].

The pretraining strategy consists of three distinct stages: (1) vision-only unimodal pretraining on ROI crops from Mass-340K using iBOT framework, (2) cross-modal alignment of generated morphological descriptions at ROI-level using 423k pairs of ROIs and synthetic captions, and (3) cross-modal alignment at WSI-level using 183k pairs of WSIs and clinical reports [2]. This staged approach enables TITAN to learn hierarchical representations that capture both local histological patterns and global slide-level context.

To address the computational challenges of processing gigapixel WSIs, TITAN employs several innovations: using larger patch sizes (512×512 pixels at 20× magnification), random cropping of the 2D feature grid into region crops of 16×16 features (covering 8,192×8,192 pixels), and attention with linear bias (ALiBi) for long-context extrapolation [2]. These technical choices enable TITAN to efficiently process variable-sized WSIs while maintaining critical spatial context.

Multimodal Output Capabilities and Experimental Validation

TITAN demonstrates exceptional capabilities in zero-shot classification, cross-modal retrieval, and pathology report generation without task-specific fine-tuning [2]. The model's slide representations outperform both region-of-interest and slide foundation models across diverse machine learning settings, including linear probing, few-shot learning, and rare cancer retrieval [2].

A particularly notable capability is TITAN's performance in rare cancer retrieval tasks, where it successfully identifies diagnostically challenging cases with limited training examples [2]. This addresses a critical clinical need in anatomic pathology practice, where rare entities often present diagnostic difficulties due to their infrequency. TITAN also enables bidirectional cross-modal retrieval, allowing pathologists to query similar cases by either image or textual description, significantly enhancing diagnostic workflow efficiency.

In comprehensive evaluations, TITAN demonstrated superior performance compared to existing slide foundation models, particularly in low-data regimes and language-guided zero-shot classification [2]. The incorporation of synthetic fine-grained morphological descriptions generated by PathChat proved especially valuable, suggesting substantial potential for scaling TITAN's pretraining with synthetic data [2].

Experimental Protocols and Implementation

Feature Extraction and Model Inference

Implementing UNI and TITAN for research applications requires specific technical setups and workflows. For UNI, feature extraction from histopathology regions-of-interest follows a standardized protocol using the timm library for model loading and inference [18]. The recommended approach involves:

• Loading the model with pretrained weights: model = timm.create_model("hf-hub:MahmoodLab/uni", pretrained=True, init_values=1e-5, dynamic_img_size=True)
• Applying the appropriate image transforms: transform = create_transform(resolve_data_config(model.pretrained_cfg, model=model))
• Extracting features via forward pass: feature_emb = model(image) [18]

For TITAN, the feature extraction process operates on precomputed CONCHv1.5 patch features organized in HDF5 files containing feature tensors and coordinate information [26]. Slide-level embedding extraction follows this protocol:

• Loading the model: titan = AutoModel.from_pretrained('MahmoodLab/TITAN', trust_remote_code=True)
• Loading patch features and coordinates from HDF5 files
• Extracting slide embeddings: slide_embedding = model.encode_slide_from_patch_features(features, coords, patch_size_lv0) [26]

Both models support extraction of features for downstream tasks without full model fine-tuning, enabling efficient transfer learning through linear probing, k-nearest neighbors classification, or multiple instance learning approaches.

Downstream Task Adaptation Strategies

Adapting UNI and TITAN to specific clinical tasks requires careful selection of fine-tuning strategies based on available labeled data. Recent benchmarking studies have identified optimal approaches for pathology foundation model adaptation [21]:

• Full fine-tuning: Effective when sufficient labeled data is available (>1,000 labeled examples)
• Parameter-efficient fine-tuning (PEFT): Optimal for medium-data regimes (100-1,000 examples)
• Linear probing: Suitable for few-shot settings (<100 examples)
• Zero-shot learning: Possible with TITAN using natural language prompts

For slide-level classification tasks, the conventional paradigm involves first pre-extracting patch-level features from tissue-containing patches in the WSI using the pretrained encoder, followed by training an attention-based multiple instance learning (ABMIL) algorithm [1]. This approach has demonstrated state-of-the-art performance for cancer classification and subtyping tasks across multiple cancer types.

The Scientist's Toolkit: Essential Research Reagents

Table 3: Key Research Reagents and Computational Resources for UNI and TITAN Implementation

Resource	Type	Function	Access Method
UNI Model Weights	Pretrained Model	General-purpose feature extraction from histopathology images	Hugging Face Hub (MahmoodLab/UNI) [18]
TITAN Model Weights	Multimodal Model	Slide-level encoding and vision-language tasks	Hugging Face Hub (MahmoodLab/TITAN) [26]
CONCH v1.5	Patch Encoder	Patch feature extraction for TITAN preprocessing	Integrated in TITAN codebase [2]
Mass-100K Features	Precomputed Features	UNI embeddings for specific datasets	Available through model repositories [18]
TCGA TITAN Features	Precomputed Features	TITAN slide embeddings for TCGA samples	Provided as .pkl files [26]
DINOv2 Framework	SSL Algorithm	Self-supervised learning backbone for UNI	GitHub repository [18]
CLAM Algorithm	MIL Framework	Slide classification with multiple instance learning	GitHub repository [18]

Successful implementation of UNI and TITAN requires specific computational resources and dependencies. UNI requires PyTorch with specific versions of timm, einops, and other dependencies listed in the model card [18]. The model was trained using 32 Nvidia A100 80GB GPUs for approximately 32 hours (1024 GPU hours total) [18], though inference requires significantly less computational resources.

TITAN has similar requirements with additional dependencies for handling multimodal inputs and processing whole-slide images [26]. The recommended environment includes torch==2.0.1, timm==1.0.3, einops==0.6.1, and transformers==4.46.0 [26]. For both models, utilizing precomputed features can significantly reduce computational requirements during experimental evaluation.

UNI and TITAN represent significant milestones in the development of foundation models for computational pathology, demonstrating the transformative potential of large-scale self-supervised learning on diverse histopathology datasets. UNI establishes a new state-of-the-art for general-purpose visual encoding in pathology, while TITAN pioneers multimodal capabilities that bridge visual patterns with clinical language. Their performance across diverse clinical tasks—from rare cancer classification to pathology report generation—highlights the practical utility of these models in both research and clinical settings.

The continued evolution of pathology foundation models will likely focus on several key directions: increased multimodal integration with genomic and clinical data, more efficient architectures for processing gigapixel images, federated learning approaches to leverage distributed data sources while maintaining privacy, and improved interpretability methods for clinical translation. As these models mature, they are poised to become indispensable tools in the development of precision diagnostics and therapeutics, ultimately enhancing patient care through more accurate, efficient, and standardized pathological assessment.

The Mass-340K dataset represents a pivotal advancement in computational pathology, serving as the foundational training corpus for developing powerful whole-slide foundation models. This massive dataset, formally known as Mass-340K, comprises 335,645 whole-slide images (WSIs) and 182,862 corresponding medical reports across 20 different organ types, incorporating diverse stains, tissue types, and scanner variants [2]. This scale and diversity have enabled the training of sophisticated models like TITAN (Transformer-based pathology Image and Text Alignment Network), which leverages this extensive data through a multi-stage pretraining paradigm to address complex clinical challenges including cancer subtyping, biomarker prediction, prognosis, and slide retrieval [2].

The significance of Mass-340K lies in its application to slide-level representation learning. While previous patch-based foundation models excelled at encoding regional histopathology patterns, translating these capabilities to patient- and slide-level clinical tasks remained constrained by limited clinical data, especially for rare conditions [2]. The Mass-340K dataset directly addresses this limitation by enabling the development of models that can encode entire gigapixel WSIs into general-purpose slide representations, facilitating diverse downstream applications without requiring extensive task-specific fine-tuning [2].

Model Architecture and Pretraining Methodology

TITAN: A Multimodal Whole-Slide Foundation Model

The TITAN model represents a breakthrough in whole-slide analysis, employing a Vision Transformer (ViT) architecture specifically designed to handle the unique challenges of gigapixel WSIs [2]. Unlike conventional patch-based approaches, TITAN operates on pre-extracted patch features arranged in a two-dimensional spatial grid that preserves the topological relationships between tissue regions [2].

Table 1: TITAN Model Specifications and Pretraining Data

Component	Specification	Description
Base Architecture	Vision Transformer (ViT)	Processes WSIs as sequences of patch embeddings
Patch Feature Extraction	CONCHv1.5 encoder	Generates 768-dimensional features from 512×512 patches at 20× magnification
Input Representation	2D feature grid (16×16 region crops)	Covers 8,192×8,192 pixel regions (4×4 mm² at 20×)
Pretraining Data	335,645 WSIs + 182,862 reports	Mass-340K dataset spanning 20 organ types
Synthetic Captions	423,122 ROI-text pairs	Generated via PathChat multimodal AI copilot
Positional Encoding	Attention with Linear Biases (ALiBi)	Enables long-context extrapolation for variable-sized WSIs

Three-Stage Pretraining Paradigm

TITAN undergoes a sophisticated three-stage pretraining process to develop comprehensive visual and multimodal capabilities:

Stage 1: Vision-Only Unimodal Pretraining The model initializes with self-supervised learning on ROI crops using the iBOT framework, which combines masked image modeling and knowledge distillation objectives. This stage trains the model to understand histomorphological patterns at the region level [2].

Stage 2: ROI-Level Cross-Modal Alignment The vision encoder learns to align with fine-grained morphological descriptions by contrasting with 423,122 synthetic captions generated using PathChat, a multimodal generative AI copilot for pathology [2].

Stage 3: WSI-Level Cross-Modal Alignment The final stage aligns entire whole-slide representations with corresponding pathology reports, enabling slide-level language understanding and retrieval capabilities [2].

Diagram 1: TITAN Three-Stage Pretraining Workflow (63 characters)

Experimental Protocols for Downstream Clinical Applications

Zero-Shot Classification Methodology

For cancer subtyping and classification tasks, TITAN employs a sophisticated zero-shot inference approach that requires no task-specific fine-tuning. Given a WSI, the model processes the entire slide by dividing it into smaller tiles, computing similarity scores between each tile and text prompts representing different diagnostic classes, then aggregating these scores into a slide-level prediction [27].

The text prompt engineering follows an ensemble approach where multiple phrasings of the same concept are combined to improve robustness. For example, "invasive lobular carcinoma (ILC) of the breast" and "breast ILC" might both be used as prompts for the same class, with the final prediction based on aggregated similarity scores across all prompt variations [27].

Slide Retrieval and Report Generation Protocols

For slide retrieval tasks, TITAN leverages its cross-modal alignment capabilities to compute similarity between query slides and database entries, or between text queries and whole-slide images. The model encodes both modalities into a shared embedding space where semantic similarity can be measured using cosine distance [2].

Pathology report generation employs the multimodal fusion decoder to generate free-text morphological descriptions based on visual features extracted from WSIs. This capability is particularly valuable for generating preliminary reports or assisting with standardized reporting in resource-limited settings [2].

Performance Evaluation and Comparative Analysis

Quantitative Results Across Clinical Tasks

Table 2: Performance Comparison of Foundation Models on Cancer Subtyping Tasks

Task/Dataset	Model	Metric	Performance	Performance Advantage
NSCLC Subtyping (TCGA)	CONCH	Accuracy	90.7%	+12.0% vs PLIP [27]
	PLIP	Accuracy	78.7%	Baseline
RCC Subtyping (TCGA)	CONCH	Accuracy	90.2%	+9.8% vs PLIP [27]
	PLIP	Accuracy	80.4%	Baseline
BRCA Subtyping (TCGA)	CONCH	Accuracy	91.3%	~+35% vs other models [27]
	BiomedCLIP	Accuracy	55.3%	Near-random performance
LUAD Pattern Classification (DHMC)	CONCH	Cohen's κ	0.200	+0.12 vs PLIP [27]
Gleason Pattern Classification (SICAP)	CONCH	Quadratic κ	0.690	+0.140 vs BiomedCLIP [27]

TITAN demonstrates particular strength in low-data regimes and rare disease scenarios. The model outperforms both region-of-interest (ROI) and existing slide foundation models across multiple machine learning settings, including linear probing, few-shot learning, and zero-shot classification [2]. This capability is crucial for real-world clinical applications where labeled data for rare conditions is often scarce.

Performance in Resource-Limited Scenarios

The Mass-340K-pretrained models show exceptional capability in handling challenging clinical scenarios with limited resources. In rare cancer retrieval tasks, TITAN significantly outperforms existing methods by leveraging its comprehensive understanding of histomorphological patterns acquired during large-scale pretraining [2]. The model's cross-modal retrieval capabilities enable clinicians to find similar cases based on either image queries or text descriptions, facilitating knowledge transfer and decision support for uncommon conditions.

Diagram 2: Zero-Shot Evaluation Workflow for Clinical Tasks (55 characters)

The Scientist's Toolkit: Essential Research Reagents

Table 3: Key Foundation Models and Computational Tools in Computational Pathology

Resource	Type	Key Features	Clinical Applications
TITAN	Multimodal Whole-Slide Foundation Model	ViT architecture, ALiBi positional encoding, 3-stage pretraining	Zero-shot classification, slide retrieval, report generation [2]
CONCH	Visual-Language Foundation Model	Contrastive learning + captioning objectives, 1.17M image-text pairs	Tile & WSI classification, segmentation, cross-modal retrieval [27]
PLIP	Vision-Language Model	Open-source, contrastive learning on pathology-specific data	ROI classification, similarity search [28]
DINOv2	Computer Vision Foundation Model	Self-supervised learning on natural images, strong feature extraction	Feature extraction for downstream pathology tasks [28]
CTransPath	Transformer-based Feature Extractor	Pretrained on histology images, optimized for tissue features	Tile-level feature extraction [28]
Concentriq Embeddings	Commercial Platform	Integrated foundation model access, simplified WSI processing	Rapid prototyping, embedding generation for clinical AI [28]

The Mass-340K dataset has proven instrumental in developing sophisticated foundation models that excel at diverse downstream clinical applications including cancer subtyping, biomarker prediction, prognosis, and slide retrieval. The scale and diversity of this dataset enable models like TITAN to overcome the limitations of previous approaches, particularly in low-data scenarios and rare disease contexts.

Future research directions include expanding the multimodal capabilities of these models to incorporate genomic and transcriptomic data, enhancing few-shot learning performance for ultra-rare conditions, and developing more efficient inference methods for real-time clinical deployment. The continued evolution of pathology foundation models trained on massive, diverse datasets like Mass-340K promises to significantly accelerate the development of robust AI tools for diagnostic pathology, ultimately enhancing patient care through improved diagnostic accuracy and workflow efficiency.

Navigating Challenges and Scaling Performance with Mass-100K and Mass-340K

The development of pathology foundation models represents a paradigm shift in computational pathology, enabling the application of artificial intelligence to complex clinical tasks such as cancer diagnosis, prognosis, and biomarker prediction. However, translating the capabilities of patch-based foundation models to address patient- and slide-level clinical challenges remains constrained by the immense scale of gigapixel whole-slide images (WSIs) and the limited size of disease-specific patient cohorts, particularly for rare conditions. The fundamental computational hurdle stems from the fact that a single WSI can encompass billions of pixels, creating input sequences orders of magnitude longer than those encountered in natural image processing. This article examines how recent research, centered on the Mass-100K and Mass-340K datasets, has pioneered new architectural and methodological approaches to overcome these challenges, thereby enabling the development of transformative models like TITAN that effectively process entire slides while capturing both local morphological details and global tissue architecture.

The Mass-100K and Mass-340K Datasets: Foundation for Innovation

The creation of large-scale, diverse datasets has been instrumental in addressing the computational challenges of WSI analysis. Research initiatives have demonstrated that the Cancer Genome Atlas (TCGA), while valuable, contains insufficient data for effective foundation model development. This recognition spurred the creation of larger, more comprehensive datasets [29].

The Mass-100K dataset emerged as a significant milestone, containing over 100,000 whole-slide images across 20 tissue types collected from Mass General Hospital, Brigham & Women's Hospital, and the Genotype-Tissue Expression (GTEx) consortium [29]. This dataset provided the foundational diversity necessary for developing models capable of generalizing across multiple organs and disease types.

Building upon this foundation, the Mass-340K dataset expanded the scale dramatically to 335,645 whole-slide images from a diverse set of neoplastic, infectious, and inflammatory cases at Mass General Brigham [2] [26]. The dataset's composition across 20 organs, different stains, diverse tissue types, and various scanner types ensured the morphological diversity essential for robust model training [2]. Additionally, Mass-340K incorporated rich multimodal data, including 182,862 medical reports and 423,122 synthetic captions generated using PathChat, a multimodal generative AI copilot for pathology [2] [26]. This extensive data collection provided the necessary substrate for developing and testing approaches to manage gigapixel images and long-sequence inputs.

Table 1: Mass-340K Dataset Composition

Component	Scale	Sources	Applications
Whole-Slide Images	335,645 WSIs	Mass General Brigham, GTEx consortium	Visual self-supervised learning
Medical Reports	182,862 reports	Accompanying clinical data	Vision-language alignment
Synthetic Captions	423,122 captions	Generated via PathChat	Fine-grained morphological description

Technical Architectures for Gigapixel Image Processing

Hierarchical Feature Extraction Framework

Processing gigapixel WSIs requires a hierarchical approach that balances computational feasibility with morphological preservation. The TITAN model introduces a sophisticated framework that operates in the feature embedding space rather than directly on raw pixels [2]. This approach begins with dividing each WSI into non-overlapping patches of 512×512 pixels at 20× magnification, significantly larger than the commonly used 256×256 patches [2]. Each patch is processed through a pre-trained patch encoder (CONCH v1.5) to extract 768-dimensional feature representations [26]. These patch features are then spatially arranged in a two-dimensional grid that mirrors the original tissue organization, effectively creating a "feature map" of the entire slide at a greatly reduced computational scale while preserving spatial relationships [2].

Managing Long Input Sequences with Adaptive Cropping

The feature grid approach reduces but does not eliminate the sequence length challenge. A complete WSI can still yield feature grids containing over 10,000 patches, creating input sequences far exceeding the capabilities of standard Transformer architectures. To address this, researchers developed a multi-scale cropping strategy during training [2]. From the initial feature grid, region crops of 16×16 features (covering 8,192×8,192 pixels at 20×) are randomly sampled. From these region crops, the model extracts two random global crops (14×14 features) and ten local crops (6×6 features) for self-supervised pretraining using the iBOT framework [2]. This approach enables the model to learn representations at multiple scales while maintaining computational tractability.

Table 2: Multi-Scale Processing Architecture

Processing Level	Spatial Resolution	Feature Dimension	Context Captured
Patch Level	512×512 pixels	768-dimensional vectors	Cellular and sub-cellular features
Local Crop	6×6 features (3,072×3,072 pixels)	6×6×768	Tissue microarchitecture
Global Crop	14×14 features (7,168×7,168 pixels)	14×14×768	Regional tissue patterns
Slide Level	Variable (entire WSI)	1×768	Whole-slide representation

Positional Encoding for Tissue Context Preservation

Preserving spatial context across the irregularly shaped, gigapixel canvas of a WSI presents unique challenges. Standard Transformer positional encodings struggle with the extreme sequence lengths and two-dimensional spatial relationships inherent in tissue sections. The TITAN model addresses this through Attention with Linear Biases (ALiBi), extended to two dimensions [2]. This approach replaces traditional positional embeddings with a bias term based on the relative Euclidean distance between features in the tissue space [2]. The linear bias is determined by the actual physical distances between patches in the tissue, allowing the model to better extrapolate to varying slide sizes and shapes during inference while maintaining awareness of spatial relationships critical for pathological assessment.

Experimental Protocols and Methodologies

Three-Stage Pretraining Methodology

The development of effective whole-slide foundation models requires a carefully structured pretraining approach. The TITAN framework implements a three-stage methodology that progressively builds capabilities [2]:

Stage 1: Vision-Only Unimodal Pretraining In this initial stage, the model undergoes self-supervised learning using only the WSI data from Mass-340K. The training employs the iBOT framework, which combines masked image modeling with online tokenizer learning [2]. This approach enables the model to learn robust visual representations without requiring manual annotations. The model learns to reconstruct masked portions of the feature crops while simultaneously developing a compact representation of tissue morphology.

Stage 2: ROI-Level Cross-Modal Alignment The second stage introduces fine-grained morphological descriptions at the region-of-interest (ROI) level. The model learns to align 8K×8K pixel ROIs with synthetic captions generated by PathChat [2]. This stage bridges the gap between visual patterns and textual descriptions, enabling the model to understand and eventually generate detailed morphological descriptions.

Stage 3: WSI-Level Cross-Modal Alignment The final stage operates at the whole-slide level, aligning entire WSIs with their corresponding pathology reports [2]. This stage provides clinical context and enables slide-level multimodal reasoning, essential for applications such as cross-modal retrieval and report generation.

Evaluation Framework for Whole-Slide Representations

Rigorous evaluation of whole-slide foundation models requires diverse benchmarks that test generalization across multiple clinical scenarios. Researchers have established comprehensive evaluation protocols assessing models across multiple machine learning settings [2]:

• Linear Probing: Training a linear classifier on frozen slide embeddings to assess representation quality
• Few-Shot Learning: Evaluating performance with limited labeled examples
• Zero-Shot Classification: Testing generalization to unseen categories without task-specific training
• Rare Cancer Retrieval: Assessing performance on diagnostically challenging rare diseases
• Cross-Modal Retrieval: Evaluating alignment between visual and textual representations
• Pathology Report Generation: Testing the model's ability to generate clinically relevant text descriptions

This multifaceted evaluation strategy ensures that models are assessed not only on standard classification tasks but also on capabilities essential for real-world clinical application.

Visualization of Core Architectures and Workflows

TITAN Model Architecture: This diagram illustrates the hierarchical processing of gigapixel whole-slide images into compact slide embeddings, showcasing the key steps from patch extraction through multi-scale cropping to final representation generation.

Three-Stage Training Pipeline: This workflow details the progressive training methodology used to develop TITAN, from vision-only pretraining through multimodal alignment at both region and whole-slide levels.

The Scientist's Toolkit: Essential Research Reagents

Table 3: Computational Pathology Research Toolkit

Tool/Resource	Type	Function	Application in WSI Analysis
CONCH v1.5	Patch Encoder	Extracts 768-dimensional features from 512×512 image patches	Foundation feature extraction for hierarchical processing
iBOT Framework	Self-Supervised Algorithm	Combines masked image modeling with online tokenizer learning	Pretraining without manual annotations
ALiBi (2D Extension)	Positional Encoding Scheme	Uses relative Euclidean distance for spatial context	Handling long sequences in gigapixel images
PathChat	Synthetic Caption Generator	Generates fine-grained morphological descriptions	Providing textual supervision for vision-language alignment
IFQuant	Web-Based Analysis Tool	Processes multiplexed immunofluorescence data	Supporting multimodal tissue analysis in consortia like IMMUcan
Layer-wise Relevance Propagation (LRP)	Explanation Method	Generates high-resolution heatmaps for model decisions	Interpreting model predictions and detecting biases

The computational hurdles inherent in processing gigapixel WSIs and managing long input sequences represent significant barriers to the development of effective pathology foundation models. However, through strategic approaches centered on hierarchical processing, multi-scale representation learning, and innovative positional encoding schemes, researchers have demonstrated viable pathways forward. The Mass-100K and Mass-340K datasets have played pivotal roles in this progress, providing the scale and diversity necessary to develop and validate these approaches. As the field advances, future research directions will likely focus on improving computational efficiency further, enhancing robustness to institutional biases, and developing more sophisticated multimodal understanding capabilities. The successful integration of these technological advances with clinical workflows holds the promise of transforming pathology practice through more accurate diagnostics, personalized treatment strategies, and improved patient outcomes.

The development of powerful artificial intelligence (AI) in computational pathology hinges on the creation of foundation models—versatile, pre-trained neural networks that can be adapted to numerous downstream clinical tasks. The Mass-100K and Mass-340K datasets represent cornerstone pretraining collections that have enabled researchers to empirically study how model and data size impact performance on complex pathology tasks. These datasets provide the substrate for investigating scaling laws in a domain where gigapixel whole-slide images (WSIs) present unique computational challenges and where models must recognize morphological patterns across diverse disease states and tissue types.

Research demonstrates that foundation models pretrained on these large, diverse datasets exhibit significantly enhanced capabilities in diagnostic accuracy, prognostic insight, and prediction of therapeutic responses. The Mass-100K dataset, comprising over 100 million tissue patches from 100,426 diagnostic H&E-stained WSIs across 20 major tissue types, has served as a benchmark for visual self-supervised learning in pathology [1] [30]. Its larger counterpart, Mass-340K, expands this foundation with 335,645 WSIs and incorporates multimodal elements, including corresponding pathology reports and 423,122 synthetic captions, enabling vision-language pretraining [2]. The systematic evaluation of models trained on these datasets has revealed clear scaling relationships: increasing both model complexity and pretraining data volume and diversity leads to substantial performance gains across challenging clinical tasks, particularly for rare cancers and fine-grained disease subtyping [2] [1].

Experimental Protocols for Investigating Scaling Laws

Model Architecture and Pretraining Methodologies

Research into scaling laws for pathology foundation models has employed rigorous experimental protocols centered on self-supervised learning (SSL) applied to large-scale histopathology data. The fundamental approach involves pretraining model encoders on unlabeled image data using pretext tasks that generate their own supervisory signals, forcing the model to learn meaningful semantic features of histopathology without expensive manual annotations [31].

The UNI model, a visual-centric foundation model, exemplifies this approach. It utilizes a Vision Transformer (ViT) architecture pretrained using the DINOv2 framework on the Mass-100K dataset [1] [30]. The pretraining process involves dividing WSIs into non-overlapping patches, which then undergo feature extraction. The self-supervised objective requires the model to produce consistent representations for different augmented views of the same image, enabling learning of transferable features without labels [31].

For multimodal understanding, the TITAN (Transformer-based pathology Image and Text Alignment Network) model employs a three-stage pretraining strategy on the larger Mass-340K dataset: (1) vision-only unimodal pretraining on region-of-interest (ROI) crops using masked image modeling and knowledge distillation; (2) cross-modal alignment of generated morphological descriptions at the ROI-level; and (3) cross-modal alignment at the WSI-level with clinical reports [2]. This progressive approach enables the model to capture histomorphological semantics at multiple scales while integrating visual and language representations.

Evaluation Frameworks and Downstream Tasks

To quantitatively assess scaling effects, researchers have established comprehensive evaluation frameworks spanning diverse clinical tasks of varying diagnostic difficulty. These include:

• Slide-level tasks: Cancer subtyping, biomarker prediction, outcome prognosis, and slide retrieval
• Region-of-interest (ROI) tasks: Nuclear segmentation, tissue classification, and image retrieval
• Multimodal tasks: Cross-modal retrieval, pathology report generation, and zero-shot classification

A particularly revealing evaluation has been the introduction of hierarchical rare cancer classification based on the OncoTree cancer classification system. This includes the OT-43 (43 cancer types) and OT-108 (108 OncoTree codes) tasks, where 90 of the 108 cancer types are designated as rare according to the RARECARE project and NCI-SEER Program [1]. These tasks assess model capabilities on fine-grained, real-world diagnostic challenges that reflect the complexity of actual pathology practice.

For slide-level classification, the standard evaluation paradigm involves first pre-extracting patch-level features from tissue-containing patches in the WSI using a pretrained encoder, followed by training an attention-based multiple instance learning (ABMIL) algorithm [1]. Performance is measured using top-K accuracy (K = 1, 3, 5), weighted F1 score, and area under the receiver operating characteristic curve (AUROC) to fully capture label complexity challenges.

Table 1: Key Pathology Foundation Models and Their Pretraining Specifications

Model	Architecture	Pretraining Data	Pretraining Method	Parameters	Multimodal
UNI	ViT-Large	Mass-100K (100M+ patches, 100K+ WSIs)	DINOv2	~307M	No
TITAN	Vision Transformer	Mass-340K (335,645 WSIs)	iBOT + Vision-Language Alignment	Not specified	Yes (image + text)
CONCH	ViT-B/16	1.17M image-text pairs	iBOT/CoCa	86.3M	Yes (image + text)
CTransPath	Swin-T/14	TCGA + PAIP	MoCov3	28.3M	No

Quantitative Analysis of Scaling Effects

Data Scaling Laws

Empirical results from pathology foundation model research demonstrate clear data scaling laws, with performance on downstream tasks improving monotonically as pretraining dataset size and diversity increase. On the challenging OT-43 and OT-108 cancer classification tasks, researchers observed significant performance gains when scaling UNI from Mass-1K (1 million images, 1,404 WSIs) to Mass-22K (16 million images, 21,444 WSIs), and further to the full Mass-100K dataset [1].

When scaling UNI using ViT-L from Mass-1K to Mass-22K, performance increased by +4.2% in top-1 accuracy on OT-43 and by +3.5% on OT-108 (P < 0.001 for both) [1]. Further scaling from Mass-22K to Mass-100K yielded additional gains of +3.7% and +3.0% on OT-43 and OT-108, respectively (P < 0.001) [1]. These improvements demonstrate that increased pretraining data volume and diversity directly enhance model capability on complex, fine-grained diagnostic tasks.

The TITAN model, pretrained on the even larger Mass-340K dataset, showed additional capabilities in zero-shot classification, rare cancer retrieval, and pathology report generation, outperforming existing slide foundation models across machine learning settings including linear probing, few-shot, and zero-shot classification [2]. This suggests that scaling beyond hundreds of thousands of WSIs continues to yield performance benefits, particularly for multimodal understanding and low-resource scenarios.

Model Scaling and Architecture Effects

Research has also revealed significant effects of model scale on performance. Ablation studies with UNI compared two different Vision Transformer architecture sizes—ViT-Base (ViT-B) and ViT-Large (ViT-L)—across different data scales [1]. The results showed that larger model architectures consistently outperformed smaller ones when pretrained on equivalent data, with the performance gap widening as dataset size increased.

However, the scaling relationship between model and data size follows a predictable pattern: performance gains from increased model size diminish if the pretraining dataset is not sufficiently large and diverse [1] [30]. This highlights the importance of balanced scaling—increasing both model capacity and training data volume—to achieve optimal performance.

Table 2: Performance Scaling with Data and Model Size on Cancer Subtyping Tasks

Model Scale	Data Scale	OT-43 Top-1 Accuracy	OT-108 Top-1 Accuracy	Notable Capabilities
ViT-Base	Mass-1K (1M images)	Baseline	Baseline	Basic tissue recognition
ViT-Base	Mass-22K (16M images)	+3.9%	+3.2%	Improved cancer subtyping
ViT-Base	Mass-100K (100M+ images)	+4.1%	+3.3%	Plateaus in some tasks
ViT-Large	Mass-1K (1M images)	+1.2% over ViT-B	+1.0% over ViT-B	Better feature quality
ViT-Large	Mass-22K (16M images)	+5.1% over ViT-B	+4.2% over ViT-B	Strong few-shot learning
ViT-Large	Mass-100K (100M+ images)	+8.8% over ViT-B	+7.5% over ViT-B	Rare cancer identification

Scaling Laws in Pathology Foundation Models

Practical Implications for Complex Pathology Tasks

Enhanced Performance on Rare and Challenging Diseases

The scaling laws observed in pathology foundation models have particularly significant implications for diagnosing rare and challenging diseases. Models pretrained on large, diverse datasets like Mass-100K and Mass-340K demonstrate remarkable capabilities in identifying rare cancers and fine-grained disease subtypes that pose diagnostic challenges even for expert pathologists.

On a challenging 12-class brain tumor subtyping task based on the EBRAINS Digital Tumor Atlas, UNI achieved a balanced accuracy of 88.3%, outperforming ResNet-50 by 53.6%, CTransPath by 21.7%, and REMEDIS by 19.6% [30]. In few-shot settings for this task, the 4-shot performance of UNI matched the 32-shot performance of REMEDIS, representing an 8× improvement in label efficiency [30]. This dramatic improvement demonstrates how scaling enables practical applications in scenarios with limited annotated examples.

For the TITAN model, pretraining on the massive Mass-340K dataset enabled strong performance in rare cancer retrieval scenarios, where the model must identify similar cases from a database of rare diseases with limited training examples [2]. This capability has direct clinical utility for assisting pathologists facing diagnostically challenging cases by retrieving morphologically similar cases and their associated reports.

Resolution-Agnostic Classification and Few-Shot Learning

Scaled foundation models exhibit emergent capabilities beyond basic classification tasks. UNI demonstrates resolution-agnostic tissue classification, maintaining robust performance across varying image resolutions and microns per pixel (mpp) values [30]. This contrasts with other pretrained encoders that deteriorate in performance when image resolution changes, highlighting how scale enables more flexible and adaptable representations.

Another significant capability is few-shot class prototyping, where models can learn representative feature vectors ("class prototypes") that characterize class-specific morphological patterns. Using the SimpleShot framework with UNI features, researchers developed "MI-SimpleShot," a highly efficient system for slide classification that works by averaging extracted features per class to create prototypes, then using a 1-nearest neighbor classifier to label test examples [30]. With only 30-70 annotated ROIs per slide and just {1,2,4} slides per class, this approach can match or outperform trained AI models for non-small cell lung cancer (NSCLC) subtyping and renal cell carcinoma (RCC) subtyping [30].

Table 3: Research Reagent Solutions for Pathology Foundation Model Development

Research Reagent	Function	Example Implementation
Mass-100K Dataset	Pretraining corpus for visual foundation models	100M+ patches from 100K+ WSIs across 20 tissue types
Mass-340K Dataset	Multimodal pretraining corpus	335,645 WSIs with pathology reports and synthetic captions
DINOv2 Framework	Self-supervised learning algorithm	Knowledge distillation with no labels for UNI model
iBOT Framework	Self-supervised learning with masked image modeling	Used for TITAN pretraining with knowledge distillation
Vision Transformer (ViT)	Model architecture for feature extraction	Scalable transformer architecture used in UNI and TITAN
ABMIL Aggregator	Slide-level feature aggregation	Attention-based Multiple Instance Learning for WSI classification
OncoTree Classification	Evaluation framework for cancer subtyping	108-class cancer classification system for model assessment

The empirical investigation of scaling laws using the Mass-100K and Mass-340K datasets has yielded fundamental insights for pathology foundation model research. The relationship between model performance and scale follows predictable patterns: increasing both model capacity and pretraining data volume and diversity leads to substantial gains across diverse clinical tasks, with particularly dramatic improvements for rare diseases and few-shot learning scenarios.

These scaling laws have enabled the development of foundation models with versatile capabilities, from resolution-agnostic classification to few-shot prototyping and multimodal understanding. As the field advances, the continued systematic study of scaling relationships will guide resource allocation and architectural decisions, ultimately accelerating the development of more capable, efficient, and clinically useful AI systems for pathology.

Pathology Foundation Model Pretraining Workflow

The advent of large-scale datasets like Mass-100K (100,426 whole-slide images) and Mass-340K (335,645 whole-slide images) has catalyzed a paradigm shift in computational pathology, enabling the development of powerful foundation models such as UNI and TITAN [2] [1]. These datasets, characterized by their extensive scale and diversity across multiple organ types, stains, and scanner systems, provide the foundational substrate for training models capable of versatile downstream applications. However, a critical challenge persists: despite being trained on massive, diverse datasets, foundation models inherently encode non-biological technical artifacts alongside genuine morphological features, potentially limiting their reliability in real-world clinical deployment [13] [32].

Technical variations in histopathology—arising from differences in staining protocols, section thickness, scanner models, and imaging parameters—create systematic batch effects that can obscure true biological signals [33] [32]. Recent comprehensive evaluations of 20 publicly available pathology foundation models revealed that all models encoded medical center information, with more than half allowing better prediction of the medical center origin than the biological class of the tissue [13]. This susceptibility to technical confounding factors represents a significant barrier to clinical adoption, as models must demonstrate consistent performance across diverse healthcare settings and technical protocols. This technical guide examines the sources of these variations, quantitatively assesses their impact on model performance, and presents a comprehensive framework of mitigation strategies to ensure robust generalization in computational pathology.

In histopathology image analysis, batch effects represent systematic variations introduced by technical rather than biological factors. These variations can be categorized into pre-analytical, analytical, and post-analytical phases:

• Staining Variations: Differences in hematoxylin and eosin (H&E) staining protocols, including staining time, reagent batch, and pH levels, create significant color and intensity variations between samples [34] [32]. These differences can be quantified through color distribution analysis and affect feature extraction algorithms.
• Scanner Effects: Different whole-slide scanner models (e.g., Aperio, Philips, Hamamatsu) employ distinct optical components, color calibration protocols, and compression algorithms, resulting in variations in resolution, sharpness, and color representation [35]. One study demonstrated that variations in section thickness and staining time alone reduced AI-based prostate cancer grading performance by up to 8.6 percentage points in the event-ordered concordance index [33].
• Sectioning Artifacts: Variations in section thickness (typically 3-5μm) and tissue folding introduce optical distortions that affect subsequent image analysis [33].
• Focus and Resolution Issues: Blurring artifacts from improper focusing during scanning and resolution differences between scanning sessions create challenges for high-magnification analysis [34].

Table 1: Impact of Technical Variations on Model Performance

Variation Type	Performance Drop	Evaluation Metric	Study
Section Thickness	Up to 8.6%	EOC-Index	[33]
Staining Protocols	Significant reduction	AUROC	[33]
Scanner Differences	Medical center prediction accuracy 88-98%	Classification accuracy	[13]
Multi-site Effects	Performance disparities across institutions	Robustness Index	[13]

Quantifying the Impact on Foundation Models

The PathoROB benchmark study, which evaluated 20 foundation models across 28 biological classes from 34 medical centers, introduced a Robustness Index to quantify how well models handle inter-institutional variations [13]. This metric ranges from 0 (not robust) to 1 (robust) and measures whether biological features dominate over confounding technical features in the embedding space. The study revealed:

• Robustness scores across models varied from 0.463 to 0.877, with no model achieving full robustness [13]
• A strong correlation (ρ = 0.692, p = 0.004) existed between the number of training slides and robustness, indicating the importance of dataset scale [13]
• For more than half of the models, medical center prediction actually outperformed biological class prediction [13]

Mitigation Strategies: A Multi-Layered Framework

Data-Level Robustification Techniques

At the data level, several preprocessing techniques can mitigate technical variations before model training:

Stain Normalization standardizes color distributions across images using reference templates. Common algorithms include:

• Reinhard Normalization: Transforms image color distributions to match a reference image in LAB color space [13]
• Macenko Method: Utilizes principal component analysis to separate stain concentrations and normalize to a reference appearance [13]
• Credibility-Guided Color Adaptation: An advanced method that selectively adapts color channels based on credibility metrics to preserve biological signals [33]

Super-Resolution Techniques address resolution variations between scanners. Single-image super-resolution (SISR) technology based on deep learning reconstructs high-resolution images from lower-resolution inputs, enhancing clarity without the storage and speed penalties of traditional high-resolution scanning [36]. One study demonstrated that a super-resolution system could process an entire slide in 0.25 minutes using only 0.35GB storage, compared to 15 minutes and 0.5GB for conventional scanning [36].

Quality Control and Artifact Detection automated pipelines implement quality metrics like:

• Sharpness Evaluation: Using Laplacian variance to quantify focus quality [34]
• Contrast Assessment: Implementing Contrast-Limited Adaptive Histogram Equalization (CLAHE) to standardize contrast [34]
• Artifact Detection: Identifying and excluding regions with tissue folds, bubbles, or tears [34]

Diagram 1: Comprehensive robustification framework for pathology foundation models, integrating data-level and model-level techniques.

Model-Level Robustification Strategies

Beyond data preprocessing, several model architecture and training innovations enhance robustness:

Domain-Adversarial Training employs a dual-objective approach where the model learns feature representations that simultaneously maximize biological classification accuracy while minimizing the ability to discriminate between technical domains (e.g., different scanners or medical centers) [33] [13]. In the PathoROB benchmark, models incorporating domain-adversarial components demonstrated improved robustness metrics [13].

Multi-Site Training Strategies leverage the inherent diversity in large-scale datasets. The PLUTO-4 model, trained on 551,164 WSIs from over 50 institutions, exemplifies this approach, with explicit curation of data across scanner vendors (Aperio, Philips, Ventana, Hamamatsu) and stain types [35]. This intentional diversity during training creates more invariant representations.

Self-Supervised Learning (SSL) on Diverse Data frameworks like DINOv2, used in both UNI and PLUTO-4 models, learn robust representations by leveraging the natural variations present in large-scale datasets [35] [1]. The scaling laws observed in UNI demonstrate that increasing dataset size and diversity consistently improves robustness across tissue types and disease categories [1].

Table 2: Performance Comparison of Robustification Techniques in PathoROB Benchmark

Method Category	Specific Technique	Average Robustness Improvement	Key Limitation
Data Robustification (DR)	Reinhard Stain Normalization	+16.2%	Cannot eliminate entangled features
Representation Robustification (RR)	ComBat Batch Correction	+27.4%	Risk of removing biological signals
Combined Approach	DR + RR	Highest absolute robustness	Still incomplete correction
Domain-Adversarial Training	DANN	Varies by model architecture	Training complexity

Advanced Multi-Modal and Synthesis Approaches

Emerging techniques leverage additional data modalities and synthetic data generation:

Vision-Language Alignment, as implemented in the TITAN model, uses corresponding pathology reports and synthetic captions to ground visual representations in clinical language, creating more biologically relevant features less susceptible to technical variations [2]. Models with image/text training showed higher robustness than vision-only models in the PathoROB evaluation [13].

Synthetic Data Augmentation generates artificial technical variations during training, explicitly exposing the model to a broader spectrum of potential artifacts and teaching invariance to these factors [2]. The TITAN model utilized 423,122 synthetic captions generated from a multimodal generative AI copilot to enhance training diversity [2].

Credibility-Guided Adaptation employs confidence estimation to identify and potentially exclude samples with significant technical artifacts or uncertain predictions, preventing error propagation [33].

Experimental Protocols for Robustness Evaluation

Implementing the PathoROB Benchmark Framework

To systematically evaluate model robustness, researchers can implement a benchmark framework with the following protocol:

• Dataset Curation: Construct balanced datasets from multiple medical centers, ensuring equal representation of biological classes across centers. The PathoROB benchmark used four datasets from three public sources covering 28 biological classes from 34 medical centers [13].

• Embedding Extraction: Process images through the foundation model without fine-tuning to obtain feature embeddings [13].

• Robustness Metrics Calculation:

  • Robustness Index: For each reference sample, examine neighbors that are either Same biological/Other confounding (SO) or Other biological/Same confounding (OS) [13]
  • Average Performance Drop: Measure performance variation across medical centers [13]
  • Clustering Score: Quantify whether embeddings cluster by biological class rather than medical center [13]

• Bias Introduction Testing: Artificially introduce bias by adding more data from one hospital for specific classes to test how bias affects downstream performance [13].

Cross-Validation Across Technical Domains

When evaluating model performance, implement cross-validation strategies that explicitly test generalization across technical domains:

Diagram 2: Experimental workflow for cross-domain robustness evaluation using leave-one-scanner-out validation.

• Leave-One-Scanner-Out Validation: Train on data from multiple scanners and test on held-out scanner data
• Stain-Variant Testing: Explicitly test performance across different staining protocols
• Progressive Domain Shift Evaluation: Measure performance degradation with increasing technical differences between training and test sets

The Scientist's Toolkit: Essential Research Reagents and Solutions

Table 3: Research Reagent Solutions for Handling Technical Variation

Reagent/Solution	Function	Implementation Example
Reinhard Normalization	Standardizes color distributions across images	Preprocessing step in PathoROB benchmark [13]
Macenko Normalization	Separates stain vectors for normalization	Alternative to Reinhard method [13]
ComBat Batch Correction	Removes technical batch effects from features	Representation-level correction in PathoROB [13]
Domain-Adversarial Neural Network (DANN)	Learns domain-invariant features	Model-level robustification [13]
Contrast-Limited Adaptive Histogram Equalization (CLAHE)	Enhances local contrast while limiting noise	Quality control preprocessing [34]
Single-Image Super-Resolution (SISR)	Enhances image resolution using deep learning	Resolution standardization in digital pathology [36]
Laplacian Variance Filter	Quantifies image sharpness for quality control	Blur detection in quality assessment pipelines [34]
Stochastic Curriculum Learning (SCL)	Progressive difficulty training for super-resolution	Super-resolution model training [36]

Ensuring robust generalization against scanner variation and stain artifacts remains a critical challenge in computational pathology, despite the transformative potential of foundation models trained on massive datasets like Mass-100K and Mass-340K. The comprehensive evaluation of current foundation models reveals that while all encode technical artifacts to varying degrees, strategic interventions at both data and model levels can significantly enhance robustness [13].

The most promising approaches combine multiple strategies: diverse multi-center training data (as seen in PLUTO-4's 50+ institution dataset) [35], intentional robustification techniques (like domain-adversarial training and stain normalization) [33] [13], and systematic benchmarking using frameworks like PathoROB [13]. As the field progresses, vision-language models and synthetic data generation offer additional pathways to learn biologically relevant features that transcend technical variations [2].

For researchers and drug development professionals, adopting these robustification strategies and evaluation frameworks is essential for developing models that perform consistently across diverse clinical settings. This methodological rigor will accelerate the translation of computational pathology advancements from research tools to reliable clinical decision support systems that generalize across the technical heterogeneity inherent in real-world healthcare environments.

The emergence of large-scale, histopathology-based foundation models represents a paradigm shift in computational pathology, enabling robust artificial intelligence tools for disease diagnosis, prognosis, and biomarker discovery. Central to this advancement are the Mass-100K and Mass-340K datasets—massive, diverse collections of whole-slide images (WSIs) that serve as critical pretraining resources for developing general-purpose models in pathology. These datasets provide the foundational visual data necessary for training models that can recognize intricate morphological patterns across tissue types and disease states.

A significant challenge in computational pathology, however, has been bridging the semantic gap between visual morphological patterns and rich clinical context. While vision-only foundation models demonstrate strong performance on discriminative tasks, their utility remains constrained without integrated language capabilities essential for clinical workflows such as report generation, visual question answering, and cross-modal retrieval. This limitation is particularly pronounced in resource-limited clinical scenarios and for rare diseases, where annotated data is scarce.

The integration of synthetic data—specifically, algorithmically generated captions describing histopathology images—has emerged as a powerful methodology to overcome these limitations. By creating vast volumes of paired image-text data, synthetic captions enable vision-language alignment, enriching model training without the prohibitive cost and expertise required for manual annotation. This technical guide explores how generated captions are augmenting training and enhancing language capabilities for pathology foundation models, with specific focus on their application within the Mass-100K and Mass-340K dataset frameworks.

The Mass-100K and Mass-340K Datasets: Foundation for Pathology AI

The Mass-100K and Mass-340K datasets constitute pioneering large-scale resources specifically curated for pretraining pathology foundation models. Their development addressed a critical bottleneck in computational pathology: the lack of diverse, large-scale WSI collections necessary for training models that generalize across tissue types, disease states, and clinical scenarios.

Table 1: Composition and Key Characteristics of Mass-100K and Mass-340K Datasets

Characteristic	Mass-100K	Mass-340K
Total Whole-Slide Images	100,426+ WSIs [1]	335,645 WSIs [2]
Tissue Patches/ROIs	>100 million [1] [11]	Not explicitly quantified (ROI-based)
Organ Types	20 major tissue types [1]	20 organ types [2]
Data Sources	MGH, BWH, GTEx consortium [1]	Institutional dataset (Mass-340K) [2]
Textual Data	Not initially included	182,862 medical reports [2]
Synthetic Captions	Not applicable	423,122 generated captions [2]
Primary Model Applications	UNI (visual encoder) [1] [11]	TITAN (multimodal model) [2]

The Mass-100K dataset pioneered scaling laws in computational pathology, demonstrating that performance on downstream tasks improves with increased data diversity and volume. It contains over 100 million tissue patches extracted from more than 100,000 diagnostic H&E-stained WSIs across 20 major tissue types, sourced from Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), and the Genotype-Tissue Expression (GTEx) consortium [1]. This dataset enabled the training of UNI, a general-purpose self-supervised vision encoder that advances unsupervised representation learning at scale [1] [11].

Building upon this foundation, the Mass-340K dataset significantly expanded both visual and linguistic dimensions, incorporating 335,645 WSIs alongside 182,862 medical reports [2]. This expansion enabled not only larger-scale visual pretraining but also vision-language alignment through pathology reports and synthetic captions. The Mass-340K dataset directly supported the development of TITAN (Transformer-based pathology Image and Text Alignment Network), a multimodal whole-slide foundation model that leverages both natural and synthetic language data [2].

The strategic composition of these datasets across multiple organ types, stains, and scanner types ensures diversity has proven to be a critical factor in developing models that generalize well across various clinical tasks and settings [2] [1].

The Synthetic Data Pipeline: Generation and Integration

The generation and integration of synthetic captions within pathology foundation model training involves a sophisticated multi-stage pipeline that transforms visual representations into semantically rich textual descriptions. This process addresses the fundamental scarcity of manually annotated image-text pairs in histopathology, enabling effective vision-language pretraining.

Synthetic Caption Generation Methodology

The synthetic caption generation process for the Mass-340K dataset leveraged PathChat, a multimodal generative AI copilot for pathology specifically designed for histopathology images [2] [37]. This approach generated an impressive 423,122 synthetic fine-grained captions describing region-of-interest (ROI) crops of 8,192 × 8,192 pixels at 20× magnification [2].

The technical workflow involves several sophisticated components:

• Visual Feature Extraction: High-resolution ROI crops are processed through pretrained patch encoders (such as CONCH) to extract meaningful visual representations of histopathological structures [2].

• Multimodal Generation: PathChat, built upon a vision-language architecture, interprets these visual features and generates descriptive text capturing morphologic details, tissue structures, and potential pathological findings [37].

• Quality Assurance: While not explicitly detailed in the search results, successful implementation typically involves validation by pathology experts to ensure clinical relevance and accuracy of generated captions.

This synthetic data generation process effectively creates a large-scale dataset of paired image-text examples, which is crucial for training models to understand the relationship between visual patterns in histology and their textual descriptions.

Integration with Vision-Language Pretraining

The synthetic captions are integrated into foundation model training through a structured multi-stage pretraining paradigm, as exemplified by TITAN [2]:

• Stage 1: Vision-Only Unimodal Pretraining - The model undergoes self-supervised learning (using iBOT framework) on ROI crops from Mass-340K to learn fundamental visual representations of histopathology images without using any textual data.

• Stage 2: ROI-Level Cross-Modal Alignment - The model learns to align visual features with corresponding synthetic captions, enabling fine-grained understanding of morphology-text relationships.

• Stage 3: WSI-Level Cross-Modal Alignment - The model scales its alignment capabilities to entire whole-slide images, learning to associate comprehensive slide-level visual patterns with pathology reports.

This progressive approach leverages both the fine-grained detail of synthetic captions at the ROI level and the clinical context of natural reports at the WSI level, creating a model with robust multimodal capabilities.

Experimental Protocols and Validation Frameworks

Rigorous experimental validation is essential to demonstrate the value of synthetic data in enhancing pathology foundation models. The evaluation of models trained with synthetic captions encompasses diverse clinical tasks and learning scenarios.

Benchmark Tasks and Evaluation Metrics

Models augmented with synthetic captions are evaluated across multiple clinically relevant tasks to assess their generalizability and utility:

Table 2: Key Evaluation Tasks for Pathology Foundation Models with Synthetic Data

Task Category	Specific Tasks	Evaluation Metrics
Classification	Zero-shot classification, Few-shot learning, Cancer subtyping	Accuracy, Top-K accuracy, F1-score, AUROC
Retrieval	Rare cancer retrieval, Cross-modal retrieval	Recall@K, Precision, Mean Average Precision
Generation	Pathology report generation	BLEU score, ROUGE, Clinical accuracy
Prognosis	Survival prediction, Outcome prognosis	Concordance index, Hazard ratios

For the TITAN model, which leveraged synthetic captions, evaluations demonstrated superior performance across multiple machine learning settings, including linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval, and pathology report generation [2]. Notably, without any fine-tuning or requiring clinical labels, TITAN could extract general-purpose slide representations and generate pathology reports that generalized to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis [2].

Key Experimental Findings

The incorporation of synthetic captions has yielded several empirically validated benefits:

• Enhanced Zero-Shot and Few-Shot Learning: Models trained with synthetic captions demonstrate improved performance in low-data regimes, accurately classifying lesions and tissue types without task-specific training data [2].

• Improved Cross-Modal Retrieval: The alignment of visual and textual representations enables efficient retrieval of relevant histology images based on textual queries and vice versa, facilitating knowledge discovery and clinical decision support [2].

• Robust Performance on Rare Diseases: By exposing models to a wider variety of morphological descriptions through synthetic data, performance on rare cancer retrieval and classification significantly improves, addressing a critical challenge in computational pathology [2] [1].

• Effective Pathology Report Generation: Models can generate coherent, clinically relevant pathology reports from whole-slide images, potentially reducing pathologist workload and improving reporting consistency [2].

The Scientist's Toolkit: Essential Research Reagents

Implementing synthetic data approaches for pathology foundation models requires specific computational frameworks and data resources. The following table details key components of the experimental toolkit.

Table 3: Essential Research Reagents for Synthetic Data in Pathology AI

Resource	Type	Function	Example Implementation
PathChat	Multimodal Generative AI	Generates synthetic captions for histopathology ROIs	Used to create 423k fine-grained image-text pairs [2] [37]
CONCH	Vision-Language Foundation Model	Provides patch-level feature extraction and alignment	Base model for processing ROI crops before caption generation [11]
DINOv2	Self-Supervised Learning Algorithm	Enables visual representation learning without labels	Used in UNI pretraining on Mass-100K [1] [37]
iBOT	Self-Supervised Learning Framework	Combines masked image modeling and knowledge distillation	Used for vision-only pretraining stage of TITAN [2]
Mass-100K/-340K	Curated WSI Datasets	Provides diverse pretraining data across organs/diseases	Foundational datasets with 100K+ and 335K+ WSIs respectively [2] [1]

Implementation Workflow: From Data to Deployment

The complete implementation pipeline for leveraging synthetic captions in pathology foundation model development involves sequential stages from data preparation to model deployment, each with specific technical requirements and considerations.

Critical Implementation Considerations

Successful implementation of synthetic data approaches requires careful attention to several technical aspects:

• Data Diversity and Quality: The effectiveness of synthetic captions depends heavily on the diversity and quality of the original WSI dataset. Mass-100K and Mass-340K were specifically designed with diversity across organ types, stains, and scanners to maximize model generalizability [2] [1].

• Computational Optimization: Processing gigapixel WSIs requires specialized approaches to handle long input sequences. TITAN implemented techniques like attention with linear bias (ALiBi) for long-context extrapolation and used 512×512 pixel patches (instead of 256×256) to reduce sequence length while maintaining morphological context [2].

• Multi-Scale Representation Learning: Effective pathology foundation models must capture information at multiple scales—from cellular features to tissue architecture and whole-slide patterns. The integration of ROI-level synthetic captions and WSI-level reports enables this multi-scale understanding [2].

The integration of synthetic data, particularly generated captions, represents a transformative methodology in computational pathology foundation model development. By leveraging large-scale WSI datasets like Mass-100K and Mass-340K alongside AI-generated descriptive text, researchers can create models with enhanced language capabilities that generalize across diverse clinical scenarios.

The technical approaches detailed in this guide—from synthetic caption generation using tools like PathChat to multi-stage vision-language pretraining—demonstrate how synthetic data overcomes critical bottlenecks in pathology AI. The resulting models, such as TITAN, exhibit unprecedented capabilities in zero-shot learning, cross-modal retrieval, and pathology report generation, particularly valuable for resource-limited settings and rare diseases.

As the field advances, future directions may include more sophisticated generative models for caption production, integration with additional modalities such as genomic data, and standardized benchmarking across institutions. The continued development and ethical application of these approaches holds significant promise for enhancing diagnostic accuracy, prognostic insight, and ultimately patient care in anatomic pathology.

Benchmarking Success: Validation and Comparative Analysis of Models Built on Mass-Scale Datasets

The development of robust foundation models in computational pathology is critically dependent on large-scale, diverse datasets for pretraining. The Mass-100K and Mass-340K datasets represent two of the most comprehensive histopathology image collections developed for this purpose, serving as the foundational pillars for training general-purpose artificial intelligence (AI) models in anatomic pathology [1] [11]. These datasets enable the creation of models that can be adapted to numerous downstream clinical tasks without requiring extensive retraining, addressing a significant limitation in traditional computational pathology approaches that struggle with limited annotated data, especially for rare conditions.

The Mass-100K dataset forms the pretraining basis for the UNI model and consists of over 100 million tissue patches extracted from 100,426 diagnostic hematoxylin and eosin (H&E) stained whole slide images (WSIs) across 20 major tissue types [1]. This dataset was curated from multiple sources, including Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), and the Genotype-Tissue Expression (GTEx) consortium, ensuring diversity in tissue types, disease states, and processing protocols [1].

The larger Mass-340K dataset, used for training the TITAN model, expands significantly on this scale with 335,645 WSIs and 182,862 medical reports [2]. This dataset further increases diversity across organ types, stains, and scanner types, incorporating both visual and textual data for multimodal learning [2]. The strategic assembly of these datasets addresses the crucial need for data diversity over mere quantity, enabling the development of models that generalize across diverse clinical scenarios and tissue types.

Experimental Frameworks and Model Architectures

UNI: A General-Purpose Vision Encoder for Pathology

The UNI model employs a vision transformer (ViT-Large) architecture pretrained using the DINOv2 self-supervised learning framework on the Mass-100K dataset [1] [37]. This approach enables the model to learn rich, off-the-shelf visual representations without requiring labeled data during pretraining. The pretraining strategy leverages the scaling properties of vision transformers, where increased model size and data diversity directly translate to improved performance on downstream tasks [1].

Table 1: UNI Model Pretraining Scaling Performance on OncoTree Classification Tasks

Model Architecture	Pretraining Data	OT-43 Top-1 Accuracy	OT-108 Top-1 Accuracy	Performance Trend
ViT-Large	Mass-1K (1M images)	Baseline	Baseline	Reference
ViT-Large	Mass-22K (16M images)	+4.2%	+3.5%	Significant improvement (p<0.001)
ViT-Large	Mass-100K (100M+ images)	+3.7% additional	+3.0% additional	Continued improvement (p<0.001)

TITAN: A Multimodal Whole-Slide Foundation Model

The TITAN model introduces a more complex, multi-stage pretraining approach on the Mass-340K dataset, combining visual self-supervised learning with vision-language alignment [2]. The architecture is built on a Vision Transformer (ViT) designed to process entire WSIs by leveraging pre-extracted patch features from powerful histology patch encoders [2]. The pretraining consists of three distinct stages:

• Vision-only unimodal pretraining on region-of-interest (ROI) crops from Mass-340K using the iBOT framework [2]
• Cross-modal alignment with generated morphological descriptions at the ROI-level (423,122 image-caption pairs) [2]
• Cross-modal alignment at the WSI-level (182,862 WSI-report pairs) [2]

To handle the computational complexity of processing gigapixel WSIs, TITAN employs several innovative solutions. The model processes non-overlapping patches of 512×512 pixels at 20× magnification, extracts 768-dimensional features for each patch, and uses attention with linear bias (ALiBi) for long-context extrapolation during inference [2]. This approach enables the model to handle variable-length WSI sequences while preserving spatial relationships in the tissue microenvironment.

Figure 1: TITAN Multi-Stage Pretraining Workflow on Mass-340K Dataset

Comprehensive Evaluation Across Clinical Tasks

UNI Performance on 34 Diverse Clinical Tasks

The UNI model was rigorously evaluated across 34 representative computational pathology tasks of varying diagnostic difficulty [1]. These tasks were designed to assess the model's generalization capabilities across different tissue types, disease categories, and clinical applications. The evaluation framework encompassed multiple machine learning settings, including region-of-interest (ROI) level classification, segmentation, image retrieval, and slide-level weakly supervised learning [1].

Table 2: UNI Model Performance Across Select Clinical Tasks

Task Category	Specific Tasks Evaluated	Key Performance Metrics	Comparative Advantage
Cancer Subtyping	43-class OncoTree cancer type (OT-43), 108-class OncoTree code (OT-108)	Top-1, Top-3, Top-5 Accuracy, AUROC	Outperformed CTransPath and REMEDIS by wide margin
Rare Cancer Classification	90 rare cancer types per RARECARE/SEER	Weighted F1 Score, AUROC	Demonstrated few-shot learning capabilities
Biomarker Prediction	Molecular subtyping, IHC marker prediction	Accuracy, AUROC	Enabled biomarker screening from H&E alone
Diagnostic Tasks	Primary vs metastatic cancer, cancer grading	Balanced Accuracy, F1 Score	Generalized across tissue types
Specialized Assessment	Organ transplant rejection	Sensitivity, Specificity	Effective in non-oncology contexts

The evaluation on the large-scale OncoTree classification tasks (OT-43 and OT-108) is particularly noteworthy as it included 90 rare cancer types as defined by the RARECARE project and NCI-SEER program [1]. This comprehensive assessment demonstrated UNI's capability to handle the extensive diversity of cancer diagnoses encountered in real-world anatomic pathology practice, moving beyond binary classification tasks to more clinically relevant multi-class scenarios.

TITAN's Multimodal Capabilities and Zero-Shot Learning

TITAN was evaluated on diverse clinical tasks including slide-level classification, rare cancer retrieval, cross-modal retrieval, and pathology report generation [2]. The model demonstrated exceptional performance in few-shot and zero-shot learning scenarios, which is particularly valuable for rare diseases with limited training data [2]. Without any fine-tuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios [2].

The model's cross-modal capabilities enable novel applications such as:

• Text-guided slide retrieval: Finding histologically similar cases based on textual descriptions
• Zero-shot classification: Diagnosing conditions without task-specific training
• Report generation: Generating descriptive pathology reports from WSIs
• Rare disease identification: Leveraging morphological similarities across diseases

TITAN's performance in rare cancer retrieval is particularly significant, as it addresses a critical challenge in pathology practice where limited examples are available for training [2]. By leveraging both visual and language-based similarities, the model can identify morphologically similar cases even across different cancer types, providing valuable diagnostic references for pathologists facing diagnostically challenging cases.

Rare Cancer Retrieval Performance

The evaluation of rare cancer retrieval capabilities represents one of the most rigorous tests for pathology foundation models, addressing a fundamental challenge in clinical practice. Both UNI and TITAN demonstrated exceptional performance in this domain, though through different mechanistic approaches.

UNI established its rare cancer retrieval capabilities through the large-scale OncoTree classification task, which included 90 rare cancer types [1]. The model demonstrated that scaling laws observed in natural image domains similarly apply to computational pathology - as model size and pretraining data diversity increased, so did performance on rare cancer classification [1]. This capability is mediated through the learning of rich, general-purpose visual representations that capture subtle morphological patterns distinguishing rare cancer subtypes.

TITAN advanced rare cancer retrieval further by incorporating multimodal capabilities [2]. The model demonstrated proficiency in retrieving rare cancer cases based on both visual similarity and textual descriptions, enabling more flexible retrieval scenarios that align with clinical workflows. By aligning visual representations with pathological concepts described in reports and synthetic captions, TITAN can bridge the semantic gap between image morphology and diagnostic terminology, even for exceptionally rare conditions.

Figure 2: Rare Cancer Retrieval Using Multimodal Foundation Models

The Scientist's Toolkit: Essential Research Reagents

The development and evaluation of pathology foundation models require specialized computational frameworks and data resources. The following table outlines key components of the research infrastructure enabling this work.

Table 3: Essential Research Reagents for Pathology Foundation Model Development

Research Reagent	Function/Application	Implementation in Current Work
DINOv2 Framework	Self-supervised learning for visual representation learning	Used for UNI pretraining on Mass-100K dataset [1] [37]
iBOT Algorithm	Joint masked image modeling and knowledge distillation	Employed for TITAN vision-only pretraining stage [2]
Vision Transformer (ViT)	Backbone architecture for processing image sequences	Scaled as ViT-Base and ViT-Large variants [1]
Attention with Linear Biases (ALiBi)	Long-context extrapolation for variable-size WSIs	Extended to 2D for handling gigapixel whole slide images [2]
PathChat	Multimodal generative AI copilot for synthetic caption generation	Used to create 423,122 fine-grained ROI captions for TITAN training [2]
ABMIL Framework	Weakly supervised slide-level classification	Used for downstream task evaluation without full slide annotations [1]
OncoTree Classification System	Standardized cancer type taxonomy for evaluation	Provides hierarchical structure for 108 cancer type classification task [1]

Discussion and Implications

The rigorous evaluation of UNI and TITAN across 34+ clinical tasks and rare cancer retrieval scenarios demonstrates the transformative potential of foundation models in computational pathology. The Mass-100K and Mass-340K datasets have proven to be critical enablers of this progress, providing the scale and diversity necessary for training models that generalize across diverse clinical scenarios.

Several key principles emerge from this work. First, data diversity proves more critical than sheer volume - carefully curated datasets spanning multiple tissue types, disease states, and processing protocols enable more robust feature learning [37]. Second, multimodal pretraining unlocks unique capabilities for zero-shot learning and cross-modal retrieval that are unavailable to vision-only models [2]. Third, model scaling laws observed in natural image domains similarly apply to computational pathology, with increased model size and pretraining data consistently improving downstream performance [1].

The exceptional performance of these models on rare cancer tasks is particularly promising for clinical translation. By leveraging transfer learning and few-shot learning capabilities, foundation models can address the long-tail problem in medical AI, where rare conditions historically lack sufficient data for training conventional deep learning models [2] [1]. This capability has significant implications for democratizing access to specialized diagnostic expertise, particularly in resource-limited settings where subspecialty pathology expertise may be unavailable.

Future work in this domain will likely focus on integrating additional data modalities, including genomic profiles, spatial transcriptomics, and clinical outcomes, to create even more comprehensive foundation models. The continued expansion and diversification of pretraining datasets, along with innovations in model architecture and training algorithms, will further advance the capabilities of these models to serve as general-purpose assistants in pathology practice and research.

Foundation models are revolutionizing computational pathology by learning versatile representations from large volumes of unlabeled histopathology data. This technical analysis compares two next-generation foundation models, UNI and TITAN, against established predecessors CTransPath and REMEDIS, examining their architectural innovations, pretraining methodologies on the massive Mass-100K and Mass-340K datasets, and performance across diverse clinical tasks. Quantitative evaluations demonstrate that UNI and TITAN achieve state-of-the-art performance across classification, segmentation, and multimodal tasks while exhibiting superior data efficiency and generalization capabilities, particularly in rare cancer classification and low-data scenarios. These advancements highlight the critical importance of dataset scale and diversity in developing powerful foundation models for clinical applications.

The development of powerful foundation models in computational pathology has been constrained by the limited scale and diversity of available histopathology data. Most publicly available datasets, such as The Cancer Genome Atlas (TCGA), contain approximately 29,000 whole-slide images (WSIs) primarily focused on cancer histology, limiting model generalizability for real-world clinical applications [1]. To address this fundamental limitation, researchers have developed massive internal datasets that serve as the foundation for next-generation models.

Mass-100K Dataset

The Mass-100K dataset represents one of the largest histology slide collections created for self-supervised learning, comprising more than 100 million tissue patches from 100,426 diagnostic H&E WSIs across 20 major tissue types [1]. This dataset was curated from Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), and the Genotype-Tissue Expression (GTEx) consortium, providing extensive diversity in tissue morphology and disease states that enables learning robust, general-purpose representations.

Mass-340K Dataset

Building upon this effort, the Mass-340K dataset expands further to include 335,645 whole-slide images with corresponding 182,862 medical reports [2]. This dataset spans 20 organ types, different staining protocols (H&E, IHC), diverse tissue types, and various scanner platforms, significantly increasing the pretraining data diversity that has proven crucial for developing highly adaptable foundation models.

Model Architectures and Pretraining Methodologies

UNI: Universal Neural Interface for Pathology

UNI employs a Vision Transformer Large (ViT-L) architecture pretrained on the Mass-100K dataset using the DINOv2 self-supervised learning framework [1]. This approach enables the model to learn rich, off-the-shelf representations without requiring task-specific fine-tuning. A key innovation in UNI is its demonstration of scaling laws in computational pathology—performance consistently improves as both model size and pretraining data scale increase, mirroring trends observed in natural image foundation models.

TITAN: Transformer-based Pathology Image and Text Alignment Network

TITAN introduces a multimodal whole-slide foundation model pretrained on the Mass-340K dataset through a sophisticated three-stage process [2]:

• Vision-only unimodal pretraining on ROI crops using masked image modeling and knowledge distillation (iBOT framework)
• Cross-modal alignment with synthetic fine-grained morphological descriptions at the region-of-interest (ROI) level
• Slide-level vision-language alignment with clinical pathology reports

TITAN incorporates Attention with Linear Biases (ALiBi) for long-context extrapolation, enabling it to handle gigapixel whole-slide images with variable sizes and aspect ratios—a significant challenge in computational pathology.

Previous State-of-the-Art Models

CTransPath represents an earlier foundation model trained using a hybrid transformer-CNN architecture on TCGA and PAIP datasets [1] [38]. REMEDIS employs a self-supervised approach combining contrastive learning and supervised transfer learning, also pretrained primarily on TCGA data [1]. While these models demonstrated impressive performance, their training on smaller, less diverse datasets limited their generalization capabilities across diverse real-world clinical scenarios.

Experimental Framework and Benchmarking Methodology

Evaluation Tasks and Datasets

To ensure comprehensive comparison, researchers established rigorous benchmarking protocols encompassing diverse clinical tasks of varying diagnostic difficulty:

• Cancer subtyping: Evaluation across 43 cancer types (OT-43) and 108 OncoTree codes (OT-108), including 90 rare cancer types as defined by the RARECARE project [1]
• Pathomics tasks: Molecular biomarker prediction including EGFR, BRAF, and other mutations from histology images [38]
• Morphological assessment: Tissue classification, segmentation, and image retrieval tasks [39]
• Prognostication: Survival outcome prediction and risk stratification [39]
• Multimodal capabilities: Zero-shot classification, report generation, and cross-modal retrieval [2]

Experimental Protocol

For slide-level classification tasks, the standard weakly supervised multiple instance learning (MIL) framework was employed:

• Feature extraction: Tissue-containing patches from each WSI were processed using pretrained encoders to generate patch-level embeddings
• Aggregation: An attention-based MIL (ABMIL) algorithm aggregated patch embeddings into slide-level representations [1]
• Classification: A final classification layer generated predictions based on slide-level features
• Evaluation: Performance was assessed using top-K accuracy (K=1,3,5), weighted F1 score, and area under the receiver operating characteristic curve (AUROC)

ForUNI and TITAN, additional evaluation was performed in few-shot and zero-shot settings to assess data efficiency and generalization capabilities without task-specific fine-tuning.

Comparative Performance Analysis

Quantitative Performance Comparison

Table 1: Performance comparison on cancer subtyping tasks (OT-43 and OT-108)

Model	Pretraining Data	OT-43 Top-1 Accuracy	OT-108 Top-1 Accuracy	AUROC
UNI (ViT-L)	Mass-100K (100,426 WSIs)	Significantly higher than baselines (P < 0.001)	Significantly higher than baselines (P < 0.001)	State-of-the-art
TITAN	Mass-340K (335,645 WSIs)	Outperforms slide and ROI foundation models	Superior in few-shot and zero-shot settings	Excellent generalization
CTransPath	TCGA + PAIP	Lower than UNI (reference)	Lower than UNI (reference)	Competitive but inferior to UNI
REMEDIS	TCGA	Lower than UNI (reference)	Lower than UNI (reference)	Competitive but inferior to UNI
ResNet-50	ImageNet-1K	Substantially lower than all pathology foundation models	Substantially lower than all pathology foundation models	Lowest performance

Table 2: Performance across task types based on large-scale benchmarking [39]

Model	Morphology Tasks (AUROC)	Biomarker Tasks (AUROC)	Prognosis Tasks (AUROC)	Overall Average (AUROC)
CONCH (Vision-Language)	0.77	0.73	0.63	0.71
Virchow2 (Vision-only)	0.76	0.73	0.61	0.71
UNI	0.68 (reference)	0.68 (reference)	0.68 (reference)	0.68
Prov-GigaPath	0.69 (reference)	0.72 (reference)	0.69 (reference)	0.69
CTransPath	0.67 (reference)	0.67 (reference)	0.67 (reference)	0.67
REMEDIS	Not top performer	Not top performer	Not top performer	Below UNI

Independent large-scale benchmarking studies evaluating 19 foundation models across 31 clinical tasks with 6,818 patients and 9,528 slides revealed that while UNI performs strongly, vision-language models like CONCH and very large vision-only models like Virchow2 currently achieve the highest overall performance [39]. This suggests that both scale and multimodal training contribute to superior representation learning.

Data Efficiency and Few-Shot Learning

UNI demonstrates remarkable data efficiency, achieving strong performance with limited labeled examples. When pretraining UNI on subsets of Mass-100K, performance increased monotonically with data scale: +4.2% top-1 accuracy on OT-43 and +3.5% on OT-108 when scaling from Mass-1K to Mass-22K, with further improvements of +3.7% and +3.0% respectively when scaling to the full Mass-100K [1].

TITAN excels in few-shot and zero-shot learning scenarios, particularly for rare cancer retrieval and cross-modal search tasks. Without any fine-tuning or clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios [2].

Specialized Capabilities

Resolution-agnostic classification: UNI demonstrates the novel capability of classifying tissue types irrespective of input image resolution, a valuable property for handling diverse slide scanning protocols [1].

Multimodal reasoning: TITAN enables cross-modal retrieval between histology slides and clinical reports, plus generative capabilities for pathology report generation [2].

Rare cancer classification: Both UNI and TITAN show particularly strong performance on rare cancer types, addressing a critical challenge in clinical practice where limited training data is available [2] [1].

Research Reagent Solutions

Table 3: Essential research reagents and computational resources for pathology foundation model development

Resource	Specifications	Function in Research
Whole-Slide Images	High-resolution (≥ 100,000 WSIs); Multiple scanner types; H&E, IHC, and special stains	Foundation model pretraining; Benchmark evaluation; Generalization testing
Patch Encoders	CONCH, PLUTO-4, or other pretrained models; 768-1024 dimensional embeddings	Feature extraction from image patches; Slide representation building
Computational Infrastructure	High-memory GPU clusters (e.g., NVIDIA A100/H100); Multi-node training capability	Handling long sequences in WSIs; Transformer model training
Multiple Instance Learning Framework	Attention-based MIL (ABMIL); Transformer aggregators	Slide-level prediction from patch embeddings; Weakly supervised learning
Multimodal Data Pairs	Image-text pairs (clinical reports, synthetic captions); ≥ 100,000 pairs	Vision-language pretraining; Cross-modal alignment
Benchmarking Suites	Multi-task evaluation (classification, segmentation, retrieval); Multiple cancer types	Standardized model comparison; Clinical relevance assessment

The comparative analysis demonstrates that UNI and TITAN represent significant advancements over previous state-of-the-art models like CTransPath and REMEDIS, largely attributable to their training on the massive Mass-100K and Mass-340K datasets. The scale and diversity of these datasets enable learning more robust and generalizable representations that transfer effectively across diverse clinical tasks, particularly in challenging low-data and rare disease scenarios.

While architectural innovations contribute to these improvements, the data scaling laws observed with UNI confirm that dataset size and diversity are pivotal factors in foundation model performance. The emergence of multimodal capabilities in TITAN further expands the potential applications in clinical workflows, enabling more natural interaction between pathologists and AI systems.

These advancements highlight a promising trajectory for computational pathology, where foundation models trained on massive, diverse datasets will continue to enhance diagnostic accuracy, biomarker discovery, and personalized treatment planning. Future work should focus on expanding multimodal reasoning, improving interpretability, and validating these models in prospective clinical settings.

The Mass-340K dataset represents a pivotal advancement in computational pathology, serving as a large-scale pretraining resource for developing powerful foundation models. Formally known as the "Mass-340K" internal dataset, this collection comprises 335,645 whole-slide images (WSIs) paired with 182,862 medical reports, creating a substantial multimodal resource for AI development [2]. The dataset's significance stems from its extensive scale and diversity, distributed across 20 different organ types, various stains, diverse tissue types, and images scanned with different scanner types [2]. This diversity has proven to be a critical factor in developing robust patch encoders that generalize well across multiple clinical scenarios.

Within the broader thesis on pathology foundation models, Mass-340K addresses a fundamental constraint in the field: the limited availability of clinical data for disease-specific cohorts, particularly for rare conditions [2]. Prior to the development of such large-scale datasets, translating the capabilities of patch-based foundation models to address patient and slide-level clinical challenges remained complex due to the immense scale of gigapixel WSIs and small patient cohort sizes in real-world evidence [2]. The Mass-340K dataset directly mitigates these limitations by providing the volume and variety necessary to train models like TITAN (Transformer-based pathology Image and Text Alignment Network), enabling breakthroughs in few-shot and zero-shot learning applications in computational pathology.

Core Architecture and Pretraining Methodology

TITAN: A Multimodal Whole-Slide Foundation Model

The TITAN framework represents a significant architectural innovation designed explicitly to leverage the Mass-340K dataset. Unlike previous approaches that focused on region-of-interest (ROI) encodings, TITAN introduces a scalable method for WSI-level encoding through a three-stage pretraining paradigm [2]:

Stage 1: Vision-Only Unimodal Pretraining The cornerstone of TITAN involves emulating patch encoder design at the slide level. Rather than using tokens from partitioned image patches, the slide encoder processes a sequence of patch features encoded by powerful histology patch encoders [2]. All pretraining occurs in the embedding space based on pre-extracted patch features, with the patch encoder functioning as the 'patch embedding layer' in a conventional Vision Transformer (ViT). To handle computational complexity from long input sequences, TITAN constructs the input embedding space by dividing each WSI into non-overlapping patches of 512 × 512 pixels at 20× magnification, followed by extraction of 768-dimensional features for each patch [2]. The model employs attention with linear bias (ALiBi) for long-context extrapolation at inference time, where the linear bias is based on the relative Euclidean distance between features in the feature grid [2].

Stage 2: Cross-Modal Alignment with Synthetic Captions To equip the model with fine-grained language capabilities, TITAN undergoes cross-modal alignment using 423,122 synthetic fine-grained ROI captions generated using PathChat, a multimodal generative AI copilot for pathology [2]. This stage enables the model to understand detailed morphological descriptions at the region-of-interest level.

Stage 3: Cross-Modal Alignment at WSI-Level The final stage involves cross-modal alignment of entire WSIs with their corresponding clinical reports, using 183,000 pairs of WSIs and clinical reports [2]. This stage ensures the model can operate at the appropriate clinical abstraction level for slide-level diagnoses and prognoses.

PathPT: Enhancing Few-Shot Learning for Rare Cancers

While TITAN provides a robust foundation model, the PathPT framework addresses specific challenges in few-shot learning for rare cancer subtyping. PathPT introduces three core innovations that enhance few-shot performance [40]:

• Spatially-Aware Visual Aggregation: Employs a lightweight aggregator that explicitly models short- and long-range dependencies across tissue regions, capturing complex morphological patterns critical for rare subtype diagnosis.

• Task-Adaptive Prompt Tuning: Replaces static, handcrafted language prompts with learnable textual tokens optimized end-to-end to align with histopathological semantics, thereby preserving the prior knowledge of existing vision-language models.

• Tile-Level Supervision from Slide-Level Labels: Leverages the zero-shot grounding ability of vision-language foundation models to transform weak slide-level annotations into fine-grained tile-level pseudo-labels, enabling precise spatial learning.

Experimental Protocols and Performance Benchmarks

Zero-Shot Classification Performance

The zero-shot capabilities of foundation models pretrained on Mass-340K were rigorously evaluated across multiple classification tasks. In zero-shot transfer, models classify images without task-specific fine-tuning by matching image features with text prompts in the shared embedding space [27]. CONCH, another foundation model demonstrating the utility of large-scale pretraining, was evaluated on slide-level classification tasks including TCGA BRCA (invasive breast carcinoma subtyping), TCGA NSCLC (non-small-cell lung cancer subtyping), TCGA RCC (renal cell carcinoma subtyping), and Dartmouth Hitchcock Medical Center (DHMC) LUAD (lung adenocarcinoma histologic pattern classification) [27].

Table 1: Zero-Shot Classification Performance on Slide-Level Benchmarks

Task/Dataset	Model	Performance Metric	Result	Baseline Comparison
TCGA NSCLC Subtyping	CONCH	Balanced Accuracy	90.7%	+12.0% vs. PLIP [27]
TCGA RCC Subtyping	CONCH	Balanced Accuracy	90.2%	+9.8% vs. PLIP [27]
TCGA BRCA Subtyping	CONCH	Balanced Accuracy	91.3%	~35% improvement vs. baselines [27]
DHMC LUAD Pattern Classification	CONCH	Cohen's κ	0.200	+0.12 vs. PLIP [27]

For WSI-level zero-shot classification, the MI-Zero approach divides a WSI into smaller tiles and aggregates individual tile-level scores into a slide-level prediction [27]. This method also generates heatmaps visualizing cosine-similarity scores between each tile and text prompts, providing interpretable visualizations of model reasoning [27].

Few-Shot Learning for Rare Cancer Subtyping

Comprehensive benchmarks evaluated few-shot learning capabilities on rare cancer subtyping using eight rare cancer datasets (four adult and four pediatric) spanning 56 subtypes and 2,910 WSIs [40]. Experiments were conducted under 1-shot, 5-shot, and 10-shot settings, repeated 10 times to account for variance [40]. The evaluation compared PathPT against established multi-instance learning (MIL) frameworks including ABMIL, CLAM, TransMIL, and DGRMIL using features extracted from vision-language models (PLIP, CONCH, MUSK, and KEEP) [40].

Table 2: Few-Shot Performance on Rare Cancer Subtyping (EBRAINS Dataset)

Method	Backbone Model	1-Shot Accuracy	5-Shot Accuracy	10-Shot Accuracy	Improvement over Zero-Shot
PathPT	KEEP	0.512	0.621	0.679	+0.271 absolute gain [40]
TransMIL	KEEP	0.441	0.538	0.592	-
DGRMIL	KEEP	0.433	0.529	0.583	-
CLAM	KEEP	0.402	0.501	0.551	-
ABMIL	KEEP	0.395	0.492	0.539	-
Zero-Shot Baseline	KEEP	-	-	0.408	Reference [40]

Notably, PathPT consistently delivered superior performance, achieving substantial gains in accuracy and interpretability across all few-shot settings [40]. With KEEP as the backbone, PathPT achieved 0.679 balanced accuracy on the EBRAINS dataset (30 subtypes, 10-shot), outperforming all MIL baselines [40]. The framework also demonstrated significant improvements in tumor region segmentation, even in the challenging 1-shot setting, confirming its ability to leverage minimal supervision for precise spatial localization [40].

Cross-Modal Retrieval and Report Generation

Beyond classification, TITAN exhibits strong performance in cross-modal retrieval tasks, enabling searches between histology slides and clinical reports [2]. This capability allows pathologists to retrieve similar cases based on either image content or textual descriptions, particularly valuable for rare disease diagnosis. Additionally, the model can generate pathology reports from whole-slide images, demonstrating its understanding of the complex relationship between visual morphological patterns and clinical documentation [2].

Table 3: Key Research Reagents and Computational Resources

Resource/Reagent	Type	Function in Research	Specifications/Alternatives
Mass-340K Dataset	Data Resource	Primary pretraining dataset for pathology foundation models	335,645 WSIs, 182,862 reports, 20 organ types [2]
CONCH	Foundation Model	Visual-language foundation model for multimodal pathology tasks	Pretrained on 1.17M image-text pairs [27]
TITAN	Foundation Model	Multimodal whole-slide foundation model	Three-stage pretraining; handles 8,192 × 8,192 pixel ROIs [2]
PathPT	Framework	Few-shot prompt tuning for rare cancer subtyping	Enables tile-level supervision from slide-level labels [40]
Synthetic Captions	Data Resource	Fine-grained morphological descriptions for ROI-level alignment	423,122 captions generated via PathChat [2]
Vision-Language Models	Algorithmic Resource	Base models for feature extraction and cross-modal alignment	PLIP, CONCH, MUSK, KEEP [40]
Multi-Instance Learning Frameworks	Algorithmic Resource	Baselines for WSI classification with weak supervision	ABMIL, CLAM, TransMIL, DGRMIL [40]

The Mass-340K dataset has fundamentally advanced pathology foundation model research by enabling the development of models with exceptional few-shot and zero-shot learning capabilities. Through architectures like TITAN and methodologies like PathPT, researchers can now address the critical challenge of data scarcity, particularly for rare diseases and low-resource clinical settings. The quantitative results demonstrate that properly pretrained foundation models achieve remarkable performance in zero-shot classification and maintain strong accuracy in few-shot scenarios, outperforming traditional supervised approaches. As these models continue to evolve, they hold significant promise for democratizing access to expert-level pathological diagnosis, especially in underserved regions and for rare cancer subtypes where clinical expertise is limited.

The emergence of large-scale, self-supervised foundation models represents a paradigm shift in computational pathology, enabling artificial intelligence systems to learn transferable representations from vast repositories of unannotated data. Central to this advancement are the Mass-100K and Mass-340K datasets, which provide the unprecedented scale and diversity necessary for pretraining general-purpose models that transcend traditional classification tasks. These datasets facilitate the development of pathology foundation models (PFMs) capable of sophisticated multimodal understanding, including cross-modal retrieval between histology images and clinical text, and the generation of diagnostic pathology reports. The Mass-100K dataset serves as the pretraining foundation for models like UNI, comprising over 100 million tissue patches from more than 100,000 diagnostic hematoxylin and eosin (H&E)-stained whole-slide images (WSIs) across 20 major tissue types [1]. The expanded Mass-340K dataset, consisting of 335,645 WSIs, enables the training of more advanced multimodal architectures like TITAN (Transformer-based pathology Image and Text Alignment Network) [2]. These datasets provide the critical mass of data required to overcome the limitations of previous approaches constrained by small, annotated cohorts, particularly for rare diseases and complex clinical scenarios where training data is inherently limited.

Within the multiple instance learning (MIL) framework that dominates computational pathology, PFMs significantly enhance both the feature extractor and aggregator components [31]. Conventional approaches typically utilized networks pretrained on natural images (e.g., ImageNet), which struggled to capture pathology-specific characteristics like minimal color variation, rotation-agnosticism, and hierarchical tissue organization [16]. The Mass-100K and Mass-340K datasets address this fundamental limitation by providing massive-scale histopathology-specific data for self-supervised learning, enabling models to learn morphological patterns directly from tissue samples without the need for costly manual annotations. This pretraining paradigm empowers foundation models to excel not only in traditional classification tasks but also in more complex applications like cross-modal retrieval and report generation, which require a deeper semantic understanding of both visual morphological patterns and their corresponding clinical descriptions.

Core Methodologies: Experimental Frameworks for Multimodal Validation

Model Architectures and Pretraining Strategies

The validation of cross-modal capabilities and report generation requires specialized model architectures trained using innovative methodologies. The TITAN model exemplifies this approach through a three-stage pretraining strategy that progressively builds multimodal understanding [2]:

• Stage 1 - Vision-only Pretraining: The model undergoes self-supervised learning using the iBOT framework on 335,645 WSIs from the Mass-340K dataset, learning to encode histopathology regions of interest (ROIs) into versatile visual representations.
• Stage 2 - ROI-level Vision-Language Alignment: The visual encoder is aligned with fine-grained morphological descriptions using 423,122 synthetic captions generated by PathChat, a multimodal generative AI copilot for pathology.
• Stage 3 - WSI-level Vision-Language Alignment: The model learns to associate entire whole-slide images with their corresponding pathology reports using 182,862 medical report pairs.

A critical innovation in TITAN is its approach to handling the computational challenges of gigapixel WSIs. Rather than processing raw images directly, TITAN operates on pre-extracted patch features arranged in a two-dimensional feature grid that preserves spatial relationships [2]. The model uses a Vision Transformer architecture with attention with linear bias (ALiBi) to enable long-context extrapolation at inference time, allowing it to handle variable-sized WSIs while maintaining understanding of tissue microenvironment context.

The CONCH model represents another approach to multimodal foundation models, trained on 1.17 million histopathology image-text pairs using iBOT and CoCa (Contrastive Captioner) objectives [11] [41]. This training enables both image and text understanding capabilities, allowing pathologists to interact with the model to search for morphologies of interest. Unlike vision-only models, CONCH learns a shared embedding space where images and text can be directly compared, enabling cross-modal retrieval tasks without task-specific fine-tuning.

Evaluation Metrics and Benchmark Tasks

Rigorous validation of cross-modal capabilities requires specialized evaluation protocols beyond standard classification metrics. The experimental framework for models like TITAN and CONCH encompasses multiple task types:

• Zero-shot Classification: Evaluating the model's ability to recognize disease categories without task-specific training by leveraging natural language descriptions.
• Cross-modal Retrieval: Measuring retrieval accuracy between images and text queries, including slide-to-report and report-to-slide retrieval tasks.
• Pathology Report Generation: Assessing the quality and clinical accuracy of generated reports for given whole-slide images.
• Rare Cancer Retrieval: Testing performance on rare disease categories with limited examples, simulating real-world clinical challenges.

For retrieval tasks, standard information retrieval metrics are employed, including recall@K (proportion of relevant items found in the top K results) and mean average precision (mAP). For report generation, both quantitative natural language processing metrics (e.g., BLEU, ROUGE) and clinical accuracy assessments by pathologists are utilized to ensure generated reports contain morphologically accurate and clinically relevant information.

Table 1: Evaluation Metrics for Multimodal Pathology Tasks

Task Category	Primary Metrics	Secondary Metrics	Clinical Relevance
Cross-modal Retrieval	Recall@K, Mean Average Precision	Median Rank, Mean Reciprocal Rank	Diagnostic efficiency, case similarity search
Report Generation	BLEU, ROUGE Scores	Clinical Accuracy (Pathologist Evaluation)	Diagnostic reporting quality, workflow automation
Zero-shot Classification	Accuracy, F1-Score	Area Under ROC Curve	Generalization to rare diseases, novel categories
Rare Cancer Retrieval	Rare Class Recall@K	Failure Analysis	Diagnostic support for challenging cases

Quantitative Results: Performance Benchmarks for Multimodal Capabilities

Cross-Modal Retrieval Performance

The cross-modal retrieval capabilities of pathology foundation models represent a significant advancement in clinical utility. TITAN demonstrates exceptional performance in slide-to-report and report-to-slide retrieval tasks, effectively bridging the semantic gap between visual morphological patterns and their textual descriptions in clinical reports [2]. In quantitative evaluations, TITAN outperformed both region-of-interest (ROI) and slide-level foundation models across multiple retrieval benchmarks, particularly excelling in rare disease retrieval scenarios where limited examples are available for training. This capability has profound implications for clinical practice, enabling pathologists to retrieve similar cases based on either image content or descriptive text, facilitating consultation and decision-making for challenging diagnoses.

The CONCH model similarly demonstrates strong cross-modal alignment, enabling content-based image retrieval using text queries and vice versa [11]. This functionality allows pathologists to search for morphologies of interest across vast histopathology archives without relying solely on manual annotations or structured diagnostic codes. In comprehensive evaluations across 14 clinically relevant tasks, CONCH outperformed standard models in cross-modal retrieval accuracy, demonstrating the effectiveness of vision-language pretraining on histopathology data.

Table 2: Cross-Modal Retrieval Performance Across Pathology Foundation Models

Model	Training Data	Retrieval Task	Performance Benchmark	Key Advantage
TITAN	335,645 WSIs + 423K synthetic captions + 183K reports	Slide-Report Cross-Retrieval	Outperforms ROI/slide foundations models, especially on rare diseases	Strong generalization to resource-limited scenarios
CONCH	1.17M image-text pairs	Text-to-Image and Image-to-Text Retrieval	Superior to standard models across 14 clinical tasks	Enables semantic search for morphologies of interest
PLIP	Web-scale pathology image-text pairs	Image-Text Matching	Improved retrieval accuracy over non-multimodal approaches	Demonstrates web-scale pretraining potential

Pathology Report Generation Quality

The ability to generate coherent, clinically accurate pathology reports represents one of the most advanced capabilities of multimodal pathology foundation models. TITAN demonstrates proficiency in generating diagnostic reports that capture relevant morphological findings and their clinical interpretations [2]. Through quantitative evaluation and clinical validation, generated reports show strong alignment with ground truth reports in terms of morphological descriptions, diagnostic statements, and clinical implications. The model leverages its vision-language pretraining to translate visual patterns in tissue samples into semantically appropriate textual descriptions, effectively acting as an automated assistant for pathology reporting.

A critical advantage of TITAN's report generation capability is its strong performance in resource-limited clinical scenarios, including rare disease contexts where limited examples are available for training [2]. This suggests that the model learns generalizable concepts of histopathology morphology and its relationship to diagnostic language, rather than merely memorizing common report templates. The incorporation of synthetic captions generated by PathChat during pretraining further enhances the model's ability to generate fine-grained morphological descriptions, highlighting the potential of combining human expertise with AI-generated content for training multimodal systems.

Technical Implementation: Workflows and Research Reagents

Experimental Workflows and Signaling Pathways

The experimental workflow for validating cross-modal retrieval and report generation capabilities follows a structured pipeline from data preparation through model evaluation. The key stages include data curation and preprocessing, feature extraction, model pretraining, task-specific evaluation, and clinical validation. The following diagram illustrates the comprehensive validation workflow for multimodal pathology foundation models:

Figure 1: Multimodal Pathology Foundation Model Validation Workflow

The core architecture of multimodal pathology models like TITAN employs a transformer-based design with specialized components for handling whole-slide images and text sequences in a unified framework. The model processes pre-extracted patch features from WSIs while simultaneously encoding textual descriptions, learning aligned representations through contrastive learning objectives. The following diagram illustrates the architectural components and their relationships in the TITAN model:

Figure 2: TITAN Model Architecture for Vision-Language Alignment

Research Reagent Solutions for Multimodal Pathology

Implementing and validating cross-modal retrieval and report generation capabilities requires a comprehensive suite of research reagents and computational resources. The following table details essential components derived from the Mass-100K and Mass-340K datasets and associated models:

Table 3: Essential Research Reagents for Multimodal Pathology Research

Research Reagent	Specifications	Function in Experimental Workflow
Mass-100K Dataset	100,426 WSIs, 100M+ patches, 20 tissue types	Vision-only pretraining foundation for feature learning
Mass-340K Dataset	335,645 WSIs, 182,862 reports, 20 organs	Multimodal pretraining with clinical context
Synthetic Captions (PathChat)	423,122 ROI-caption pairs	Fine-grained vision-language alignment at ROI level
CONCHv1.5 Patch Encoder	768-dimensional features, 512×512 patches	Feature extraction from histopathology patches
TITAN Model Architecture	Transformer with ALiBi, 48.5M parameters	Whole-slide encoding with long-context capability
UNI Foundation Model	ViT-Large, 307M parameters, DINOv2 pretraining	Baseline for vision-only slide representations
iBOT Pretraining Framework	Masked image modeling + knowledge distillation	Self-supervised learning for visual representations

The Mass-100K and Mass-340K datasets have fundamentally transformed the landscape of computational pathology research by enabling the development of foundation models with sophisticated multimodal capabilities. Through rigorous validation methodologies, models like TITAN and CONCH demonstrate that cross-modal retrieval and pathology report generation are not only feasible but can achieve clinically relevant performance levels, particularly for challenging scenarios like rare disease diagnosis. These advancements highlight the critical importance of large-scale, diverse datasets in moving beyond simple classification tasks toward more comprehensive AI-assisted pathology workflows.

Future research directions in multimodal pathology foundation models will likely focus on several key areas: (1) scaling to even larger datasets encompassing broader disease spectra and imaging modalities; (2) improving fine-grained understanding of tumor microenvironment and spatial relationships; (3) enhancing clinical utility through interactive systems that support pathologist workflows; and (4) addressing technical challenges in computational efficiency and model interpretability. As these models continue to evolve, they hold the potential to significantly augment pathological practice, providing powerful tools for diagnosis, prognosis, and therapeutic response prediction across a wide spectrum of diseases.

Conclusion

The Mass-100K and Mass-340K datasets represent a pivotal advancement in computational pathology, serving as the bedrock for powerful foundation models like UNI and TITAN. Their unprecedented scale and diversity have proven essential for developing AI that generalizes across a wide spectrum of diagnostically challenging tasks, particularly in low-data scenarios and for rare cancers. The success of these models, validated through extensive benchmarking, underscores a fundamental shift from building brittle, task-specific tools toward creating versatile, robust foundation models. Future directions will likely focus on deeper integration of multi-modal data—including spatial omics and detailed knowledge graphs—to further enhance clinical interpretability and predictive power. For researchers and drug developers, these datasets and the models they enable are dramatically accelerating the path from histological data to actionable insights, ultimately promising more precise diagnostics and personalized therapeutic strategies.

References

• 1. Towards a General-Purpose Foundation Model for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11403354/]
• 2. A multimodal whole-slide foundation model for pathology [https://www.nature.com/articles/s41591-025-03982-3]
• 3. mahmoodlab/UNI: Pathology Foundation Model [https://github.com/mahmoodlab/UNI]
• 4. Validation of histopathology foundation models through ... [https://www.nature.com/articles/s41598-025-88545-9]
• 5. A Survey on Computational Pathology Foundation Models [https://arxiv.org/html/2501.15724v1]
• 6. Recommendations on compiling test datasets for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9708586/]
• 7. Human BioMolecular Atlas Program (HuBMAP): 3D Human ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11978508/]
• 8. Comprehensive Benchmark Dataset for Pathological ... [https://www.nature.com/articles/s41597-025-05586-5]
• 9. Introducing Protege Evaluation Datasets and Benchmarks ... [https://www.withprotege.ai/news/introducing-protege-evaluation-datasets-and-benchmarks-for-healthcare-ai]
• 10. Deep Learning as a Potential Solution to Transform ... [https://www.pathai.com/blog/deep-learning-as-a-potential-solution-to-transform-preclinical-toxicology-and-accelerate-drug-development]
• 11. Mass General Brigham Researchers Develop AI ... [https://www.massgeneralbrigham.org/en/about/newsroom/press-releases/mass-general-brigham-researchers-develop-ai-foundation-models-to-advance-pathology]
• 12. Evaluating Vision and Pathology Foundation Models for ... [https://www.medrxiv.org/content/10.1101/2025.05.08.25327250v1.full-text]
• 13. Towards Robust Foundation Models for Digital Pathology [https://www.aignostics.com/blog/towards-robust-foundation-models-for-digital-pathology]
• 14. Multi-modal image analysis for large-scale cancer tissue ... [https://www.sciencedirect.com/science/article/pii/S2667237525002061]
• 15. Tissue concepts: Supervised foundation models in ... [https://www.sciencedirect.com/science/article/pii/S0010482524017062]
• 16. A Survey of Pathology Foundation Model [https://arxiv.org/html/2504.04045v1]
• 17. 病理AI领域的七大基础模型｜顶刊速递·24-11-11 [https://blog.csdn.net/m0_59164304/article/details/143722755]
• 18. MahmoodLab/UNI [https://huggingface.co/MahmoodLab/UNI]
• 19. DINOv2: Self-supervised Learning Model Explained [https://encord.com/blog/dinov2-self-supervised-learning-explained/]
• 20. [2304.07193] DINOv2: Learning Robust Visual Features ... [https://arxiv.org/abs/2304.07193]
• 21. Benchmarking Pathology Foundation Models: Adaptation ... [https://arxiv.org/html/2410.16038v1]
• 22. PyTorch code and models for the DINOv2 self-supervised ... [https://github.com/facebookresearch/dinov2]
• 23. Pathology Foundation Models - PMC - PubMed Central [https://pmc.ncbi.nlm.nih.gov/articles/PMC11799676/]
• 24. Unlocking the Promise of Foundation Models in Pathology ... [https://proscia.com/unlocking-the-promise-of-foundation-models-in-pathology-for-ai-driven-drug-discovery-development/]
• 25. 100000 histological images of human colorectal cancer ... [https://zenodo.org/records/1214456]
• 26. MahmoodLab/TITAN [https://huggingface.co/MahmoodLab/TITAN]
• 27. A visual-language foundation model for computational ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11384335/]
• 28. Foundation Models For Pathology AI Development At Your ... [https://proscia.com/foundation-models-for-digital-pathology-at-your-fingertips/]
• 29. What AI can tell us about microscope slides – fast.ai [https://www.fast.ai/posts/2025-02-03-cpath-foundation/]
• 30. Building a Visual-Centric Foundation Model for ... [https://www.linkedin.com/pulse/building-visual-centric-foundation-model-pathology-richard-j-chen]
• 31. A Survey of Pathology Foundation Model [https://arxiv.org/html/2504.04045v2]
• 32. Understanding and Mitigating Batch Effects in Histopathology [https://www.trillium.de/en/journals/trillium-pathology/archive/2025/tp-1/2025/computational-pathology/image-analysis-understanding-and-mitigating-batch-effects-in-histopathology.html]
• 33. A systematic analysis of the impact of data variation on AI- ... [https://www.sciencedirect.com/science/article/pii/S136184152500430X]
• 34. Deep learning for digital pathology: A critical overview of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12593639/]
• 35. PLUTO-4: Frontier Pathology Foundation Models [https://arxiv.org/html/2511.02826v1]
• 36. Feasibility study of single-image super-resolution scanning ... [https://www.frontiersin.org/journals/medicine/articles/10.3389/fmed.2025.1550512/full]
• 37. Building foundation models for understanding human ... [https://ai.meta.com/blog/mahmood-lab-human-pathology-dinov2/]
• 38. A whole-slide foundation model for digital pathology from ... [https://www.nature.com/articles/s41586-024-07441-w]
• 39. Benchmarking foundation models as feature extractors for ... [https://www.nature.com/articles/s41551-025-01516-3]
• 40. Boosting Pathology Foundation Models via Few-shot ... [https://arxiv.org/html/2508.15904]
• 41. BearCleverProud/AwesomeWSI [https://github.com/BearCleverProud/AwesomeWSI]