Leveraging Spatial Transcriptomics as Alternative to Manual Annotations for Deep Learning-Based Nuclei Analysis

arXiv:2604.23481v1 Announce Type: new Abstract: Deep learning-based nuclei segmentation and classification in pathology images typically rely on large-scale pixel-level manual annotations, which are costly and difficult to obtain across diverse tissues and staining conditions. To address this limitation, we propose a framework that leverages spatial transcriptomics (ST) data as supervision for nuclei segmentation and classification. By incorporating cell-level ST data, we obtain gene expression profiles and corresponding nuclear masks from histopathological images. Gene expression profiles are converted into cell-type labels and used as training data for image-based classification. Because existing gene expression-based cell-type classification methods are not designed for image recognition, we introduce an image-oriented classification approach that bridges gene expression-based cell typing and image-based cell classification. To evaluate generalization, we conduct segmentation experiments on previously unseen organs and compare our method with conventional supervised models. Despite being trained on fewer organ types, our framework achieves higher segmentation accuracy, demonstrating strong transferability. Classification experiments further show consistent improvements over existing approaches.

Related

• Is SAM3 ready for pathology segmentation?
• Do Instance Priors Help Weakly Supervised Semantic Segmentation?
• SemiSAM-O1: How far can we push the boundary of annotation-efficient medical image segmentation?
• VitaminP: cross-modal learning enables whole-cell segmentation from routine histology
• Detecting and refurbishing ground truth errors during training of deep learning-based echocardiography segmentation models

Source: arXiv cs.CV | 2026-04-28