TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models

arXiv:2609.04071v1 Announce Type: cross Abstract: Pathology foundation models improve transferable representation learning for histopathology, but recent gains often rely on encoders with hundreds of millions of parameters and high inference cost. We propose TAP-Path, a task-adaptive compression framework that directly restructures a pretrained Virchow2 encoder rather than distilling it into a separate student. TAP-Path combines validation-driven transformer-block selection, physical removal of redundant blocks, input-adaptive patch-token pruning, multi-depth feature recovery, and a lightweight gated task head. The final model retains 24 of 32 transformer blocks and 70% of patch tokens after pruning, reducing encoder parameters by 24.96% (631.24M to 473.70M) and analytical encoder compute by 35.20% (340.13G to 220.40G FLOPs). Across three task-head optimization seeds, TAP-Path achieved 87.98 pm 0.067% test accuracy, 81.26 pm 0.49% balanced accuracy, and 82.38 pm 0.48% macro-F1 on a 32-class histopathology benchmark, compared with 86.89% for full Virchow2 and 87.67% for UNI2-h. TAP-Path achieved a Brier score of 0.1800 pm 0.0005 and failure-detection AUROC of 0.9047 pm 0.0060. A validation-only rare-aware objective improved rare-class balanced accuracy in a secondary operating analysis. Frozen external evaluation on 433 CPTAC samples yielded 91.22 pm 0.83% accuracy and 91.10 pm 0.81% balanced accuracy. These results show that task-adaptive structural and token sparsification can improve the accuracy-efficiency trade-off of large pathology foundation models while preserving reliability under internal and external evaluation.

Related

• Benchmarking Pathology Foundation Models for Spatial Domain Understanding
• Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models
• TokenButler: Token Importance is Predictable

Source: arXiv cs.AI | 2026-09-04