dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning

arXiv:2602.10603v3 Announce Type: replace Abstract: Genomic foundation models have the potential to decode DNA syntax, yet face a fundamental tradeoff in their input representation. Standard fixed-vocabulary tokenizers fragment biologically meaningful motifs such as codons and regulatory elements, while nucleotide-level models preserve biological coherence but incur prohibitive computational costs for long contexts. We introduce dnaHNet, a state-of-the-art tokenizer-free autoregressive model that segments and models genomic sequences end-to-end. Using a differentiable dynamic chunking mechanism, dnaHNet compresses raw nucleotides into latent tokens adaptively, balancing compression with predictive accuracy. Pretrained on prokaryotic genomes, dnaHNet outperforms leading architectures including StripedHyena2 in scaling and efficiency. This recursive chunking yields quadratic FLOP reductions, enabling >3 imes inference speedup over Transformers. On zero-shot tasks, dnaHNet achieves superior performance in predicting protein variant fitness and gene essentiality, while automatically discovering hierarchical biological structures without supervision. These results establish dnaHNet as a scalable, interpretable framework for next-generation genomic modeling.

Related

• Distilling Genomic Models for Efficient mRNA Representation Learning via Embedding Matching
• Interventional Time Series Priors for Causal Foundation Models
• Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
• AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models

Source: arXiv cs.LG | 2026-04-13