Safety
OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment
arXiv:2608.18474v1 Announce Type: new Abstract: Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to wor
arXiv:2608.18474v1 Announce Type: new Abstract: Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a single granularity, so practitioners often need separate systems for word- and sentence-level alignment---especially in multilingual and long-text settings. We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment with a single lightweight model. Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices, and obtains document-level m--n sentence alignments via sentence embeddings combined with dynamic programming. To balance fine-grained alignment accuracy and sentence-representation quality, we use a four-stage training pipeline: alignment-oriented continued pre-training, self-supervised learning, supervised fine-tuning on human annotations, and sentence-embedding distillation from a strong multilingual teacher. Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks and generalizes well to unseen language pairs. Surprisingly, later-stage supervised fine-tuning on short texts further improves alignment quality while retaining the long-context understanding acquired in earlier training, keeping the model robust on long-text word alignment. normalsize {olor{blue}extbf{Code}: https://github.com/MilkDargon/OmniAlign}par {olor{blue}extbf{Model}: https://huggingface.co/WPS-Qingqiu/OmniAlign}
Related
- CLEAR: Cross-Lingual Enhancement in Alignment via Reverse-training
- Soft Token Alignment for Cross-Lingual Reasoning
- Multilingual Word-Level Forced Alignment with Self-Supervised Representations and Learned Dynamic Programming
- Cross-Lingual Sentiment Misalignment: Auditing Multilingual Language Models for Inversion Risk, Dialectal Representation, and Affective Stability
Source: arXiv cs.CL | 2026-08-20