Quit While You're Ahead: Quit for Efficient Candidate Generation in Machine Translation Reranking

arXiv:2609.00588v1 Announce Type: new Abstract: Reranking methods, such as Minimum Bayes Risk (MBR) decoding and Quality Estimation (QE) reranking, are widely used in modern neural machine translation (NMT) to select an output from a set of candidate hypotheses. However, the performance gains come at the cost of high inference latency. Existing acceleration methods target MBR decoding and reduce only reranking computation, leaving QE reranking unaddressed and candidate generation---which can be the larger computational bottleneck---largely untouched. In this work, we propose Quit (Quantifying Uncertainty for Incremental Termination), a novel early-stopping strategy for the entire generation--reranking pipeline. Viewing candidate generation as a sequential decision under uncertainty, Quit incrementally generates and reranks candidates, stopping when the highest estimated quality in the candidate set stabilizes. Comprehensive experiments on three NMT models across 19 language pairs show that Quit yields end-to-end speedups of 1.47--2.66imes for MBR and 3.43--4.12imes for QE reranking while preserving translation quality within prespecified equivalence margins.

Related

• Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding
• Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
• LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation
• Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference

Source: arXiv cs.CL | 2026-09-02