RT-SFT: Text Style Transfer from Non-Parallel Corpora by Roundtrip Translation

arXiv:2602.15013v2 Announce Type: replace Abstract: Text style transfer (TST) is naturally a supervised task - rewrite a sentence in a target style while preserving its meaning - yet the parallel corpora that supervision requires exist for only a handful of style domains. A common workaround is to normalize an input into a style-agnostic intermediate and then stylize it into the target style, but the normalizer is typically a lightweight, task-specific paraphraser applied only at test time, feeding a correspondingly small stylizer. We observe that a style-stripping normalizer already exists at scale: neural MT systems trained on hundreds of millions of general-domain sentence pairs preserve content while regressing toward generic phrasing, so roundtrip translation through a pivot language strips stylistic signal without any task-specific training. This turns normalization from an inference-time patch into a data-generation tool. Roundtrip-translating a monolingual in-style corpus yields a pseudo-parallel corpus on which we LoRA-finetune an instruction-tuned LLM as the stylizer (RT-SFT); applying the same normalizer to test queries keeps that stylizer in-distribution. We show that across four style domains, RT-SFT outperforms state-of-the-art methods, such as few-shot in-context learning, by considerable margins. We also report on effective retrieval augmentation methods for expert style domains with strict terminology and naming conventions.

Related

• Example-Guided Prompting for Document-Level Text Simplification
• SimpCue: Cue-Based Prompting for Multilingual Text Simplification
• Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data

Source: arXiv cs.CL | 2026-10-05