Model Releases
A Model with No Head and Many Thoughts
arXiv:2608.31069v1 Announce Type: cross Abstract: Large language models decode by projecting hidden states through a large vocabulary head at every step. This operation is computationally costly and f
arXiv:2608.31069v1 Announce Type: cross Abstract: Large language models decode by projecting hidden states through a large vocabulary head at every step. This operation is computationally costly and forces all reasoning to be expressed in discrete tokens. We introduce Soft Latent Thinking, a method that replaces the LM head during reasoning with a lightweight projector, enabling autoregressive rollout in embedding space where reasoning steps remain continuous rather than tokenized. Experiments on DeepSeek-Qwen-1.5B and LLaMA-3.2-3B show that Soft Latent Thinking consistently improves pass@k across all k while reducing per-step compute during chain-of-thought. Our method achieves the highest pass@32 among all soft-thinking approaches, demonstrating that effective reasoning can be carried out in continuous space without discrete token generation.
Related
- Doing More With Less: Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling
- CausalDetox: Causal Head Selection and Intervention for Language Model Detoxification
- CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
Source: arXiv cs.CL | 2026-09-01