TTSR: Test-Time Self-Evolving via Reflection

arXiv:2603.03297v2 Announce Type: replace Abstract: Test-time training (TTT) adapts large language models (LLMs) during inference using only unlabeled test inputs. Existing methods, however, face two major bottlenecks on hard reasoning tasks: (1) lack of learnable samples, as self-generated pseudo-labels on difficult questions are often noisy and yield unstable rewards; and (2) inefficient exploration, as performance gains depend on repeatedly sampling many rollouts without explicit diagnosis of why previous attempts fail. We propose extbf{TTSR} (extbf{T}est-extbf{T}ime extbf{S}elf-extbf{R}eflection), a self-evolving framework based on a reflect-then-synthesize paradigm. A single pretrained model alternates between a extit{Student} role and a extit{Teacher} role: the Student solves test questions and updates, while the Teacher analyzes failed trajectories and synthesizes targeted variant questions closer to the Student's capability frontier. TTSR further maintains a cross-iteration extit{weakness memory} and compiles persistent weaknesses into a lightweight extit{strategy note} prepended to subsequent Student inputs, so diagnostic knowledge can guide exploration and gradually fade as weaknesses are resolved. Experiments on challenging mathematical reasoning benchmarks show consistent test-time improvements, strong cross-backbone generalization, and transfer to general-domain reasoning tasks.

Related

• Procedural Knowledge at Scale Improves Reasoning
• MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization
• Strategic Scaling of Test-Time Compute: A Bandit Learning Approach
• Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scaling

Source: arXiv cs.CL | 2026-08-27