From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL

arXiv:2608.07213v1 Announce Type: new Abstract: Test-time scaling can correct difficult text-to-SQL queries, but the extra computation is normally discarded after each answer. Systems increasingly retain verified repair episodes, yet evaluations still report one end-to-end score. It cannot distinguish replay on recurring questions from help on unseen questions, or identify the responsible memory choice. We call measuring this future value the crystallization problem. Our controlled evaluation holds the single-shot solver fixed and varies one memory choice at a time. We separately measure replay, cross-question retention, and held-out same-database transfer. On BIRD, storing verified corrected queries improves held-out first-attempt accuracy by 4.34 percentage points. This gain captures 44.4% of the accuracy headroom provided by on-demand repair on the same questions. Controlled interventions identify database-specific content as the main operating ingredient. Reliable verification and broader retrieval coverage yield supported gains; richer formats and elaborate retrievers do not. Open-source code, evaluation artifacts, and reproduction instructions are available at https://github.com/ai-jiaqian/text-to-sql-memory-crystallization.

Related

• Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling
• LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification
• Optimal Aggregation of LLM and PRM Signals for Efficient Test-Time Scaling
• GradMem: Learning to Write Context into Memory with Test-Time Gradient Descent
• DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA

Source: arXiv cs.CL | 2026-08-10