Making a synthetic dataset for fine-tuning

I've been thinking about building a pipeline to generate reasoning training data for LLMs, but I want to avoid the common failure mode of synthetic data where you just generate the same template with different numbers. The rough idea: Generate an abstract reasoning task (logic, planning, graph problems, math, algorithms, etc.) using a teacher model and/or procedural generators Convert the task into natural language Solve it with a formal solver/verifier where possible Keep only examples with verified solutions Collect attempts from multiple teacher models to create better training signals Use difficulty metrics to create a curriculum The main questions I have: Are there existing papers or projects that do something similar? What are good ways to prevent synthetic reasoning data from becoming repetitive? Is it better to generate tasks from formal grammars/simulators/environments rather than relying mainly on LLM-generated problems? Has anyone experimented with this approach for smaller open models? I'm especially interested in approaches that maximise diversity of reasoning patterns rather than simply scaling the number of samples. The goal is not to train a model directly, but to create a high-quality dataset for distillation. Ideally, the framework would be model-agnostic: the generator and solver could be swapped out for different teacher/student models or even used in a self-improvement loop. Disclosure: I used an LLM to rewrite this purely to sound clearer and fix spelling mistakes. submitted by /u/Aggravating-Push-207 [link] [comments]

Related

• DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
• What's the last model trained on human-data only?
• Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control

Source: r/LocalLLaMA | 2026-07-30