RefineStat: Efficient Exploration for Probabilistic Program Synthesis

arXiv:2509.01082v3 Announce Type: replace Abstract: Probabilistic programming offers a powerful framework for modeling uncertainty, yet statistical model discovery in this domain entails navigating an immense search space under strict domain-specific constraints. When small language models are tasked with generating probabilistic programs, they frequently produce outputs that suffer from both syntactic and semantic errors, such as flawed inference constructs. Motivated by probabilistic programmers' domain expertise and debugging strategies, we introduce RefineStat, a language model--driven framework that enforces semantic constraints ensuring synthesized programs contain valid distributions and well-formed parameters, and then applies diagnostic-aware refinement by resampling prior or likelihood components whenever reliability checks fail. We evaluate RefineStat on multiple probabilistic-programming code-generation tasks using smaller language models (SLMs) and find that it produces programs that are both syntactically sound and statistically reliable, often matching or surpassing those from closed-source large language models (e.g., OpenAI o3).

Related

• Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space
• On Different Notions of Redundancy in Conditional-Independence-Based Discovery of Graphical Models
• Imitation Learning for Combinatorial Optimisation under Uncertainty
• Identifying Information from Observations with Uncertainty and Novelty

Source: arXiv cs.LG | 2026-04-21