Structured Inference with Large Language Gibbs

arXiv:2606.19264v2 Announce Type: replace-cross Abstract: The knowledge encoded in large language models (LLMs) can serve as a substrate for structured reasoning over variables describing a complex world, but accessing this knowledge in a probabilistically coherent manner poses a difficult inference problem. We propose Large Language Gibbs, a scheme for structured probabilistic inference that uses conditional distributions of an LLM as transition operators. Rather than sampling structured objects through single-pass autoregressive generation, we iteratively resample individual variables conditioned on others using an LLM's next-token conditionals. This approach avoids order-dependent biases and produces a stationary distribution that reflects a compromise between all local conditionals. We apply this approach to sampling from synthetic distributions, consistent reasoning tasks, and Bayesian structure learning. The results suggest that the use of LLM conditionals in MCMC is a practical alternative to one-pass generation for structured probabilistic inference under a world prior accessible through noisy LLM conditionals.

Related

• Efficient Reasoning on the Edge
• Dissecting Failure Dynamics in Large Language Model Reasoning
• VERGE: Formal Refinement and Guidance Engine for Verifiable LLM Reasoning
• Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models

Source: arXiv cs.CL | 2026-09-04