Coding LLMs Can Be Harnessed to Learn Interpretable Models from Data

arXiv:2610.14006v1 Announce Type: new Abstract: Since compact, interpretable classifiers correspond to simple programs, it is natural to ask if LLM-based coding agents can be used to learn interpretable models from data. In this paper we present a simple harness that repeatedly prompts a coding LLM to produce the next rule in a rule set, given a sample of labeled examples. We show that this approach is competitive with, and often superior to, state-of-the-art rule-learning methods on tabular data, especially with small training sets, and wins by large margins on text, including against another LLM-based rule learner; that this approach outperforms standard harnesses, and that the approach immediately leads to new kinds of interpretable models, since coding agents are not restricted to search a highly restricted hypothesis space. In addition to the advances made in learning interpretable models, these results are of interest because they introduce a new class of coding benchmarks - construction of interpretable classifiers - which includes quantitative measurements of output quality (i.e., test-set error) and existing mature ``specialist coding algorithms'' (i.e., existing learning methods).

Related

• Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments?
• ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor
• Evaluating Memory Condensation Strategies for Coding Agents in Data-Driven Scientific Discovery
• Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Models

Source: arXiv cs.LG | 2026-10-14