Model Releases
Improving Information Extraction with Learned Queries
arXiv:2608.31058v1 Announce Type: new Abstract: When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this
arXiv:2608.31058v1 Announce Type: new Abstract: When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.
Related
- Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity
- Improving Ad-hoc Search Effectiveness for Conversational Information Retrieval via Model Merging
- RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners
Source: arXiv cs.CL | 2026-09-01