BUSTA: Teaching Small Language Models to Reason About Binary Execution

arXiv:2610.36288v1 Announce Type: cross Abstract: Large Language Models (LLM) are increasingly applied to Reverse Engineering (RE) workflows. RE, however, differs from common LLM coding tasks as it requires models to reason about low-level program behavior. Furthermore, the deployment of tooling in resource-constrained sandboxed RE environments limits the ability to use frontier-scale solutions. We examine whether a targeted LLM training approach can transfer distinct RE skills to open models via inverse basic-block execution: given an x86-64 basic block and a desired output register state, the model predicts the input register values that produce it. Fine-tuning exttt{Qwen3-1.7B} and exttt{Nemotron-3-Nano} with Group Relative Policy Optimization (GRPO) raises knowable-register prediction accuracy from 73.2% to 85.3% and from 63.3% to 95.2%, respectively. Frontier models still score highest (up to 99.2%), but our fine-tuned exttt{Nemotron-3-Nano} comes within four points at a fraction of the parameter count. We release both fine-tuned models and the training dataset.

Related

• Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmark
• The Path Not Taken: Duality in Reasoning about Program Execution
• CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering

Source: arXiv cs.AI | 2026-10-28