Model Releases
ImmigrationReason: A Structured Dataset of U.S. Immigration Appeals for Legal Reasoning Research
arXiv:2608.20391v1 Announce Type: new Abstract: Most legal NLP resources draw from federal case law and focus on coarse classification, leaving administrative adjudication, where the vast majority of
arXiv:2608.20391v1 Announce Type: new Abstract: Most legal NLP resources draw from federal case law and focus on coarse classification, leaving administrative adjudication, where the vast majority of government decisions occur, essentially unaddressed. We introduce ImmigrationReason, a large-scale structured dataset derived from 12,375 non-precedent decisions of the U.S. Citizenship and Immigration Services (USCIS) Administrative Appeals Office (AAO) spanning 2005 to 2026. Each record captures the applicable legal framework, per-criterion evidence-sufficiency findings under a five-category label, verbatim adjudicator-criticism quotes, all citations, and final dispositions, alongside high-quality Claude-transcribed source text. Extraction quality is validated through a three-pass pipeline combining two independent modalities with comparison-prompt adjudication by Opus 4.7, and verified by domain experts on a 500-record sample. The dataset documents nearly 9,000 verbatim instances of AAO-identified legal errors, spans a natural legal-regime transition (the 2016 Dhanasar rule change), and covers 21 years of adjudication. We analyze the dataset in detail and outline research directions it enables, from outcome prediction and adjudicator-error analysis to agent design for high-stakes regulatory domains.
Related
- LegalBench-BR: A Benchmark for Evaluating Large Language Models on Brazilian Legal Decision Classification
- RTI-Bench: A Structured Dataset for Indian Right-to-Information Decision Analysis
- CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval
- CrossLex: A Source-Grounded Benchmark for Cross-Jurisdictional Legal Reasoning in Large Language Models
Source: arXiv cs.CL | 2026-08-24