One of the interesting properties we’ve observed around schema-guided, complex document extraction tasks 📑 is that coding agent harnesses a…

One of the interesting properties we’ve observed around schema-guided, complex document extraction tasks 📑 is that coding agent harnesses are good baselines (in terms of cost/accuracy) for longer documents. We tested Claude Code and Codex, along with specialized OCR tools (including LlamaParse) and raw VLMs. * On short documents, specialized OCR tools are generally a fraction of the cost of coding agents, with equivalent or higher accuracy * On longer documents, coding agents are a bit closer to the cost/accuracy Pareto curve (see bottom graph) It’s an interesting result, though ultimately not surprising. Complex document extraction is a specialized reasoning task, and coding agents are effectively generalized reasoning harnesses. Over long documents, coding agents have more room to use a variety of tools to search snippets of the document instead of loading the entire document into context. They can also make use of prompt caching to reduce total token cost even as it expands multi-step reasoning. On the flip side, they do generate a baseline degree of token usage that proves to be wasteful for shorter docs compared to specialized extractors. This specific graph is in our Appendix D in the ParseBench paper, come check it out! ArXiv: https://arxiv.org/pdf/2607.29677 ExtractBench: https://www.extractbench.ai/

Source: Jerry Liu (X) | 2026-08-22

Related