Recently, Datalab released OmniExtractBench, a benchmark that combines several benchmarks (our own ExtractBench, LongExtractBench, other ven…

Recently, Datalab released OmniExtractBench, a benchmark that combines several benchmarks (our own ExtractBench, LongExtractBench, other vendors) to measure structured document quality. The benchmark measured our "Agentic Plus" extraction model, our most powerful mode for complex document extraction. We found a compatibility workaround in the benchmark’s integration that stripped null from fields that allowed it - removing a valid way to represent missing information. We restored that option while preserving the schema’s structure and required fields. With the same scorer, we were able to get to 93.94% on the benchmark. This puts it slightly higher (or at least equal) to the highest published scores. There's a lot of vendor benchmarks, but ultimately it's also important to evaluate over your own data. We'd be happy to help you get that set up. PR: https://github.com/datalab-to/omni_extract_bench/pull/11 Our Agentic Plus mode for LlamaExtract is legitimately quite good, would love to you to try it out and let us know your thoughts! https://cloud.llamaindex.ai/

Related

• ExtractBench is one of the most comprehensive benchmarks for real-world document extraction. ✅ It covers 4869 pages, across 67 document type…
• The most dangerous document extraction failure isn't a wrong value. It's a missing row that looks like nothing is wrong. We released Extract…
• We wrote a 36-page ArXiv whitepaper on ExtractBench 🧑‍🔬 , our effort to create the most comprehensive, schema-guided, real-world document …
• ExtractBench is now live on @Kaggle. It tests schema-guided document extraction on the documents most likely to break downstream agents and …

Source: Jerry Liu (X) | 2026-09-19