Agents
The difference with using VLMs for OCR vs. other tasks is that the ratio of output/input tokens is quite high - meaning the model can spend …
The difference with using VLMs for OCR vs. other tasks is that the ratio of output/input tokens is quite high - meaning the model can spend a lot of time writing markdown. Speculative decoding offers
The difference with using VLMs for OCR vs. other tasks is that the ratio of output/input tokens is quite high - meaning the model can spend a lot of time writing markdown. Speculative decoding offers a useful approach where a fast draft model proposes several tokens, and then the main model checks them together. Accepted tokens move the output forward; rejected proposals are corrected. The net benefit here is you reduce latency on sequential decoding steps, especially if the document structure makes certain text easier to predict. Check out the video as an overview! We've built a lot of optimizations into LlamaParse to push the frontiers of accuracy and cost, and then latency. There's still a ton to come: https://cloud.llamaindex.ai/ Media
Related
- Most document OCR solutions have parse (doc->markdown) and extract (doc + schema -> structured output) endpoints. Sometimes you want to extr…
- This is pretty cool!
- You don't need heavyweight VLMs to OCR simple text-only PDFs. Doing that is like bringing a bazooka to a knife-fight, and is completely unne…
- We dropped some new LlamaDrip 🧢 Fear of Docs LlamaParse
Source: Jerry Liu (X) | 2026-09-17