Search: “benchmark”

arXiv:2601.07338v2 Announce Type: replace Abstract: Large Language Models (LLMs) have significantly advanced Machine Translation (MT), applying them to linguistically complex domains-such as Social Ne

arXiv:2604.15267v1 Announce Type: cross Abstract: It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite tr

arXiv:2602.18502v2 Announce Type: replace Abstract: Although deep learning models in medical imaging often achieve excellent classification performance, they can rely on shortcut learning, exploiting

arXiv:2604.11762v1 Announce Type: new Abstract: Deep learning underpins a wide range of applications in MRI, including reconstruction, artifact removal, and segmentation. However, progress has been dr

arXiv:2512.00336v2 Announce Type: replace Abstract: The rapid advancement of AI-generated multimodal video-audio content has raised significant concerns regarding information security and content auth

arXiv:2604.09982v1 Announce Type: cross Abstract: Reproducibility must validate architectural robustness, not just numerical accuracy. We evaluate ColBERT-v2 and ConstBERT across five dimensions, find

arXiv:2604.08757v1 Announce Type: cross Abstract: Humor is one of the most culturally embedded and socially significant dimensions of human communication, yet it remains largely unexplored as a dimens

arXiv:2604.07720v2 Announce Type: replace Abstract: Deep Research (DR) requires LLM agents to autonomously perform multi-step information seeking, processing, and reasoning to generate comprehensive r

arXiv:2601.10198v3 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and generation, serving as the foundation for advanced persona s

arXiv:2503.08292v4 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly applied to outpatient referral tasks across healthcare systems. However, there is a lack of stan

arXiv:2604.07759v1 Announce Type: new Abstract: Ship detection for navigation is a fundamental perception task in intelligent waterway transportation systems. However, existing public ship detection d

Someone prompted different LLMs to generate CAD code for a bridge under fixed constraints (2-foot span, under 500g filament, 18-hour print limit), printed them, and load-tested them to failure. The re

Jerry Liu announces that the ParseBench leaderboard is automatically updated on Kaggle, providing a continuously maintained benchmark for parsing performance metrics. The leaderboard is hosted under t

arXiv:2605.07492v1 Announce Type: new Abstract: The past year has seen over 20 open-source document parsing models, yet thefield still benchmarks almost exclusively on OmniDocBench, a 1,355-pagemanual

The tweet from @NousResearch links to a Windows Central article written by Zac Bowden that provides an in‑depth review and benchmark analysis of Nvidia’s RTX Spark. Posted on Oct 16, 2026, the review

ParseBench is now also available on Papers with Code! Find it here: https://paperswithcode.co/benchmark/parsebench We benchmarked Mistral OCR against other frontier and open-weight models on ParseBenc

really great thread/benchmark for common sense in robotics and a reminder that we are not actually close to AGI GPT-6 Astra attempted harmful actions 97% of the time when it was asked to stab a human-

Auto-generated index of all tools mentioned across the wiki.

Auto-generated index of all models mentioned across the wiki.

Editor’s note: Some of the most interesting questions in AI are being asked by information theoreticians, around how to provide context to an emerging class of AI agents. A few weeks ago, we waded int

Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus inde

arXiv:2609.18366v1 Announce Type: new Abstract: Reliable agent evaluation is complicated by automatic harness optimization, which repeatedly uses a released benchmark B_{rel} to guide a Proposer that

We wrote a 36-page ArXiv whitepaper on ExtractBench 🧑‍🔬 , our effort to create the most comprehensive, schema-guided, real-world document extraction benchmark. It’s extremely detailed and covers every

Everyone who think vision is solved should look at some enterprise documents We’re open sourcing the first document OCR benchmark for the agentic era, ParseBench. Document parsing is the foundation of

arXiv:2610.18535v1 Announce Type: new Abstract: Video understanding benchmarks are the primary yardstick for measuring progress in video-language modeling, and a growing body of work relies on them to

arXiv:2610.13461v1 Announce Type: new Abstract: AI systems are making rapid progress in mathematics, but this progress is hard to measure. Benchmarks of solved problems saturate, and in benchmarks of

Unlike other players in the OCR space, we don't block our competitors from accessing our platform. We're also fully committed towards open, reproducible benchmarks for document parsing and extraction

Auto-generated index of all companies mentioned across the wiki.

arXiv:2505.00759v3 Announce Type: replace-cross Abstract: The steady improvements of text-to-image (T2I) generative models lead to slow deprecation of automatic evaluation benchmarks that rely on stat

arXiv:2608.28884v1 Announce Type: new Abstract: We introduce MineCEraft (Minecraft Construction Engineering Benchmark, pronounced mine-see-ee-raft), an easy-to-use, open-source benchmark designed to s

arXiv:2607.02032v1 Announce Type: new Abstract: Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure. A single evaluation c

arXiv:2603.13966v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models are increasingly evaluated across multiple simulation benchmarks, yet adding each benchmark to an evaluation pip

Auto-generated index of all papers mentioned across the wiki.

Hey r/LocalLLaMA, We’ve released our full ShapeLearn GGUFs for Qwen 3.8 27B. Blog / Download models TL;DR 3.84 bpw (GPU-5) reaches 99.63% of BF16’s aggregate score of 8 benchmarks, being the most accu

arXiv:2609.06535v1 Announce Type: new Abstract: We introduce 3DHarnessBench, a benchmark that evaluates the agentic ability of frontier vision-language models (VLMs) to recover 3D geometry as Blender

arXiv:2609.03261v1 Announce Type: cross Abstract: A benchmark score credits final answers, but not the route by which an item can be answered. In medical multimodal multiple-choice questions (MCQs), t

If you are thinking what model will fit best your HW specs and tasks you are doing here is one table with all currently popular models that still can be considered as local. LLM Test Scores Feature De

arXiv:2608.05086v1 Announce Type: new Abstract: Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to tr

arXiv:2601.15307v2 Announce Type: replace-cross Abstract: The rapid development of automated survey generation technology has made it increasingly important to establish a comprehensive benchmark to e

Join the ARC Prize team -- help us build ARC-AGI-4 and ARC-AGI-5 Platform Engineer - Benchmark Lead ARC Prize Foundation is hiring a senior engineer to build our benchmark platform * Expand ARC-AGI-3

arXiv:2610.17841v1 Announce Type: cross Abstract: Evaluating fashion retrieval systems remains challenging due to the limited availability of benchmarks that capture the fine-grained, open-ended queri

Source: whatmodel.sh, an open source AI leaderboard I built (and launched here yesterday). Every point comes from published benchmark results: 22,113 of them across 987 benchmarks, from 1,011 sources,

'TypeSafe AI sells Jev as a frontier-class reasoner that cannot hallucinate, built by the co-inventor of ChatGPT - fast, and almost free. We ran it live on 16,379 benchmark requests, measured its late

arXiv:2609.39909v1 Announce Type: cross Abstract: We introduce DoGBENCH (Documentation Generation Benchmark), to our knowledge, the first benchmark for generating and maintaining real user-facing soft

arXiv:2609.39490v1 Announce Type: cross Abstract: Recent advances have enabled unified omni-modal models in understanding audio, vision, and language. However, existing benchmarks, training data, and

arXiv:2510.27065v2 Announce Type: replace Abstract: We present MLPerf Automotive, the first standardized public performance benchmark for evaluating Machine Learning systems that are deployed for AI a