Search: “benchmark”

arXiv:2609.07373v1 Announce Type: new Abstract: Modern Visual Place Recognition (VPR) methods excel on standard benchmarks yet remain brittle in feature-poor environments. By treating each query image

arXiv:2503.04666v2 Announce Type: replace Abstract: High-quality benchmarks are crucial for driving progress in machine learning research. However, despite the growing interest in video generation, th

The missing control is visible in sudoingX’s Ling-3.0-flash benchmark graphics. The earlier table leaves Ling’s no-speculation baseline as “not measured.” The later code/prose graphic fills it in: abo

arXiv:2609.05178v1 Announce Type: new Abstract: Vision-Language-Action (VLA) or World Action (WAM) models have recently demonstrated remarkable performance in robotic manipulation. On LIBERO, SOTA met

arXiv:2609.05381v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated on molecular property benchmarks, but accuracy cannot distinguish a model that predicts a proper

arXiv:2609.04550v1 Announce Type: new Abstract: Video-language benchmarks are usually constructed by the dataset authors without published reliability statistics, leaving the noise floor of the constr

arXiv:2609.03526v1 Announce Type: new Abstract: Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cult

arXiv:2510.17460v2 Announce Type: replace Abstract: Idioms remain a persistent challenge in natural language processing due to their figurative and culturally grounded meanings, which distinguish them

arXiv:2609.03781v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in multilingual settings, yet their safety is still evaluated primarily in English. This limits our

arXiv:2609.03620v1 Announce Type: cross Abstract: Audio deepfake detection is commonly formulated as clip-level binary classification of single-domain audio. However, real-world manipulated audio can

arXiv:2606.21144v2 Announce Type: replace-cross Abstract: Long-term memory systems allow LLM agents to preserve information beyond a single context window, but most systems focus on storing and retrie

arXiv:2609.01888v1 Announce Type: new Abstract: Recent inference-time hallucination mitigation methods for large vision-language models (LVLMs) report strong gains on hallucination benchmarks. However

GPT-6 Astra is coming to Devin. On FrontierCode 1.1, Astra performs within 0.4 points of Fable 5 at a 64% lower cost. It also sets a new SOTA on our internal testing benchmark, generating more compreh

arXiv:2609.02095v1 Announce Type: new Abstract: An AI agent can perform well on benchmarks and still be unsuitable for deployment. Existing AI-agent benchmarks measure whether an agent can complete re

arXiv:2609.00004v1 Announce Type: new Abstract: This paper studies a finite-horizon multi-item capacitated lot-sizing problem in which demand quantities are deterministic, while demand-arrival periods

arXiv:2606.18237v2 Announce Type: replace-cross Abstract: Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to eva

arXiv:2609.00844v1 Announce Type: new Abstract: Customer-service QA in an AI contact center (AICC) runs under deployment constraints that benchmark QA misses: tight voice-hotline latency and a high co

arXiv:2609.00921v1 Announce Type: new Abstract: Personalized Large Language Models (PLLMs) aim to tailor responses to individual users, where a central challenge is preference reasoning: inferring que

arXiv:2608.24621v2 Announce Type: replace Abstract: Can language models be trusted in safety- critical operations? In such settings, strong per- formance on semantic metrics does not guaran- tee opera

arXiv:2608.30762v1 Announce Type: new Abstract: Chess board recognition is the task of mapping the image of a chess board to the information of which piece is on which square. So far this task has two

arXiv:2608.31029v1 Announce Type: new Abstract: End-to-end autonomous driving models plan future trajectories from raw sensor input. While earlier driving benchmarks often measured deviation from the

arXiv:2608.29431v1 Announce Type: new Abstract: Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluation

arXiv:2608.29372v1 Announce Type: new Abstract: Retrospective backtests provide a limited test of adaptive trading agents: they cannot rule out historical contamination, expose sensitivity to a single

arXiv:2608.28623v1 Announce Type: cross Abstract: Large multimodal reasoning models (LMRMs) are getting increasingly capable, primarily through generating explicit chain-of-thought reasoning before an

arXiv:2608.28675v1 Announce Type: cross Abstract: Video reasoning tasks such as grounded video question answering and temporal grounding require selecting temporal evidence that supports the query. In

arXiv:2608.29601v1 Announce Type: cross Abstract: We present N_0-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal dat

arXiv:2608.30241v1 Announce Type: new Abstract: Recent efforts have aimed to automate scientific diagram generation from paper content (Lin et al., 2026; Zhu et al., 2026a). However, fully satisfying

arXiv:2608.30156v1 Announce Type: new Abstract: Plane geometry problem (PGP) solving has become a critical benchmark for multimodal reasoning because it requires accurate visual perception and precise

arXiv:2601.13206v2 Announce Type: replace Abstract: Large Language Models (LLMs) generate text token-by-token in discrete time, yet real-world communication, from therapy sessions to business negotiat

arXiv:2608.30714v1 Announce Type: new Abstract: Verifying image authenticity is increasingly difficult, posing serious risks across journalism, law enforcement, and political domains. Most existing fo

arXiv:2606.16011v2 Announce Type: replace Abstract: Standard accuracy benchmarks evaluate whether large language models (LLMs) reach correct answers. However, they do not test whether models maintain

arXiv:2608.28343v1 Announce Type: new Abstract: High-throughput phenotyping depends on accurate 3D reconstruction of plants across growth stages, yet the development and evaluation of temporal complet

arXiv:2512.17581v2 Announce Type: replace Abstract: Benchmarking competitions are central to the development of artificial intelligence (AI) in medical imaging, defining performance standards and shap

// Your LLM judge disagrees with the experts // LLM Judges can be tricky to build. Here is an interesting showcasing why: There propose a reference-full benchmark of hundreds of complete human-to-huma

arXiv:2608.26204v1 Announce Type: cross Abstract: Computer Use Agents (CUAs) are increasingly deployed to navigate mobile and desktop applications on behalf of users, yet no benchmark comprehensively

arXiv:2608.26993v1 Announce Type: new Abstract: Explicit visual intermediates can help multimodal large language models (MLLMs) externalize spatial evidence and updated visual states, but their utilit

arXiv:2505.14104v2 Announce Type: replace Abstract: Legal rules encompass not only codified statutes but also implicit adjudicatory principles derived from precedents that contain discretionary norms,

arXiv:2608.27328v1 Announce Type: new Abstract: High similarity between first-visit and return frames does not necessarily show that a video world model remembered the scene; the intervening rollout m

spec: Add benchmark-only synthetic speculative acceptance options (#27711) Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli Address review comments Address review comm

arXiv:2608.25630v1 Announce Type: new Abstract: Volumetric medical VQA requires reasoning over long and redundant 3D visual token sequences, especially in multi-sequence MRI where complementary modali

arXiv:2608.24940v1 Announce Type: new Abstract: Partial differential equations (PDEs) often have high-frequency and multi-scale features that neural networks struggle to approximate. Physics-Informed

arXiv:2608.23636v1 Announce Type: new Abstract: Small-object detection and instance segmentation remain challenging in orchard environments because of green-on-green similarity, occlusion, and limited

arXiv:2608.24191v1 Announce Type: cross Abstract: Urdu, the world's tenth most spoken language with 246 million speakers, remains almost entirely absent from mainstream LLM safety evaluation and nine

arXiv:2608.21832v1 Announce Type: new Abstract: Computer-use agents ground natural-language instructions in screenshots to locate interface elements, yet existing benchmarks do not isolate whether mod

arXiv:2608.22504v1 Announce Type: new Abstract: Existing AI-for-PDE benchmarks primarily assess models in terms of predictive or approximation accuracy. In physics research, however, AI outputs often

arXiv:2608.20634v1 Announce Type: cross Abstract: Agents learn to act through interaction with environments, yet the environments used for training are often manually constructed or synthesized around