Search: “benchmark”

arXiv:2511.11934v3 Announce Type: replace-cross Abstract: We present a systematic benchmark of out-of-distribution (OOD) detection CSFs through a representation-centric lens. Our study spans CNN and V

arXiv:2605.17270v1 Announce Type: new Abstract: Modern visual object trackers show impressive results on general targets, yet their performance drops substantially when dealing with scene text. Althou

arXiv:2605.18727v1 Announce Type: cross Abstract: Evaluating embodied systems on real dexterous hardware requires more than isolated primitive skills: an agent must perceive a changing tabletop scene,

arXiv:2605.17478v1 Announce Type: new Abstract: Visual Geometry Grounded Transformers (VGGT) have set new benchmarks in high-fidelity 3D scene reconstruction. However, as the sequence length increases

arXiv:2507.22057v2 Announce Type: replace Abstract: Difficult few-shot image recognition has significant application prospects, yet remaining the substantial technical gaps with the conventional large

arXiv:2605.17152v1 Announce Type: new Abstract: Multimodal LLMs are evolving from vision-language to tri-modality that see, hear, and read, yet pipelines and benchmarks remain English-centric and comp

arXiv:2605.17360v1 Announce Type: new Abstract: Real-time duplex interaction is essential for multimodal AI systems operating in real-world scenarios, where models must continuously process streaming

arXiv:2605.17860v1 Announce Type: cross Abstract: While modern Automatic Speech Recognition (ASR) systems achieve high accuracy on benchmark corpora, their performance often degrades when there is rea

arXiv:2605.18128v1 Announce Type: new Abstract: Existing Multivariate Time Series Anomaly Detection (MTSAD) frameworks increasingly rely on integrating Graph Neural Networks (GNNs) with sequence model

Read more on what we've learned while developing long-horizon agent evals: Curious finding while creating evals and benchmarks for long-horizon (100+ turn) agents While it’s generally thought that a d

arXiv:2605.16628v1 Announce Type: new Abstract: The SCARED dataset is a widely used benchmark for endoscopic depth estimation, offering ground-truth 3D reconstructions captured with a structured light

arXiv:2512.17843v4 Announce Type: replace-cross Abstract: By evaluating Large Language Models (LLMs) through uniform, text-only interfaces, current academic benchmarks obscure how the unique designs a

arXiv:2605.17789v1 Announce Type: cross Abstract: Memory systems for AI assistants were built for single-user dialogue and fail characteristically when applied to multi-party social group settings. Th

arXiv:2605.16381v1 Announce Type: cross Abstract: Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to r

arXiv:2605.17153v1 Announce Type: new Abstract: Neural network verifiers aim to provide formal guarantees on model behavior, but existing verification benchmarks are fundamentally limited by their lac

arXiv:2605.16410v1 Announce Type: new Abstract: Test-time scaling (TTS) methods have proven highly effective for LLMs, yet their application to vision-language models (VLMs) remains relatively underex

arXiv:2605.18063v1 Announce Type: new Abstract: Object counting is a foundational vision task with over a decade of dedicated research, yet state-of-the-art models still fail systematically in the mix

arXiv:2304.10726v3 Announce Type: replace-cross Abstract: We introduce the Deep Learning Vulnerability Analyzer (DLVA) for Ethereum smart contracts based on neural networks. We train DLVA to judge byt

arXiv:2605.18580v1 Announce Type: new Abstract: Outcome-only evaluation can certify economically unsafe agents: a policy can hit a business KPI while violating deployable behavioral discipline. In hot

arXiv:2605.18132v1 Announce Type: cross Abstract: Generative 3D models are deployed in gaming, robotics, and immersive creation, making source attribution critical: given a 3D asset, can we identify w

arXiv:2605.15708v1 Announce Type: new Abstract: Recent advances in 3D datasets and multimodal models have greatly improved natural language 3D scene understanding. However, most 3D referring segmentat

arXiv:2605.15906v1 Announce Type: new Abstract: Image degradations can occur during acquisition, processing, and transmission, altering visual appearance and affecting downstream vision tasks. They ar

arXiv:2605.15761v1 Announce Type: new Abstract: Evaluation leaderboards such as LMArena play a central role in benchmarking large language models by aggregating pairwise human preferences into model r

arXiv:2605.15518v1 Announce Type: new Abstract: The effective detection and governance of Large Language Model (LLM) generated content has become increasingly critical due to the growing risk of misus

arXiv:2603.03066v2 Announce Type: replace Abstract: Existing AI-generated video quality assessment (AIGVQA) methods mainly focus on global perceptual realism and coarse text-video alignment, while ove

arXiv:2508.20810v3 Announce Type: replace Abstract: Rigorous evaluation of domain-specific language models requires benchmarks that are comprehensive, contamination-resistant, and maintainable. Static

GLM-5.1 from @Zai_org is now live on OrcaRouter • #1 open-source model on SWE-Bench Pro • Beats closed source models on real-world repo repair benchmarks • MIT licensed • 200K context • Built for long

arXiv:2605.15226v1 Announce Type: cross Abstract: We ask whether agentic AI systems built for software engineering transfer to realistic hardware engineering. Existing hardware LLM benchmarks isolate

arXiv:2605.15235v1 Announce Type: new Abstract: Multimodal physiological data powers clinical AI systems from intensive care units to wearable devices, but sensors routinely fail in practice. Two fail

arXiv:2605.15509v1 Announce Type: new Abstract: While Isaac Lab provides massive parallel UAV simulation, OmniSafe and safe-control-gym provide constrained-RL benchmarks, and CBFKit provides control-b

arXiv:2605.15846v1 Announce Type: cross Abstract: Coding agents are increasingly deployed in real software development, where a single version iteration requires months of coordinated work across many

arXiv:2605.15868v1 Announce Type: new Abstract: In this work, we address the critical yet underexplored challenge of symmetric multimodal-to-multimodal (MM2MM) retrieval, where queries and contexts ar

arXiv:2605.15607v1 Announce Type: new Abstract: Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from

The Open Agent Leaderboard is a benchmarking system hosted on Hugging Face that evaluates and ranks AI agents based on their performance across various tasks and capabilities. It provides a standardiz

arXiv:2605.15672v1 Announce Type: cross Abstract: Vision-language models (VLMs) achieve strong performance on multimodal benchmarks, but may still lack robust control over basic visual operations. We

arXiv:2605.16046v1 Announce Type: cross Abstract: Semantic code search has been widely adopted in both academia and industry. These approaches embed natural-language queries and code snippets into a s

The AMD Ryzen AI Max+ 395 'Strix Halo' APU outperforms the NVIDIA RTX 5080 by up to 3x in AI benchmarks, particularly for language models, with 128GB of accessible VRAM compared to the RTX 5080's 16GB

arXiv:2510.02837v2 Announce Type: replace Abstract: Although recent tool-augmented benchmarks involve complex requests, evaluation remains limited to answer matching, neglecting critical trajectory as

arXiv:2509.23023v3 Announce Type: replace Abstract: Large language models are increasingly deployed in multi-agent settings whose outcomes hinge on social intelligence, motivating evaluations of their

arXiv:2511.13397v2 Announce Type: replace-cross Abstract: The remarkable progress of Vision-Language Models (VLMs) on a variety of tasks has raised interest in their application to automated driving.

EBM are so back! @ylecun has been pointing here for years: AI reasoning needs systems that check structure before they answer. Aleph from @logic_int now leads the major formal reasoning benchmarks – l

arXiv:2605.14186v1 Announce Type: new Abstract: Large language models (LLMs) often expose useful signals of self-monitoring: before solving a problem, they can estimate whether they are likely to succ

arXiv:2605.15071v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied to cultural heritage materials, from digital archives to educational platforms. This work ident

arXiv:2605.14890v1 Announce Type: new Abstract: Foundation models tokenize Ukrainian legal text with vastly different efficiency, yet no systematic comparison exists for this domain. We benchmark seve

arXiv:2605.14253v1 Announce Type: new Abstract: Purpose: Mechanical thrombectomy (MT) improves stroke outcomes, but is limited by a lack of local treatment access. Widespread distribution of reinforce

arXiv:2605.12816v1 Announce Type: new Abstract: The Average Gradient Outer Product (AGOP) governs feature learning in neural networks: the Neural Feature Ansatz states that weight Gram matrices at eac