Search: “benchmark”

Sphere-Depth: A Benchmark for Depth Estimation Methods with Varying Spherical Camera Orientations

arXiv:2604.23432v1 Announce Type: cross Abstract: Reliable depth estimation from spherical images is crucial for 360{eg} vision in robotic navigation and immersive scene understanding. However, the on

28 Apr 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents

DGX agent

arXiv:2512.07538v3 Announce Type: replace Abstract: Recognizing semantic differences across documents is crucial for text generation evaluation and content alignment, especially in cross-lingual setti

model-releasesarxiv-cs-cl

28 Apr 2026

Safety

Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines

arXiv:2604.23001v1 Announce Type: cross Abstract: Despite remarkable progress in Vision--Language--Action (VLA) models, a central bottleneck remains underexamined: the data infrastructure that underli

safetyarxiv-cs-ai

28 Apr 2026

FETS Benchmark: Foundation Models Outperform Dataset-specific Machine Learning in Energy Time Series Forecasting

arXiv:2604.22328v1 Announce Type: cross Abstract: Driven by the transition towards a climate-neutral energy system, accurate energy time series forecasting is critical for planning and operation. Yet,

27 Apr 2026

PSI: A Benchmark for Human Interpretation and Response in Traffic Interactions

arXiv:2112.02604v3 Announce Type: replace-cross Abstract: Accurately modeling pedestrian intention and understanding driver decision-making processes are critical for the development of safe and socia

27 Apr 2026

SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments

arXiv:2604.22409v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced static visual--spatial reasoning, yet they often fail to preserve long-horizon spatial coherence

27 Apr 2026

ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response

arXiv:2604.21199v1 Announce Type: cross Abstract: Time series question-answering (TSQA), in which we ask natural language questions to infer and reason about properties of time series, is a promising

Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision

arXiv:2604.21461v1 Announce Type: new Abstract: Egocentric AI agents, such as smart glasses, rely on pointing gestures to resolve referential ambiguities in natural language commands. However, despite

OptiVerse: A Comprehensive Benchmark towards Optimization Problem Solving

arXiv:2604.21510v1 Announce Type: new Abstract: While Large Language Models (LLMs) demonstrate remarkable reasoning, complex optimization tasks remain challenging, requiring domain knowledge and robus

RailVQA: A Benchmark and Framework for Efficient Interpretable Visual Cognition in Automatic Train Operation

arXiv:2603.27112v2 Announce Type: replace Abstract: As Automatic Train Operation (ATO) advances toward GoA4 and beyond, it increasingly depends on efficient, reliable cab-view visual perception and de

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

arXiv:2604.21696v1 Announce Type: new Abstract: Tabular foundation models aim to learn universal representations of tabular data that transfer across tasks and domains, enabling applications such as t

Agents

AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite

arXiv:2510.21652v2 Announce Type: replace Abstract: AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and

agentsarxiv-cs-ai

CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs

arXiv:2604.20460v1 Announce Type: new Abstract: Safety-critical traffic reasoning requires contrastive consistency: models must detect true hazards when an accident occurs, and reliably reject plausib

IVY-FAKE: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detection

arXiv:2506.00979v5 Announce Type: replace-cross Abstract: The rapid development of Artificial Intelligence Generated Content (AIGC) techniques has enabled the creation of high-quality synthetic conten

RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Ranking

arXiv:2604.20623v1 Announce Type: cross Abstract: Traditional change detection identifies where changes occur, but does not explain what changed in natural language. Existing remote sensing change cap

BALTIC: A Benchmark and Cross-Domain Strategy for 3D Reconstruction Across Air and Underwater Domains Under Varying Illumination

arXiv:2604.19133v1 Announce Type: new Abstract: Robust 3D reconstruction across varying environmental conditions remains a critical challenge for robotic perception, particularly when transitioning be

Benchmarking Quantum Kernel Support Vector Machines Against Classical Baselines on Tabular Data: A Rigorous Empirical Study with Hardware Validation

arXiv:2604.18837v1 Announce Type: cross Abstract: Quantum kernel methods have been proposed as a promising approach for leveraging near-term quantum computers for supervised learning, yet rigorous ben

Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences

arXiv:2601.04925v2 Announce Type: replace Abstract: Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmfu

From Natural Language to Executable Narsese: A Neuro-Symbolic Benchmark and Pipeline for Reasoning with NARS

arXiv:2604.18873v1 Announce Type: new Abstract: Large language models (LLMs) are highly capable at language generation, but they remain unreliable when reasoning requires explicit symbolic structure,

HarDBench: A Benchmark for Draft-Based Co-Authoring Jailbreak Attacks for Safe Human-LLM Collaborative Writing

arXiv:2604.19274v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as co-authors in collaborative writing, where users begin with rough drafts and rely on LLMs to compl

Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models

arXiv:2601.22737v2 Announce Type: replace Abstract: The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current

MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation

arXiv:2604.18914v1 Announce Type: cross Abstract: While multilingual large language models (LLMs) perform well on high-level tasks like translation and question answering, their ability to handle gram

Pixels or Positions? Benchmarking Modalities in Group Activity Recognition

arXiv:2511.12606v3 Announce Type: replace Abstract: Group Activity Recognition (GAR) is well studied on the video modality for surveillance and indoor team sports (e.g., volleyball, basketball). Yet,

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains

arXiv:2604.19001v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm

Adverse-to-the-eXtreme Panoptic Segmentation: URVIS 2026 Study and Benchmark

arXiv:2604.16984v1 Announce Type: new Abstract: This paper presents the report of the URVIS 2026 challenge on adverse-to-extreme panoptic segmentation. As the first challenge of its kind, it attracted

Camo-M3FD: A New Benchmark Dataset for Cross-Spectral Camouflaged Pedestrian Detection

arXiv:2604.16582v1 Announce Type: new Abstract: Pedestrian detection is fundamental to autonomous driving, robotics, and surveillance. Despite progress in deep learning, reliable identification remain

Chain Of Interaction Benchmark (COIN): When Reasoning meets Embodied Interaction

model-releasesarxiv-cs-ro

arXiv:2604.16886v1 Announce Type: new Abstract: Generalist embodied agents must perform interactive, causally-dependent reasoning, continually interacting with the environment, acquiring information,

Research

Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks

arXiv:2604.17761v1 Announce Type: cross Abstract: Interpretability tools are increasingly used to analyze failures of Large Language Models (LLMs), yet prior work largely focuses on short prompts or t

researcharxiv-cs-cl

Deep Learning for Virtual Reality User Identification: A Benchmark

arXiv:2604.16341v1 Announce Type: cross Abstract: Virtual Reality (VR) applications require robust user identification systems to ensure secure access to equipment and protect worker identities. Motio

E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes

arXiv:2604.17969v1 Announce Type: new Abstract: Visual search in 3D environments requires embodied agents to actively explore their surroundings and acquire task-relevant evidence. However, existing v

MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge

arXiv:2604.18164v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have been increasingly used as automatic evaluators-a paradigm known as MLLM-as-a-Judge. However, their reliabi

PFDelta: A Benchmark Dataset for Power Flow under Load, Generation, and Topology Variations

arXiv:2510.22048v3 Announce Type: replace Abstract: Power flow (PF) calculations are the backbone of real-time grid operations, across workflows such as contingency analysis (where repeated PF evaluat

Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation

arXiv:2604.17260v1 Announce Type: new Abstract: Evaluating meeting effectiveness is crucial for improving organizational productivity. Current approaches rely on post-hoc surveys that yield a single c

SciImpact: A Multi-Dimensional, Multi-Field Benchmark for Scientific Impact Prediction

arXiv:2604.17141v1 Announce Type: new Abstract: The rapid growth of scientific literature calls for automated methods to assess and predict research impact. Prior work has largely focused on citation-

SeekerGym: A Benchmark for Reliable Information Seeking

arXiv:2604.17143v1 Announce Type: new Abstract: Despite their substantial successes, AI agents continue to face fundamental challenges in terms of trustworthiness. Consider deep research agents, taske

CTSCAN: Evaluation Leakage in Chest CT Segmentation and a Reproducible Patient-Disjoint Benchmark

arXiv:2604.15561v1 Announce Type: cross Abstract: Reported chest CT segmentation performance can be strongly inflated when train and test partitions mix slices from the same study. We present CTSCAN,

DASB -- Discrete Audio and Speech Benchmark

arXiv:2406.14294v3 Announce Type: replace-cross Abstract: Discrete audio tokens have recently gained considerable attention for their potential to bridge audio and language processing, enabling multim

From Benchmarking to Reasoning: A Dual-Aspect, Large-Scale Evaluation of LLMs on Vietnamese Legal Text

arXiv:2604.16270v1 Announce Type: cross Abstract: The complexity of Vietnam's legal texts presents a significant barrier to public access to justice. While Large Language Models offer a promising solu

HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning

arXiv:2604.15648v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) consistently require new arenas to guide their expanding boundaries, yet their capabilities with hypergraphs remain

IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering

arXiv:2510.23536v2 Announce Type: replace Abstract: Intent identification serves as the foundation for generating appropriate responses in personalized question answering (PQA). However, existing benc

MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications

arXiv:2511.13131v2 Announce Type: replace Abstract: Large Language Models (LLMs) have emerged as powerful tools for automating complex reasoning and decision-making tasks. In telecommunications, they

VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects

arXiv:2604.16272v1 Announce Type: cross Abstract: As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured

GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis

arXiv:2604.13888v1 Announce Type: new Abstract: The integration of Large Language Models (LLMs) into Geographic Information Systems (GIS) marks a paradigm shift toward autonomous spatial analysis. How

Tutorials

How to Correctly Make Mistakes: A Framework for Constructing and Benchmarking Mistake Aware Egocentric Procedural Videos

arXiv:2604.15134v1 Announce Type: new Abstract: Reliable procedural monitoring in video requires exposure to naturally occurring human errors and the recoveries that follow. In egocentric recordings,

tutorialsarxiv-cs-cv

LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligence

arXiv:2512.04578v3 Announce Type: replace Abstract: Legal general intelligence (GI) refers to artificial intelligence (AI) that encompasses legal understanding, reasoning, and decision-making, simulat

Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games

arXiv:2506.03610v3 Announce Type: replace Abstract: Large Language Model (LLM) agents are reshaping the game industry, by enabling more intelligent and human-preferable characters. Yet, current game b

PolyBench: Benchmarking LLM Forecasting and Trading Capabilities on Live Prediction Market Data

arXiv:2604.14199v1 Announce Type: cross Abstract: Predicting real-world events from live market signals demands systems that fuse qualitative news with quantitative order-book dynamics under strict te

XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics

arXiv:2604.14934v1 Announce Type: new Abstract: Automatic evaluation metrics are essential for building multilingual translation systems. The common practice of evaluating these systems is averaging m