Search: “benchmark”

When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution

arXiv:2605.14504v1 Announce Type: new Abstract: Long-horizon household tasks demand robust high-level planning and sustained reasoning capabilities, which are largely overlooked by existing embodied A

15 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

DRIFT: A Benchmark for Task-Free Continual Graph Learning with Continuous Distribution Shifts

DGX agent

arXiv:2605.12998v1 Announce Type: new Abstract: Continual graph learning (CGL) aims to learn from dynamically evolving graphs while mitigating catastrophic forgetting. Existing CGL approaches typicall

model-releasesarxiv-cs-lg

14 May 2026

Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling

arXiv:2605.13062v1 Announce Type: new Abstract: Recent image editing models have achieved remarkable progress in instruction following, multimodal understanding, and complex visual editing. However, e

Follow-Bench: A Unified Motion Planning Benchmark for Socially-Aware Robot Person Following

model-releasesarxiv-cs-ro

arXiv:2509.10796v4 Announce Type: replace Abstract: Robot person following (RPF) -- mobile robots that follow and assist a specific person -- has emerging applications in personal assistance, security

PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media

arXiv:2605.12525v1 Announce Type: cross Abstract: Current emotion analysis in social media is predominantly author-centric, failing to capture the subjective nature of emotional responses across diver

PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editors

arXiv:2605.13493v1 Announce Type: new Abstract: Can general-purpose image editors predict physical maps from a single RGB image? General-purpose image editors differ from standard task-specific dense-

RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation

arXiv:2605.13542v1 Announce Type: new Abstract: Intensive care units (ICU) generate long, dense and evolving streams of clinical information, where physicians must repeatedly reassess patient states u

SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management

arXiv:2602.07342v2 Announce Type: replace Abstract: Large language models (LLMs) have shown promise in complex reasoning and tool-based decision making, motivating their application to real-world supp

Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition

arXiv:2605.13087v1 Announce Type: cross Abstract: Fine-tuning multilingual ASR models like Whisper for low-resource languages often improves read speech but degrades spontaneous audio performance, a p

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark

arXiv:2605.12437v1 Announce Type: new Abstract: Retrospective novel view synthesis (NVS) of dynamic scenes is fundamental to applications such as sports. Recent dynamic 3D Gaussian Splatting (3DGS) ap

ASD-Bench: A Four-Axis Comprehensive Benchmark of AI Models for Autism Spectrum Disorder

arXiv:2605.11091v1 Announce Type: new Abstract: Automated ASD screening tools remain limited by single-architecture evaluations, axis-restricted assessment, and near-exclusive focus on adult cohorts,

B4DL: A Benchmark for 4D LiDAR LLM in Spatio-Temporal Understanding

arXiv:2508.05269v2 Announce Type: replace Abstract: Understanding dynamic outdoor environments requires capturing complex object interactions and their evolution over time. LiDAR-based 4D point clouds

Research

EpiCastBench: Datasets and Benchmarks for Multivariate Epidemic Forecasting

arXiv:2605.11598v1 Announce Type: new Abstract: The increasing adoption of data-driven decision-making in public health has established epidemic forecasting as a critical area of research. Recent adva

researcharxiv-cs-lg

gym-invmgmt: An Open Benchmarking Framework for Inventory Management Methods

arXiv:2605.11355v1 Announce Type: new Abstract: Inventory-policy comparisons are often difficult to interpret because performance depends on the evaluation contract as much as on the policy itself. Di

PreScam: A Benchmark for Predicting Scam Progression from Early Conversations

arXiv:2605.12243v1 Announce Type: new Abstract: Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lotter

XWOD: A Real-World Benchmark for Object Detection under Extreme Weather Conditions

arXiv:2605.11521v1 Announce Type: new Abstract: Autonomous driving and intelligent transportation systems remain vulnerable under extreme weather. The U.S. Federal Highway Administration reports that

AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents

arXiv:2605.10876v1 Announce Type: cross Abstract: Recent advances in machine learning and large-scale biological data collections have revived the prospect of building a virtual cell, a computational

ASTRA-QA: A Benchmark for Abstract Question Answering over Documents

arXiv:2605.10168v1 Announce Type: new Abstract: Document-based question answering (QA) increasingly includes abstract questions that require synthesizing scattered information from long documents or a

BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD

arXiv:2605.10865v1 Announce Type: new Abstract: Industrial Computer-Aided Design (CAD) code generation requires models to produce executable parametric programs from visual or textual inputs. Beyond r

Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption

arXiv:2605.09722v1 Announce Type: new Abstract: Obtaining an accurate short-term forecasting for heat demand is an essential part of operating district heating networks cost-efficient and reliable. He

Beyond Toy Benchmarks: A Systematic Evaluation of OOD Detection Methods For Plant Pathology Classification

arXiv:2605.08618v1 Announce Type: new Abstract: Out-of-distribution (OOD) detection is essential for reliable deployment of deep learning systems, yet the majority of existing methods are evaluated on

ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics

arXiv:2605.09676v1 Announce Type: cross Abstract: Spatio-temporal graph neural networks (STGNNs) are widely used for short-term forecasting in dynamic physical systems such as traffic and weather. How

DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules

arXiv:2605.08614v1 Announce Type: new Abstract: Monitoring complex industrial assets relies on engineer-authored symbolic rules that trigger based on sensor conditions and prompt technicians to perfor

Do Benchmarks Underestimate LLM Performance? Evaluating Hallucination Detection With LLM-First Human-Adjudicated Assessment

arXiv:2605.08462v1 Announce Type: cross Abstract: Hallucination remains a persistent challenge in Large Language Models (LLMs), particularly in context-grounded settings such as RAG and agentic AI sys

EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding

arXiv:2605.09874v1 Announce Type: cross Abstract: Next-generation visual assistants, such as smart glasses, embodied agents, and always-on life-logging systems, must reason over an entire day or more

EmoS: A High-Fidelity Multimodal Benchmark for Fine-grained Streaming Emotional Understanding

arXiv:2605.08847v1 Announce Type: new Abstract: In the context of today's high-pressure, aging society, the demand for large-scale emotional models capable of providing empathetic support is more crit

EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents

arXiv:2605.09826v1 Announce Type: new Abstract: Theory of Mind (ToM), the ability to track others epistemic state, makes humans efficient collaborators. AI agents need the same capacity in multi agent

EpiGraph: A Knowledge Graph and Benchmark for Evidence-Intensive Reasoning in Epilepsy

arXiv:2605.09505v1 Announce Type: new Abstract: Epilepsy diagnosis and treatment require evidence-intensive reasoning across heterogeneous clinical knowledge, including biosignal patterns, genetic mec

FormalRewardBench: A Benchmark for Formal Theorem Proving Reward Models

arXiv:2605.10141v1 Announce Type: new Abstract: Recent neural theorem provers use reinforcement learning with verifiable rewards (RLVR), where proof assistants provide binary correctness signals. Whil

FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence

arXiv:2605.08820v1 Announce Type: cross Abstract: Artificial Intelligence (AI)-generated images have become increasingly realistic and readily adaptable to concrete real-world claims, creating new cha

LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments

arXiv:2605.10779v1 Announce Type: cross Abstract: The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content s

M^2E-UAV: A Benchmark and Analysis for Onboard Motion-on-Motion Event-Based Tiny UAV Detection

arXiv:2605.10496v1 Announce Type: new Abstract: Tiny UAV detection from an onboard event camera is difficult when the observer and target move at the same time. In this motion-on-motion regime, ego-mo

MDGYM: Benchmarking AI Agents on Molecular Simulations

arXiv:2605.08941v1 Announce Type: new Abstract: The promise of AI-driven scientific discovery hinges on whether AI agents can autonomously design and execute the computational workflows that underpin

Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization

arXiv:2605.09996v1 Announce Type: new Abstract: While multimodal large language models have advanced across text, image, and audio, personalization research has remained primarily vision-language, wit

PolarVSR: A Unified Framework and Benchmark for Continuous Space-Time Polarization Video Reconstruction

arXiv:2605.10275v1 Announce Type: new Abstract: Polarimetric imaging captures surface polarization characteristics, such as the Degree of Linear Polarization (DoLP) and the Angle of Polarization (AoP)

Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmark

arXiv:2410.14702v2 Announce Type: replace Abstract: Multi-modal Large Language Models (MLLMs) exhibit impressive problem-solving abilities in various domains, but their visual comprehension and abstra

ReaMOT: A Benchmark and Framework for Reasoning-based Multi-Object Tracking

arXiv:2505.20381v4 Announce Type: replace Abstract: Referring Multi-Object Tracking (RMOT) aims to track targets specified by language instructions. However, existing RMOT paradigms heavily rely on ex

RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark

model-releasesarxiv-cs-ro

arXiv:2605.10921v1 Announce Type: new Abstract: Memory is a critical component of robotic intelligence, as robots must rely on past observations and actions to accomplish long-horizon tasks in partial

SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation

arXiv:2605.10187v1 Announce Type: new Abstract: Scientific reasoning is a key aspect of human intelligence, requiring the integration of multimodal inputs, domain expertise, and multi-step inference a

Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs

arXiv:2605.09063v1 Announce Type: new Abstract: Following the recent achievement of gold-medal performance on the IMO by frontier LLMs, the community is searching for the next meaningful and challengi

V4FinBench: Benchmarking Tabular Foundation Models, LLMs, and Standard Methods on Corporate Bankruptcy Prediction

arXiv:2605.10896v1 Announce Type: new Abstract: Corporate bankruptcy prediction is a high-stakes financial task characterized by severe class imbalance and multi-horizon forecasting demands. Public da

VidNum-1.4K: A Comprehensive Benchmark for Video-based Numerical Reasoning

arXiv:2604.03701v2 Announce Type: replace Abstract: Video-based numerical reasoning provides a premier arena for testing whether Vision-Language Models (VLMs) truly 'understand' real-world dynamics, a

VISTA: A Benchmark for Real-Time Video Streaming under Network Impairments in Surgical Teleoperation

model-releasesarxiv-cs-ro

arXiv:2605.08886v1 Announce Type: cross Abstract: Real-time video streaming is crucial in surgical teleoperation, yet reproducible evaluation under realistic network impairments remains limited. This

VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning

arXiv:2605.08146v1 Announce Type: cross Abstract: Multi-model learning has attracted great attention in visual-text tasks. However, visual-tabular data, which plays a pivotal role in high-stakes domai

Safety

When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents

arXiv:2605.08828v1 Announce Type: new Abstract: Large language model agents increasingly operate through environment-facing scaffolds that expose files, web pages, APIs, and logs. These observations i

safetyarxiv-cs-ai

WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation

arXiv:2605.10912v1 Announce Type: new Abstract: Large language and vision-language models increasingly power agents that act on a user's behalf through command-line interface (CLI) harnesses. However,

Beyond Retrieval: A Multitask Benchmark and Model for Code Search

arXiv:2605.04615v2 Announce Type: replace-cross Abstract: Code search has usually been evaluated as first-stage retrieval, even though production systems rely on broader pipelines with reranking and d

11 May 2026

Bridging the Last Mile of Circuit Design: PostEDA-Bench, a Hierarchical Benchmark for PPA Convergence and DRC Fixing

arXiv:2605.06936v1 Announce Type: cross Abstract: LLM-based agents are increasingly applied to the 'last mile' of Electronic Design Automation (EDA): repairing residual sign-off Design Rule Check (DRC