Search: “benchmark”

On Improving Multimodal Pedestrian Trajectory Prediction with CVAE: A Study on Benchmark and Robot Data

model-releasesarxiv-cs-ro

arXiv:2605.18262v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction is crucial for autonomous systems operating in complex environments, such as modular buses and delivery robots

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents

DGX agent

arXiv:2506.16042v2 Announce Type: replace Abstract: Generative AI is being leveraged to solve a variety of computer-use tasks involving desktop applications. State-of-the-art systems have focused sole

model-releasesarxiv-cs-ai

19 May 2026

PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts

arXiv:2605.17028v1 Announce Type: cross Abstract: Large language models (LLMs) hallucinate with confidence: their outputs can be fluent, authoritative, and simply wrong. In medical, legal, and scienti

REBAR: Reference Ethical Benchmark for Autonomy Readiness

model-releasesarxiv-cs-ro

arXiv:2605.18423v1 Announce Type: new Abstract: As autonomous systems grow more advanced, objective metrics to evaluate their ethical and legal compliance are critical for informing end users of their

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

arXiv:2603.04639v2 Announce Type: replace-cross Abstract: Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipula

SomaliWeb v1: A Quality-Filtered Somali Web Corpus with a Matched Tokenizer and a Public Language-Identification Benchmark

arXiv:2605.18232v1 Announce Type: cross Abstract: Somali is a Cushitic language of the Horn of Africa with ~25 million speakers, yet no documented dedicated Somali pretraining corpus with a companion

SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

arXiv:2605.17946v1 Announce Type: new Abstract: Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external too

Temporal Decay of Co-Citation Predictability: A 20-Year Statute Retrieval Benchmark from 396M Ukrainian Court Citations

arXiv:2605.17639v1 Announce Type: new Abstract: Co-citation structure is widely assumed to provide stable retrieval signal in legal information systems. We test this assumption longitudinally by const

UAVFF3D: A Geometry-Aware Benchmark for Feed-Forward UAV 3D Reconstruction

arXiv:2605.17942v1 Announce Type: new Abstract: Feed-forward 3D reconstruction has recently demonstrated strong generalization across diverse scenes, yet its performance in UAV imagery remains underex

UniPPTBench: A Unified Benchmark for Presentation Generation Across Diverse Input Settings

arXiv:2605.17356v1 Announce Type: new Abstract: Existing works typically focus on presentation generation under isolated input settings, whereas real-world use cases span diverse scenarios, including

Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning

arXiv:2601.06943v2 Announce Type: replace-cross Abstract: In real-world video question answering scenarios, videos often provide only localized visual cues, while verifiable answers are distributed ac

WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments

arXiv:2605.16402v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have revolutionized GUI automation, yet their efficacy is largely established on idealized, single-layer interf

ChronoEarth-492K: A Large Scale and Long Horizon Spatiotemporal Hyperspectral Earth Observation Dataset and Benchmark

arXiv:2605.15666v1 Announce Type: new Abstract: Hyperspectral imaging (HSI) provides dense spectral information for the Earth's surface, enabling material-level understanding of land cover and ecosyst

ELDOR: A Dataset and Benchmark for Illegal Gold Mining in the Amazon Rainforest

arXiv:2605.15397v1 Announce Type: new Abstract: Illegal gold mining in the Amazon rainforest causes deforestation, water contamination, and long-term ecosystem disruption, yet remains difficult to mon

MorphoHELM: A Comprehensive Benchmark for Evaluating Representations for Microscopy-Based Morphology Assays

arXiv:2605.15383v1 Announce Type: new Abstract: Microscopy images contain rich information about how cells respond to perturbations, making them essential to applications like drug screening. To quant

Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation

arXiv:2605.15669v1 Announce Type: new Abstract: Manufacturable chip layouts must satisfy thousands of geometry-based design rules, and design rule checking (DRC) enforces them by running executable DR

TSBOW -- Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditions

arXiv:2602.05414v2 Announce Type: replace Abstract: Global warming has intensified the frequency and severity of extreme weather events, which degrade CCTV signal and video quality while disrupting tr

ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents

arXiv:2605.14153v1 Announce Type: cross Abstract: Exploitation is not a binary event. It is a ladder of acquiring progressive capabilities, from executing a single buggy line of code to taking full co

Herculean: An Agentic Benchmark for Financial Intelligence

arXiv:2605.14355v1 Announce Type: new Abstract: As AI agents improve, the central question is no longer whether they can solve isolated well-defined financial tasks, but whether they can reliably carr

JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation

arXiv:2512.12772v2 Announce Type: replace-cross Abstract: Understanding videos inherently requires reasoning over both visual and auditory information. To properly evaluate Omni-Large Language Models

MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models

arXiv:2605.14906v1 Announce Type: new Abstract: Memory is essential for large vision-language models (LVLMs) to handle long, multimodal interactions, with two method directions providing this capabili

MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

arXiv:2510.23477v2 Announce Type: replace Abstract: Effective math tutoring requires not only solving problems but also diagnosing students' difficulties and guiding them step by step. While multimoda

Research

NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework

arXiv:2605.15058v1 Announce Type: cross Abstract: The rapid expansion of spiking neural networks (SNNs) has led to a proliferation of training algorithms that differ widely in biological inspiration,

researcharxiv-cs-ai

PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts

arXiv:2605.14002v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) embedded in agentic frameworks have transformed information retrieval from static, long context question answering into op

RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation

arXiv:2605.14543v1 Announce Type: cross Abstract: Inpatient medication recommendation requires clinicians to repeatedly select specific medications, doses, and routes as a patient's condition evolves.

SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades

arXiv:2605.14415v1 Announce Type: cross Abstract: Coding agents powered by large language models are increasingly expected to perform realistic software maintenance tasks beyond isolated issue resolut

Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks

arXiv:2605.14604v1 Announce Type: new Abstract: This position paper argues that effective tutoring requires corrective friction: surfacing misconceptions and challenging them supportively to drive con

TILBench: A Systematic Benchmark for Tabular Imbalanced Learning Across Data Regimes

arXiv:2605.14915v1 Announce Type: new Abstract: Imbalanced learning remains a fundamental challenge in tabular data applications. Despite decades of research and numerous proposed algorithms, a system

When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution

arXiv:2605.14504v1 Announce Type: new Abstract: Long-horizon household tasks demand robust high-level planning and sustained reasoning capabilities, which are largely overlooked by existing embodied A

DRIFT: A Benchmark for Task-Free Continual Graph Learning with Continuous Distribution Shifts

arXiv:2605.12998v1 Announce Type: new Abstract: Continual graph learning (CGL) aims to learn from dynamically evolving graphs while mitigating catastrophic forgetting. Existing CGL approaches typicall

Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling

arXiv:2605.13062v1 Announce Type: new Abstract: Recent image editing models have achieved remarkable progress in instruction following, multimodal understanding, and complex visual editing. However, e

Follow-Bench: A Unified Motion Planning Benchmark for Socially-Aware Robot Person Following

model-releasesarxiv-cs-ro

arXiv:2509.10796v4 Announce Type: replace Abstract: Robot person following (RPF) -- mobile robots that follow and assist a specific person -- has emerging applications in personal assistance, security

PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media

arXiv:2605.12525v1 Announce Type: cross Abstract: Current emotion analysis in social media is predominantly author-centric, failing to capture the subjective nature of emotional responses across diver

PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editors

arXiv:2605.13493v1 Announce Type: new Abstract: Can general-purpose image editors predict physical maps from a single RGB image? General-purpose image editors differ from standard task-specific dense-

RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation

arXiv:2605.13542v1 Announce Type: new Abstract: Intensive care units (ICU) generate long, dense and evolving streams of clinical information, where physicians must repeatedly reassess patient states u

SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management

arXiv:2602.07342v2 Announce Type: replace Abstract: Large language models (LLMs) have shown promise in complex reasoning and tool-based decision making, motivating their application to real-world supp

Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition

arXiv:2605.13087v1 Announce Type: cross Abstract: Fine-tuning multilingual ASR models like Whisper for low-resource languages often improves read speech but degrades spontaneous audio performance, a p

3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark

arXiv:2605.12437v1 Announce Type: new Abstract: Retrospective novel view synthesis (NVS) of dynamic scenes is fundamental to applications such as sports. Recent dynamic 3D Gaussian Splatting (3DGS) ap

ASD-Bench: A Four-Axis Comprehensive Benchmark of AI Models for Autism Spectrum Disorder

arXiv:2605.11091v1 Announce Type: new Abstract: Automated ASD screening tools remain limited by single-architecture evaluations, axis-restricted assessment, and near-exclusive focus on adult cohorts,

B4DL: A Benchmark for 4D LiDAR LLM in Spatio-Temporal Understanding

arXiv:2508.05269v2 Announce Type: replace Abstract: Understanding dynamic outdoor environments requires capturing complex object interactions and their evolution over time. LiDAR-based 4D point clouds

Research

EpiCastBench: Datasets and Benchmarks for Multivariate Epidemic Forecasting

arXiv:2605.11598v1 Announce Type: new Abstract: The increasing adoption of data-driven decision-making in public health has established epidemic forecasting as a critical area of research. Recent adva

researcharxiv-cs-lg

gym-invmgmt: An Open Benchmarking Framework for Inventory Management Methods

arXiv:2605.11355v1 Announce Type: new Abstract: Inventory-policy comparisons are often difficult to interpret because performance depends on the evaluation contract as much as on the policy itself. Di

PreScam: A Benchmark for Predicting Scam Progression from Early Conversations

arXiv:2605.12243v1 Announce Type: new Abstract: Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lotter

XWOD: A Real-World Benchmark for Object Detection under Extreme Weather Conditions

arXiv:2605.11521v1 Announce Type: new Abstract: Autonomous driving and intelligent transportation systems remain vulnerable under extreme weather. The U.S. Federal Highway Administration reports that

AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents

arXiv:2605.10876v1 Announce Type: cross Abstract: Recent advances in machine learning and large-scale biological data collections have revived the prospect of building a virtual cell, a computational

ASTRA-QA: A Benchmark for Abstract Question Answering over Documents

arXiv:2605.10168v1 Announce Type: new Abstract: Document-based question answering (QA) increasingly includes abstract questions that require synthesizing scattered information from long documents or a

BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD

arXiv:2605.10865v1 Announce Type: new Abstract: Industrial Computer-Aided Design (CAD) code generation requires models to produce executable parametric programs from visual or textual inputs. Beyond r

Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption

arXiv:2605.09722v1 Announce Type: new Abstract: Obtaining an accurate short-term forecasting for heat demand is an essential part of operating district heating networks cost-efficient and reliable. He