Search: “benchmark”

RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

arXiv:2601.14289v2 Announce Type: replace-cross Abstract: Understanding research papers remains challenging for foundation models due to specialized scientific discourse and complex figures and tables

1 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation

DGX agent

arXiv:2604.26923v1 Announce Type: cross Abstract: LLMs have achieved strong results on both function-level code synthesis and repository-level code modification, yet a capability that falls between th

model-releasesarxiv-cs-cl

30 Apr 2026

Graph-based Semantic Calibration Network for Unaligned UAV RGBT Image Semantic Segmentation and A Large-scale Benchmark

arXiv:2604.26893v1 Announce Type: new Abstract: Fine-grained RGBT image semantic segmentation is crucial for all-weather unmanned aerial vehicle (UAV) scene understanding. However, UAV RGBT semantic s

30 Apr 2026

Human-in-the-Loop Benchmarking of Heterogeneous LLMs for Automated Competency Assessment in Secondary Level Mathematics

arXiv:2604.26607v1 Announce Type: new Abstract: As Competency-Based Education (CBE) is gaining traction around the world, the shift from marks-based assessment to qualitative competency mapping is a m

30 Apr 2026

LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation

arXiv:2603.06198v2 Announce Type: replace Abstract: Retrieval-Augmented Generation (RAG) is a framework in which a Generator, such as a Large Language Model (LLM), produces answers by retrieving docum

30 Apr 2026

Benchmarking and Improving GUI Agents in High-Dynamic Environments

arXiv:2604.25380v1 Announce Type: new Abstract: Recent advancements in Graphical User Interface (GUI) agents have predominantly focused on training paradigms like supervised fine-tuning (SFT) and rein

Benchmarking OCR Pipelines with Adaptive Enhancement for Multi-Domain Retail Bill Digitization

arXiv:2604.25176v1 Announce Type: new Abstract: The digitization of multi-domain retail billing documents remains a challenging task due to variability in scan quality, layout heterogeneity, and domai

DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams

arXiv:2604.25231v1 Announce Type: cross Abstract: Diagram question answering (DQA) requires models to interpret structured visual representations such as charts, maps, infographics, circuit schematics

DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios

arXiv:2604.25914v1 Announce Type: new Abstract: Real-world data visualization (DV) requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Yet, existing benc

EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling

model-releasesarxiv-cs-ro

arXiv:2604.25782v1 Announce Type: cross Abstract: Earth observation satellite imaging scheduling is a challenging NP-hard combinatorial optimisation problem central to space mission operations. While

PSP: An Interpretable Per-Dimension Accent Benchmark for Indic Text-to-Speech

arXiv:2604.25476v1 Announce Type: cross Abstract: Standard text-to-speech (TTS) evaluation measures intelligibility (WER, CER) and overall naturalness (MOS, UTMOS) but does not quantify accent. A synt

Benchmarking and Mitigating Sycophancy in Medical Vision Language Models

arXiv:2509.21979v4 Announce Type: replace-cross Abstract: Visual language models (VLMs) have the potential to transform medical workflows. However, the deployment is limited by sycophancy. Despite thi

Benchmarking Pathology Foundation Models for Breast Cancer Survival Prediction

arXiv:2604.24679v1 Announce Type: new Abstract: Pathology foundation models (PFMs) have recently emerged as powerful pretrained encoders for computational pathology, enabling transfer learning across

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

arXiv:2509.20374v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance across general NLP tasks, but their utility in automating numerical experime

CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning

arXiv:2601.14952v2 Announce Type: replace-cross Abstract: While large language models now handle million-token contexts, their capacity for reasoning across entire document repositories remains largel

CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation

arXiv:2604.24001v1 Announce Type: new Abstract: The evaluation of generated reports remains a critical challenge in Computed Tomography (CT) report generation, due to the large volume of text, the div

DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models

arXiv:2604.22822v1 Announce Type: cross Abstract: Object level hallucination remains a central reliability challenge for vision language models (VLMs), particularly in binary object existence verifica

Event-based SLAM Benchmark for High-Speed Maneuvers

model-releasesarxiv-cs-ro

arXiv:2604.24033v1 Announce Type: new Abstract: Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution,

SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning

arXiv:2506.05425v2 Announce Type: replace-cross Abstract: Understanding social interaction, which encompasses perceiving numerous and subtle multimodal cues, inferring unobservable mental states and r

SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning

model-releasesarxiv-cs-lg

arXiv:2604.24729v1 Announce Type: new Abstract: Specification-guided reinforcement learning (RL) provides a principled framework for encoding complex, temporally extended tasks using formal specificat

Sphere-Depth: A Benchmark for Depth Estimation Methods with Varying Spherical Camera Orientations

arXiv:2604.23432v1 Announce Type: cross Abstract: Reliable depth estimation from spherical images is crucial for 360{eg} vision in robotic navigation and immersive scene understanding. However, the on

SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents

arXiv:2512.07538v3 Announce Type: replace Abstract: Recognizing semantic differences across documents is crucial for text generation evaluation and content alignment, especially in cross-lingual setti

Safety

Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines

arXiv:2604.23001v1 Announce Type: cross Abstract: Despite remarkable progress in Vision--Language--Action (VLA) models, a central bottleneck remains underexamined: the data infrastructure that underli

safetyarxiv-cs-ai

FETS Benchmark: Foundation Models Outperform Dataset-specific Machine Learning in Energy Time Series Forecasting

arXiv:2604.22328v1 Announce Type: cross Abstract: Driven by the transition towards a climate-neutral energy system, accurate energy time series forecasting is critical for planning and operation. Yet,

27 Apr 2026

PSI: A Benchmark for Human Interpretation and Response in Traffic Interactions

arXiv:2112.02604v3 Announce Type: replace-cross Abstract: Accurately modeling pedestrian intention and understanding driver decision-making processes are critical for the development of safe and socia

27 Apr 2026

SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments

arXiv:2604.22409v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced static visual--spatial reasoning, yet they often fail to preserve long-horizon spatial coherence

27 Apr 2026

ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response

arXiv:2604.21199v1 Announce Type: cross Abstract: Time series question-answering (TSQA), in which we ask natural language questions to infer and reason about properties of time series, is a promising

Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision

arXiv:2604.21461v1 Announce Type: new Abstract: Egocentric AI agents, such as smart glasses, rely on pointing gestures to resolve referential ambiguities in natural language commands. However, despite

OptiVerse: A Comprehensive Benchmark towards Optimization Problem Solving

arXiv:2604.21510v1 Announce Type: new Abstract: While Large Language Models (LLMs) demonstrate remarkable reasoning, complex optimization tasks remain challenging, requiring domain knowledge and robus

RailVQA: A Benchmark and Framework for Efficient Interpretable Visual Cognition in Automatic Train Operation

arXiv:2603.27112v2 Announce Type: replace Abstract: As Automatic Train Operation (ATO) advances toward GoA4 and beyond, it increasingly depends on efficient, reliable cab-view visual perception and de

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

model-releasesarxiv-cs-lg

arXiv:2604.21696v1 Announce Type: new Abstract: Tabular foundation models aim to learn universal representations of tabular data that transfer across tasks and domains, enabling applications such as t

Agents

AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite

arXiv:2510.21652v2 Announce Type: replace Abstract: AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and

agentsarxiv-cs-ai

CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs

arXiv:2604.20460v1 Announce Type: new Abstract: Safety-critical traffic reasoning requires contrastive consistency: models must detect true hazards when an accident occurs, and reliably reject plausib

IVY-FAKE: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detection

arXiv:2506.00979v5 Announce Type: replace-cross Abstract: The rapid development of Artificial Intelligence Generated Content (AIGC) techniques has enabled the creation of high-quality synthetic conten

RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Ranking

arXiv:2604.20623v1 Announce Type: cross Abstract: Traditional change detection identifies where changes occur, but does not explain what changed in natural language. Existing remote sensing change cap

BALTIC: A Benchmark and Cross-Domain Strategy for 3D Reconstruction Across Air and Underwater Domains Under Varying Illumination

arXiv:2604.19133v1 Announce Type: new Abstract: Robust 3D reconstruction across varying environmental conditions remains a critical challenge for robotic perception, particularly when transitioning be

Benchmarking Quantum Kernel Support Vector Machines Against Classical Baselines on Tabular Data: A Rigorous Empirical Study with Hardware Validation

model-releasesarxiv-cs-lg

arXiv:2604.18837v1 Announce Type: cross Abstract: Quantum kernel methods have been proposed as a promising approach for leveraging near-term quantum computers for supervised learning, yet rigorous ben

Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences

arXiv:2601.04925v2 Announce Type: replace Abstract: Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmfu

From Natural Language to Executable Narsese: A Neuro-Symbolic Benchmark and Pipeline for Reasoning with NARS

arXiv:2604.18873v1 Announce Type: new Abstract: Large language models (LLMs) are highly capable at language generation, but they remain unreliable when reasoning requires explicit symbolic structure,

HarDBench: A Benchmark for Draft-Based Co-Authoring Jailbreak Attacks for Safe Human-LLM Collaborative Writing

arXiv:2604.19274v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as co-authors in collaborative writing, where users begin with rough drafts and rely on LLMs to compl

Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models

arXiv:2601.22737v2 Announce Type: replace Abstract: The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current

MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation

arXiv:2604.18914v1 Announce Type: cross Abstract: While multilingual large language models (LLMs) perform well on high-level tasks like translation and question answering, their ability to handle gram

Pixels or Positions? Benchmarking Modalities in Group Activity Recognition

arXiv:2511.12606v3 Announce Type: replace Abstract: Group Activity Recognition (GAR) is well studied on the video modality for surveillance and indoor team sports (e.g., volleyball, basketball). Yet,

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains

arXiv:2604.19001v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm

Adverse-to-the-eXtreme Panoptic Segmentation: URVIS 2026 Study and Benchmark

arXiv:2604.16984v1 Announce Type: new Abstract: This paper presents the report of the URVIS 2026 challenge on adverse-to-extreme panoptic segmentation. As the first challenge of its kind, it attracted

Camo-M3FD: A New Benchmark Dataset for Cross-Spectral Camouflaged Pedestrian Detection

arXiv:2604.16582v1 Announce Type: new Abstract: Pedestrian detection is fundamental to autonomous driving, robotics, and surveillance. Despite progress in deep learning, reliable identification remain

Chain Of Interaction Benchmark (COIN): When Reasoning meets Embodied Interaction

model-releasesarxiv-cs-ro

arXiv:2604.16886v1 Announce Type: new Abstract: Generalist embodied agents must perform interactive, causally-dependent reasoning, continually interacting with the environment, acquiring information,

Research

Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks

arXiv:2604.17761v1 Announce Type: cross Abstract: Interpretability tools are increasingly used to analyze failures of Large Language Models (LLMs), yet prior work largely focuses on short prompts or t

researcharxiv-cs-cl