Search: “benchmark”

UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model

arXiv:2605.04409v1 Announce Type: new Abstract: Remote Sensing Image Change Captioning (RSICC) aims to generate spatially grounded natural language descriptions of scene evolution from bi-temporal ima

7 May 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

A Benchmark for Interactive World Models with a Unified Action Generation Framework

DGX agent

arXiv:2605.03941v1 Announce Type: new Abstract: Achieving Artificial General Intelligence (AGI) requires agents that learn and interact adaptively, with interactive world models providing scalable env

model-releasesarxiv-cs-cv

6 May 2026

Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks

arXiv:2605.03759v1 Announce Type: new Abstract: While Large Vision-Language Models (LVLMs) offer powerful capabilities, they pose privacy risks by unintentionally memorizing sensitive personal informa

Local Ai

Benchmarking Local Language Models for Social Robots using Edge Devices

arXiv:2605.03111v1 Announce Type: cross Abstract: Social-educational robots designed for socially interactive pedagogical support, such as the Robot Study Companion (RSC), rely on responsive, privacy-

local-aiarxiv-cs-cl

DataClaw: A Process-Oriented Agent Benchmark for Exploratory Real-World Data Analysis

arXiv:2605.02503v1 Announce Type: new Abstract: Evaluating autonomous data analysis agents requires testing their ability to perform exploratory analysis in underexplored data environments. However, m

EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solving

arXiv:2509.17677v2 Announce Type: replace Abstract: Large language models (LLMs) have shown strong performance on mathematical reasoning under well-defined conditions. However, real-world engineering

From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level

arXiv:2601.03731v3 Announce Type: replace-cross Abstract: As large language models (LLMs) evolve into autonomous agents, evaluating repository-level reasoning, the ability to maintain logical consiste

MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers

arXiv:2602.00933v2 Announce Type: replace-cross Abstract: The Model Context Protocol (MCP) is rapidly becoming the standard interface for Large Language Models (LLMs) to discover and invoke external t

MHPR: Multidimensional Human Perception and Reasoning Benchmark for Large Vision-Languate Models

arXiv:2605.03485v1 Announce Type: new Abstract: Multidimensional human understanding is essential for real-world applications such as film analysis and virtual digital humans, yet current LVLM benchma

ReLeaf: Benchmarking Leaf Segmentation across Domains and Species

arXiv:2605.03784v1 Announce Type: new Abstract: Rising global food demand and growing climate pressure increase the need for sustainable, precise agricultural practices. Automated, individualized plan

Benchmarking Single-Pose Docking, Consensus Rescoring, and Supervised ML on the LIT-PCBA Library: A Critical Evaluation of DiffDock, AutoDock-GPU, GNINA, and DiffDock-NMDN

arXiv:2605.01681v1 Announce Type: new Abstract: Virtual screening performance depends heavily on the chosen docking and scoring methods. Recent AI-based tools such as DiffDock and NMDN have reported s

Differentiable Multiphysics Co-Optimization via Implicit Neural Representations: A Transient Hamburger-Cooking Benchmark

arXiv:2605.01040v1 Announce Type: cross Abstract: The co-optimization of geometry and physical parameters remains challenging in transient multiphysics systems involving moving boundaries, nonlinear m

Extreme Weather Bench: A framework and benchmark for evaluation of high-impact weather

arXiv:2605.01126v1 Announce Type: new Abstract: Forecasting the wide variety of high-impact weather events experienced globally is a challenge for both Artificial Intelligence (AI) and Numerical Weath

GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models

arXiv:2605.01203v1 Announce Type: cross Abstract: Currently, process reward models (PRMs) have exhibited remarkable potential for test-time scaling. Since large language models (LLMs) regularly genera

LUMINA: A Grid Foundation Model for Benchmarking AC Optimal Power Flow Surrogate Learning

arXiv:2605.02133v1 Announce Type: new Abstract: AC optimal power flow (ACOPF) is foundational yet computationally expensive in power grid operations, driving learning-based surrogates for large-scale

PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction

arXiv:2605.01945v1 Announce Type: new Abstract: Tandem mass spectrometry provides a high-throughput framework for identifying and quantifying proteins in complex biological samples. In computational p

RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferences

arXiv:2605.01831v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback has become the standard paradigm for language model alignment, where reward models directly determine alignme

Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark

arXiv:2603.27259v2 Announce Type: replace Abstract: Long video understanding (LVU) remains a core challenge in multimodal learning. Although recent vision-language models (VLMs) have made notable prog

SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation

arXiv:2508.15658v5 Announce Type: replace Abstract: The rapid growth of academic literature makes the manual creation of scientific surveys increasingly infeasible. While large language models show pr

Towards High Fidelity Face Swapping: A Comprehensive Survey and New Benchmark

arXiv:2605.00883v1 Announce Type: new Abstract: Face swapping has witnessed significant progress in recent years, largely driven by advances in deep generative models such as GANs and diffusion models

BanglaSocialBench: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interaction

arXiv:2603.15949v3 Announce Type: replace Abstract: Large Language Models have demonstrated strong multilingual fluency, yet fluency alone does not guarantee socially appropriate language use. In high

ControBench: An Interaction-Aware Benchmark for Controversial Discourse Analysis on Social Networks

arXiv:2605.00513v1 Announce Type: new Abstract: Understanding how people argue across ideological divides online is important for studying political polarization, misinformation, and content moderatio

Research

Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues

arXiv:2605.00119v1 Announce Type: new Abstract: There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. M

researcharxiv-cs-cl

Foresight Arena: An On-Chain Benchmark for Evaluating AI Forecasting Agents

arXiv:2605.00420v1 Announce Type: cross Abstract: Evaluating the true forecasting ability of AI agents requires environments resistant to overfitting, free from centralized trust, and grounded in ince

AppTek Call-Center Dialogues: A Multi-Accent Long-Form Benchmark for English ASR

arXiv:2604.27543v1 Announce Type: new Abstract: Evaluating English ASR systems for conversational AI applications remains difficult, as many publicly available corpora are either pre-segmented into sh

Assessing Pancreatic Ductal Adenocarcinoma Vascular Invasion: the PDACVI Benchmark

arXiv:2604.27582v1 Announce Type: new Abstract: Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate asse

Do Papers Tell the Whole Story? A Benchmark and Framework for Uncovering Hidden Implementation Gaps in Bioinformatics

arXiv:2603.22018v2 Announce Type: replace Abstract: Ensuring consistency between research papers and their corresponding software code implementations is a fundamental prerequisite for guaranteeing th

Research

From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks

arXiv:2604.27453v1 Announce Type: new Abstract: Large language models have achieved remarkable progress in text generation but still struggle with generative writing tasks. In terms of evaluation, exi

researcharxiv-cs-cl

RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

arXiv:2601.14289v2 Announce Type: replace-cross Abstract: Understanding research papers remains challenging for foundation models due to specialized scientific discourse and complex figures and tables

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation

arXiv:2604.26923v1 Announce Type: cross Abstract: LLMs have achieved strong results on both function-level code synthesis and repository-level code modification, yet a capability that falls between th

Graph-based Semantic Calibration Network for Unaligned UAV RGBT Image Semantic Segmentation and A Large-scale Benchmark

arXiv:2604.26893v1 Announce Type: new Abstract: Fine-grained RGBT image semantic segmentation is crucial for all-weather unmanned aerial vehicle (UAV) scene understanding. However, UAV RGBT semantic s

Human-in-the-Loop Benchmarking of Heterogeneous LLMs for Automated Competency Assessment in Secondary Level Mathematics

arXiv:2604.26607v1 Announce Type: new Abstract: As Competency-Based Education (CBE) is gaining traction around the world, the shift from marks-based assessment to qualitative competency mapping is a m

LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation

arXiv:2603.06198v2 Announce Type: replace Abstract: Retrieval-Augmented Generation (RAG) is a framework in which a Generator, such as a Large Language Model (LLM), produces answers by retrieving docum

Benchmarking and Improving GUI Agents in High-Dynamic Environments

arXiv:2604.25380v1 Announce Type: new Abstract: Recent advancements in Graphical User Interface (GUI) agents have predominantly focused on training paradigms like supervised fine-tuning (SFT) and rein

Benchmarking OCR Pipelines with Adaptive Enhancement for Multi-Domain Retail Bill Digitization

arXiv:2604.25176v1 Announce Type: new Abstract: The digitization of multi-domain retail billing documents remains a challenging task due to variability in scan quality, layout heterogeneity, and domai

DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams

arXiv:2604.25231v1 Announce Type: cross Abstract: Diagram question answering (DQA) requires models to interpret structured visual representations such as charts, maps, infographics, circuit schematics

DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios

arXiv:2604.25914v1 Announce Type: new Abstract: Real-world data visualization (DV) requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Yet, existing benc

EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling

model-releasesarxiv-cs-ro

arXiv:2604.25782v1 Announce Type: cross Abstract: Earth observation satellite imaging scheduling is a challenging NP-hard combinatorial optimisation problem central to space mission operations. While

PSP: An Interpretable Per-Dimension Accent Benchmark for Indic Text-to-Speech

arXiv:2604.25476v1 Announce Type: cross Abstract: Standard text-to-speech (TTS) evaluation measures intelligibility (WER, CER) and overall naturalness (MOS, UTMOS) but does not quantify accent. A synt

Benchmarking and Mitigating Sycophancy in Medical Vision Language Models

arXiv:2509.21979v4 Announce Type: replace-cross Abstract: Visual language models (VLMs) have the potential to transform medical workflows. However, the deployment is limited by sycophancy. Despite thi

Benchmarking Pathology Foundation Models for Breast Cancer Survival Prediction

arXiv:2604.24679v1 Announce Type: new Abstract: Pathology foundation models (PFMs) have recently emerged as powerful pretrained encoders for computational pathology, enabling transfer learning across

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

arXiv:2509.20374v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance across general NLP tasks, but their utility in automating numerical experime

CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning

arXiv:2601.14952v2 Announce Type: replace-cross Abstract: While large language models now handle million-token contexts, their capacity for reasoning across entire document repositories remains largel

CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation

arXiv:2604.24001v1 Announce Type: new Abstract: The evaluation of generated reports remains a critical challenge in Computed Tomography (CT) report generation, due to the large volume of text, the div

DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models

arXiv:2604.22822v1 Announce Type: cross Abstract: Object level hallucination remains a central reliability challenge for vision language models (VLMs), particularly in binary object existence verifica

Event-based SLAM Benchmark for High-Speed Maneuvers

model-releasesarxiv-cs-ro

arXiv:2604.24033v1 Announce Type: new Abstract: Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution,

SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning

arXiv:2506.05425v2 Announce Type: replace-cross Abstract: Understanding social interaction, which encompasses perceiving numerous and subtle multimodal cues, inferring unobservable mental states and r

SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning

arXiv:2604.24729v1 Announce Type: new Abstract: Specification-guided reinforcement learning (RL) provides a principled framework for encoding complex, temporally extended tasks using formal specificat