Search: “benchmark”

CzechTopic: A Benchmark for Zero-Shot Topic Localization in Historical Czech Documents

arXiv:2603.03884v2 Announce Type: replace-cross Abstract: Topic localization aims to identify spans of text that express a given topic defined by a name and description. To study this task, we introdu

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

NanoBench: A Multi-Task Benchmark Dataset for Nano-Quadrotor System Identification, Control, and State Estimation

DGX agent

arXiv:2603.09908v2 Announce Type: replace Abstract: Existing aerial-robotics benchmarks target vehicles from hundreds of grams to several kilograms and typically expose only high-level state data. The

model-releasesarxiv-cs-ro

17 Sept 2026

PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics

arXiv:2505.23126v5 Announce Type: replace Abstract: Although many benchmarks evaluate the reasoning abilities of Large Language Models (LLMs) within domains such as mathematics, coding, or data wrangl

Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failures

arXiv:2609.18396v1 Announce Type: cross Abstract: Virtual sensing, the estimation of hard-to-measure quantities from available sensor measurements, is a critical enabler for control and monitoring in

RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents

arXiv:2609.17985v1 Announce Type: new Abstract: AI agents are usually evaluated by whether they complete a task. In interactive service settings, a successful agent can still frustrate users by asking

StableEval Arena: A Cost-Aware Agentic Benchmark for Stablecoin Price Stability Prediction

arXiv:2609.18949v1 Announce Type: cross Abstract: We introduce StableEval Arena, a cost-aware benchmark framework for evaluating agentic AI systems on stablecoin peg-risk prediction. StableEval Arena

DisasterInsight: A Multimodal Benchmark for Function-Aware and Grounded Disaster Assessment

arXiv:2601.18493v2 Announce Type: replace Abstract: Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-cen

GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing

arXiv:2603.12264v2 Announce Type: replace Abstract: Unified multimodal models target joint understanding, reasoning, and generation, but current image editing benchmarks are largely confined to natura

PARSA-Bench: A Comprehensive Persian Audio-Language Model Benchmark

arXiv:2603.14456v2 Announce Type: replace Abstract: Persian poses unique audio understanding challenges through its classical poetry, traditional music, and pervasive code-switching, none of which is

VPRef: A Cross-Domain Benchmark for Referring Remote Sensing Image Segmentation

arXiv:2609.16486v1 Announce Type: new Abstract: Rapid advancements in vision-language models have propelled Referring Remote Sensing Image Segmentation (RRSIS) to the forefront of Earth observation. H

MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use

arXiv:2512.24565v4 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly serving as autonomous agents, and their utilization of external tools via the Model Context Protocol (

15 Sept 2026

PuzzleMate: Benchmarking MLLMs for Egocentric Puzzle Assistance

arXiv:2609.14473v1 Announce Type: new Abstract: Personal AI assistants hold the potential to evolve from digital interfaces into embodied companions capable of guiding users through complex physical a

15 Sept 2026

EgoMaize: A First-Person Maize Instance Segmentation Benchmark under Severe Field Occlusion

arXiv:2609.12350v1 Announce Type: new Abstract: Close-range first-person field images are important for mobile maize phenotyping because many plant-level traits depend on in-canopy structures that are

Research

Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage

arXiv:2609.13003v1 Announce Type: new Abstract: Binary-choice truth benchmarks ask models to choose between a correct and an incorrect answer, but if the two answers differ systematically in surface-l

researcharxiv-cs-cl

MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification

arXiv:2609.12884v1 Announce Type: new Abstract: A medical claim's correctness often depends not on the claim alone, but on the clinical structure around it. A claim may require a lab reference range,

MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation

arXiv:2512.14691v3 Announce Type: replace Abstract: Modern multimodal generative models can synthesize visually compelling images and videos, but it remains unclear whether this visual fluency reflect

ParaRecover: A Process-Level Benchmark for Error Localization and Recovery in Parallel Tool-Use Agents

arXiv:2609.12345v1 Announce Type: new Abstract: Existing agent benchmarks mainly evaluate final task success or tool-call correctness, providing limited insight into whether agents can reliably diagno

NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment

arXiv:2609.11234v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in peer review at major AI conferences, yet novelty remains a persistent weak point. Existing benchma

12 Sept 2026

What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead

arXiv:2609.10962v1 Announce Type: cross Abstract: Studies of the Model Context Protocol (MCP) server ecosystem draw their samples in ways that quietly select for servers that work: reference sets, pop

12 Sept 2026

From Evaluation to Enhancement: Benchmarking and Improving Think-with-Video Reasoning for Video Generative Models

arXiv:2609.11242v1 Announce Type: new Abstract: Video generation has advanced to produce visually compelling and temporally coherent results. Yet, whether these models can genuinely think with video--

11 Sept 2026

Leveraging Avatar Fingerprinting: A Multi-Generator Photorealistic Talking-Head Public Database and Benchmark

arXiv:2603.26934v2 Announce Type: replace Abstract: Recent advances in photorealistic avatar generation have enabled highly realistic talking-head avatars, raising security concerns regarding identity

11 Sept 2026

LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents

arXiv:2609.09754v1 Announce Type: new Abstract: As large language models are increasingly deployed as tool-augmented legal agents, they introduce agentic hallucinations where tool-call and reasoning e

11 Sept 2026

AgroVisNet: A lightweight Convolutional Network and the BD-PlantDX Expert-Validated Benchmark for Radish, Potato and Pointed Gourd Disease Classification

arXiv:2609.10469v1 Announce Type: new Abstract: Automated plant disease diagnosis is increasingly deployed on farmer-held devices in regions where agronomic expertise is scarce and network connectivit

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

arXiv:2609.09410v1 Announce Type: new Abstract: While autonomous agents have made significant strides in 'deep research' by iteratively navigating the open web to synthesize information, real-world pr

Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation

arXiv:2609.08275v1 Announce Type: new Abstract: Recent multi-shot audio-video generators can produce increasingly coherent and cinematic outputs, but coherence does not imply the ability to execute ed

Beyond Final Decisions: A Process-Centric Benchmark for Transparent AI-Assisted Peer Review

arXiv:2609.05947v1 Announce Type: new Abstract: Peer review is central to quality control in science. However, existing evaluations of AI-assisted peer review mainly focus on the overall quality of ge

CIPHER: Benchmarking Cross-record Inference over Privacy-Hardened Evidence Records

arXiv:2609.07022v1 Announce Type: cross Abstract: Reasoning over privacy-constrained records requires combining structured attributes with evidence from free-text narratives. We introduce CIPHER (Cros

FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?

arXiv:2609.07603v1 Announce Type: new Abstract: Financial scenarios are diverse and complex, spanning varying data conditions, tool configurations, and workflows. Yet existing CUA, Computer-Using Agen

NormViz: A Benchmark and Framework for Grounding Multimodal Reasoning in Global Cultures

arXiv:2609.06831v1 Announce Type: new Abstract: AI systems are used worldwide, but they struggle to serve the needs of culturally diverse populations. Prior work on cultural understanding evaluates AI

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs

arXiv:2609.06149v1 Announce Type: cross Abstract: Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat

TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs

arXiv:2609.08226v1 Announce Type: new Abstract: Temporal graph learning models the evolution of dynamic systems, where both structural interactions and semantic states change over time. However, exist

Benchmarking Storage Systems for Machine Learning Workloads Using NIO Bench

arXiv:2609.05418v1 Announce Type: cross Abstract: Machine learning training workloads place unique demands on storage systems, yet most existing benchmarks focus on computational throughput rather tha

Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras

arXiv:2609.06928v1 Announce Type: new Abstract: Emotion analysis is a fundamental task in computer vision, but its practical deployment remains constrained by the privacy risks inherent to conventiona

How Long Until Your Robot Ignores You? A Safety Benchmark for LLM Orchestrators in Human-Humanoid Collaboration

model-releasesarxiv-cs-ro

arXiv:2609.07288v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly employed to orchestrate robot behavior through natural-language interfaces, yet no benchmark exists to eva

MLIP Detective: Active Failure Mode Discovery Beyond Benchmark Scores for Machine-Learning Interatomic Potentials

arXiv:2609.08399v1 Announce Type: new Abstract: Universal machine-learning interatomic potentials (u-MLIPs) aim to generalize across diverse configurations. Benchmarks enable reproducible evaluation b

ONE CYLinder: A Benchmark for Graph-Based Surrogate Modeling of Unsteady Bluff-Body Flows

arXiv:2609.08947v1 Announce Type: cross Abstract: Graph-based surrogate models offer a promising route to accelerate computational fluid dynamics (CFD) simulations on unstructured meshes. However, the

Unexplored flaws in multiple-choice VQA make benchmarking unreliable

arXiv:2511.22341v2 Announce Type: replace Abstract: Previous works identify sensitivity to option order as a key issue in multiple-choice VQA (MC-VQA) evaluation and propose protocols to mitigate this

A Cost-Aware Agentic Architecture for NL-to-SQL over Nested Enterprise Schemas, with a New Benchmark

arXiv:2609.04641v1 Announce Type: new Abstract: Natural-language-to-SQL systems have ad- vanced rapidly on academic benchmarks, yet production enterprise schemas exhibit graph- like, semi-structured,

7 Sept 2026

WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

arXiv:2609.05405v1 Announce Type: new Abstract: Recent advances in wearable sensing enable continuous monitoring of physiological and behavioral signals, yet existing benchmarks rarely evaluate whethe

7 Sept 2026

Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements

arXiv:2609.03654v1 Announce Type: cross Abstract: The comparative analysis of banks' financial statements poses significant challenges for automated question answering systems due to their complexity,

GPS-Bench: A Governance Policy Benchmark for Automating Policy Analysis

arXiv:2609.03553v1 Announce Type: new Abstract: Policy analysis requires more than predicting whether a proposal will pass: it requires identifying who will be affected, how those actors respond, and

HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

arXiv:2609.03580v1 Announce Type: new Abstract: The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but u

LexIssue: Benchmarking Legal Issue Identification in Chinese Civil Litigation

arXiv:2609.02954v1 Announce Type: new Abstract: Identifying the issues disputed between litigating parties is a crucial component of real-world litigation. However, legal issues remain comparatively u

LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues

arXiv:2609.03507v1 Announce Type: cross Abstract: Tracking depression from multi-session counseling dialogues requires estimating both current symptom severity and how it changes across sessions. Yet

MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT

arXiv:2609.03690v1 Announce Type: new Abstract: The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advanc

Persistent Identity Preservation in Generative Image Models: A Benchmark and Evaluation System

arXiv:2609.04151v1 Announce Type: new Abstract: Generative image models can now produce high-quality images, follow complex instructions, and support precise edits, but they still struggle to preserve

Hardware

RobustSeiz: An Open-Source Framework for Benchmarking the Robustness of EEG Seizure Detection Models

arXiv:2609.04007v1 Announce Type: new Abstract: Despite strong performance on held-out electroencephalography (EEG) data, seizure detectors may fail under real-world acquisition variability, artifacts

hardwarearxiv-cs-lg

Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications

arXiv:2609.03804v1 Announce Type: new Abstract: Communities are fundamental spatial units that shape urban form and social life. Whether a residential compound is spatially open or enclosed affects mo