Search: “benchmark”

XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics

arXiv:2604.14934v1 Announce Type: new Abstract: Automatic evaluation metrics are essential for building multilingual translation systems. The common practice of evaluating these systems is averaging m

17 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage

DGX agent

arXiv:2604.13060v1 Announce Type: new Abstract: Dental triage is a safety-critical clinical routing task that requires integrating multimodal clinical information (e.g., patient complaints and radiogr

model-releasesarxiv-cs-cl

16 Apr 2026

EgoEsportsQA: An Egocentric Video Benchmark for Perception and Reasoning in Esports

arXiv:2604.12320v1 Announce Type: cross Abstract: While video large language models (Video-LLMs) excel in understanding slow-paced, real-world egocentric videos, their capabilities in high-velocity, i

15 Apr 2026

HSG-12M: A Large-Scale Benchmark of Spatial Multigraphs from the Energy Spectra of Non-Hermitian Crystals

arXiv:2506.08618v4 Announce Type: replace-cross Abstract: AI is transforming scientific research by revealing new ways to understand complex physical systems, but its impact remains constrained by the

15 Apr 2026

LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics

arXiv:2604.12218v1 Announce Type: new Abstract: System log anomaly detection is critical for maintaining the reliability of large-scale software systems, yet traditional methods struggle with the hete

15 Apr 2026

A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation

arXiv:2604.10456v1 Announce Type: new Abstract: The surging demand for adapting long-form cinematic content into short videos has motivated the need for versatile automatic video compilation systems.

C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts

arXiv:2604.11796v1 Announce Type: cross Abstract: Recently, large language models (LLMs) are capable of generating highly fluent textual content. While they offer significant convenience to humans, th

CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics

arXiv:2512.21877v3 Announce Type: replace-cross Abstract: Cricket is the second most popular sport worldwide, with billions of fans seeking advanced statistical insights unavailable through standard w

DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain

arXiv:2604.10425v1 Announce Type: new Abstract: Recent advancements in Vision-Language Models (VLMs) have revolutionized general visual understanding. However, their application in the food domain rem

Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems

arXiv:2604.09666v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) and its graph-based extensions (GraphRAG) are effective paradigms for improving large language model (LLM) reason

Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models

arXiv:2601.03926v2 Announce Type: replace Abstract: The deployment of Large Vision-Language Models (LVLMs) for real-world document question answering is often constrained by dynamic, user-defined poli

Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts

arXiv:2604.10200v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) become integral to educational decision-making, ensuring their fairness is paramount. However, current text-centric eva

Exploring the best way for UAV visual localization under Low-altitude Multi-view Observation Condition: a Benchmark

arXiv:2503.10692v2 Announce Type: replace Abstract: Absolute Visual Localization (AVL) enables an Unmanned Aerial Vehicle (UAV) to determine its position in GNSS-denied environments by establishing ge

Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark

arXiv:2604.10580v1 Announce Type: new Abstract: Spoken meaning often depends not only on what is said, but also on which word is emphasized. The same sentence can convey correction, contrast, or clari

LookBench: A Live and Holistic Open Benchmark for Fashion Image Retrieval

arXiv:2601.14706v3 Announce Type: replace Abstract: In this paper, we present LookBench (We use the term 'look' to reflect retrieval that mirrors how people shop -- finding the exact item, a close sub

ReXSonoVQA: A Video QA Benchmark for Procedure-Centric Ultrasound Understanding

arXiv:2604.10916v1 Announce Type: cross Abstract: Ultrasound acquisition requires skilled probe manipulation and real-time adjustments. Vision-language models (VLMs) could enable autonomous ultrasound

Simulating Organized Group Behavior: New Framework, Benchmark, and Analysis

arXiv:2604.09874v1 Announce Type: new Abstract: Simulating how organized groups (e.g., corporations) make decisions (e.g., responding to a competitor's move) is essential for understanding real-world

SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

arXiv:2604.09557v1 Announce Type: cross Abstract: Speculative Decoding (SD) has emerged as a critical technique for accelerating Large Language Model (LLM) inference. Unlike deterministic system optim

SRBench: A Comprehensive Benchmark for Sequential Recommendation with Large Language Models

arXiv:2604.09553v1 Announce Type: cross Abstract: LLM development has aroused great interest in Sequential Recommendation (SR) applications. However, comprehensive evaluation of SR models remains lack

Unified Removal of Raindrops and Reflections: A New Benchmark and A Novel Pipeline

arXiv:2603.16446v3 Announce Type: replace Abstract: When capturing images through glass surfaces or windshields on rainy days, raindrops and reflections frequently co-occur to significantly reduce the

Compositional-Degradation UAV Image Restoration: Conditional Decoupled MoE Network and A Benchmark

arXiv:2604.09313v1 Announce Type: cross Abstract: UAV images are critical for applications such as large-area mapping, infrastructure inspection, and emergency response. However, in real-world flight

LuMon: A Comprehensive Benchmark and Development Suite with Novel Datasets for Lunar Monocular Depth Estimation

arXiv:2604.09352v1 Announce Type: new Abstract: Monocular Depth Estimation (MDE) is crucial for autonomous lunar rover navigation using electro-optical cameras. However, deploying terrestrial MDE netw

MedConceal: A Benchmark for Clinical Hidden-Concern Reasoning Under Partial Observability

arXiv:2604.08788v1 Announce Type: new Abstract: Patient-clinician communication is an asymmetric-information problem: patients often do not disclose fears, misconceptions, or practical barriers unless

R2G: A Multi-View Circuit Graph Benchmark Suite from RTL to GDSII

arXiv:2604.08810v1 Announce Type: new Abstract: Graph neural networks (GNNs) are increasingly applied to physical design tasks such as congestion prediction and wirelength estimation, yet progress is

SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment

arXiv:2604.08988v1 Announce Type: new Abstract: Current LLM-based agents demonstrate strong performance in episodic task execution but remain constrained by static toolsets and episodic amnesia, faili

TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice

arXiv:2604.08948v1 Announce Type: new Abstract: While Large Language Models (LLMs) excel in various general domains, they exhibit notable gaps in the highly specialized, knowledge-intensive, and legal

Temporal Dropout Risk in Learning Analytics: A Harmonized Survival Benchmark Across Dynamic and Early-Window Representations

arXiv:2604.08870v1 Announce Type: cross Abstract: Student dropout is a persistent concern in Learning Analytics, yet comparative studies frequently evaluate predictive models under heterogeneous proto

Riemann-Bench: A Benchmark for Moonshot Mathematics

arXiv:2604.06802v1 Announce Type: new Abstract: Recent AI systems have achieved gold-medal-level performance on the International Mathematical Olympiad, demonstrating remarkable proficiency at competi

10 Apr 2026

The Traveling Thief Problem with Time Windows: Benchmarks and Heuristics

arXiv:2604.06724v1 Announce Type: cross Abstract: While traditional optimization problems were often studied in isolation, many real-world problems today require interdependence among multiple optimiz

10 Apr 2026

Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions

arXiv:2511.16221v2 Announce Type: replace-cross Abstract: Despite their advanced reasoning capabilities, state-of-the-art Multimodal Large Language Models (MLLMs) still lack a core component of human

RLDF: A Region-Level Deepfake Dataset and Benchmark for Partial-Face Manipulation Detection

arXiv:2610.21438v1 Announce Type: new Abstract: Existing deepfake detection models are built upon manipu- lated images that are tampered with either exclusively fully swapped or fully synthesised face

VALCON: Benchmarking and Enhancing Value Consistency in Large Language Models via Dual-Role Variation

arXiv:2610.21304v1 Announce Type: new Abstract: Current value alignment evaluations for large language models (LLMs) heavily rely on output compliance, overlooking whether LLMs maintain value consiste

Agents

WEAVE: Weaving Real-World Code Evolution into Dependency-Structured Tasks for Benchmarking and Training Code Agents

arXiv:2610.22039v1 Announce Type: cross Abstract: Real-world software engineering rarely consists of isolated changes. Features, migrations, and systematic fixes often unfold across multiple pull requ

agentsarxiv-cs-cl

Distinguishing Process Change, Instrument Fault and Data-Path Corruption in Streaming Telemetry: A Reference Implementation and Benchmark

arXiv:2610.18904v1 Announce Type: cross Abstract: A reading departs from what a monitoring system expects for one of three reasons: the physical process changed, the instrument is failing, or the valu

15 Oct 2026

Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders

arXiv:2610.18879v1 Announce Type: new Abstract: Protein function annotation operates over an expanding label space, making class-incremental learning (CIL) a natural framework for incorporating newly

15 Oct 2026

ForceBench: Benchmarking Force Integration for Robotic Manipulation

arXiv:2610.19604v1 Announce Type: new Abstract: Force feedback provides vision-language-action (VLA) policies with physical observations that can guide action selection and execution. However, differe

15 Oct 2026

[AINews] AI Engineer NYC: Gemini 4 Argon, Vals AI Benchmarks

model-releaseslatent-space

Google’s Gemini 4 Argon is now generally available (GA), offering coding, knowledge‑work, enterprise‑workflow, and cyber‑defense features via the paid Gemini API, Google AI Pro/Ultra in the Gemini app

CryptoFlowBench: Benchmarking Multi-Hop Reasoning on Blockchains for Cryptocurrency Fund Tracing

arXiv:2610.16449v1 Announce Type: cross Abstract: Pseudonymous accounts and complex smart-contract ecosystems substantially hinder the tracing of illicit cryptocurrency fund flows. Existing Graph Neur

Hardware

Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Hardware-Validated Benchmarking Study of Forward-Only Algorithms

arXiv:2511.01061v2 Announce Type: replace Abstract: Backpropagation (BP) powers modern deep learning, but it must hold every activation in memory until a global backward sweep consumes it, and that sw

hardwarearxiv-cs-lg

Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework

arXiv:2610.17089v1 Announce Type: new Abstract: Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode. Yet reaching a goal may require

LabInstruct: Benchmarking Situated Instructional Video Generation for Lab Procedures

arXiv:2610.14020v1 Announce Type: new Abstract: Self-driving laboratories (SDLs) aim to automate the full experimental loop, from scientific decision-making to physical execution. Ideally, AI-generate

LynxScenes: A Large-Scale Real-World Benchmark for Outdoor Wheel-Legged Robot Perception, Navigation, Imagination and Reconstruction

arXiv:2610.18093v1 Announce Type: new Abstract: Wheel-legged robots operating in outdoor spaces shared with humans require accurate perception, socially compliant navigation, future-scene reasoning, a

OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains

arXiv:2610.17143v1 Announce Type: new Abstract: Large language model (LLM)-based multi-agent systems are increasingly applied across diverse domains, yet different applications require different forms

Agents

Pixels or Code? Benchmarking Instruction-Guided Visual Artifact Operation

arXiv:2610.16444v1 Announce Type: new Abstract: Visual artifacts such as charts, technical diagrams, webpages, and document pages underpin scientific, commercial, and engineering workflows. Editing th

agentsarxiv-cs-cv

RoboBugBench: Benchmarking Automated Repair of Real Robot Software Under an Audited Behavioral Oracle

arXiv:2610.16369v1 Announce Type: new Abstract: Language agents increasingly modify robot software, but whether they can repair it is unmeasured, because the domain supplies no oracle to inherit: a ro

Successful but Unsafe: An Analytical Benchmark for Revealing Collisions in Vision-and-Language Navigation

arXiv:2610.14682v1 Announce Type: new Abstract: Vision-and-language navigation (VLN) agents are typically evaluated by whether they reach a target and how efficiently they follow an instructed route,

Tanawpamana: Benchmarking Vision-Language Models on Philippine Cultural Heritage Sites

arXiv:2610.17596v1 Announce Type: new Abstract: Vision-language models are increasingly used to identify and describe places, yet their recognition and knowledge of specific cultures remain poorly und

AutoBio+: Benchmarking Dexterous Manipulation with Dynamic Experimental Phenomena in Realistic Laboratories

arXiv:2610.13643v1 Announce Type: new Abstract: Automating biological laboratories requires robots to perform dexterous tool interactions under realistic visual conditions while handling dynamic exper