Search: “benchmark”

CogScale: Scalable Benchmark for Sequence Processing

arXiv:2605.19758v1 Announce Type: new Abstract: The ability to maintain and manipulate information over time is a fundamental aspect of living beings and Artificial Intelligence. While modern models h

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing

DGX agent

arXiv:2605.19484v1 Announce Type: cross Abstract: While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workfl

model-releasesarxiv-cs-ai

20 May 2026

Research

Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models

arXiv:2605.19859v1 Announce Type: new Abstract: Vision-language models (VLMs) have rapidly evolved into general-purpose multimodal reasoners with strong zero-shot generalization. In this context, VLMs

researcharxiv-cs-cv

MixRea: Benchmarking Explicit-Implicit Reasoning in Large Language Models

arXiv:2605.20128v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly integrated into high-stakes decision-making. Inspired by the theory of inattentional blindness in human co

POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents

arXiv:2605.19127v1 Announce Type: new Abstract: LLM agents increasingly have access to private user data and act on the user's behalf when interacting with third-party systems. The user defines what m

The Evaluation Game: Beyond Static LLM Benchmarking

arXiv:2605.19377v1 Announce Type: cross Abstract: As jailbreaks, adversarially crafted inputs that bypass safety constraints, continue to be discovered in Large Language Models, practitioners increasi

XNote: Benchmarking Automated Community Notes Generation for Image-based Contextual Deception

arXiv:2603.22453v2 Announce Type: replace Abstract: Community Notes have emerged as an effective crowd-sourced mechanism for combating online deception on social media platforms. However, its reliance

A Comprehensive Survey of Action Quality Assessment: Method and Benchmark

arXiv:2412.11149v2 Announce Type: replace Abstract: Action Quality Assessment (AQA) aims to automatically evaluate how well human actions are performed and has been widely applied in sports analysis,

A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generation

arXiv:2605.17278v1 Announce Type: new Abstract: Abstract reasoning ability reflects the intelligence and generalization capacity of LLMs to extract and apply abstract rules. However, accurately measur

AgroCoT: A Chain-of-Thought Benchmark for Evaluating Reasoning in Vision-Language Models for Agriculture

arXiv:2511.23253v3 Announce Type: replace Abstract: Recent advancements in Vision-Language Models (VLMs) have significantly impacted various industries. In agriculture, these multimodal capabilities h

Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models

arXiv:2605.18504v1 Announce Type: new Abstract: Machine Translation (MT) for Ancient Greek (AG) to Modern Greek (MG) is a low-resource task, constrained by the lack of large-scale, high-quality parall

BlendedNet++: A dataset and benchmark for field-resolved aerodynamics and inverse design of blended wing body aircraft

arXiv:2512.03280v2 Announce Type: replace-cross Abstract: The conceptual design of Blended Wing Body (BWB) aircraft is often constrained by the high computational cost of resolving complex aerodynamic

CANSURF: An ASV-View Can Dataset and Benchmark for Detection and Tracking of Surface-Level Debris

arXiv:2605.16774v1 Announce Type: cross Abstract: Surface-level marine debris remains a practical bottleneck for autonomous clean-up, where small, reflective targets (e.g., aluminum cans) must be dete

Causely: A Causal Intelligence Layer for Enterprise AI A Benchmark Study on SRE and Reliability Workflows

arXiv:2605.18327v1 Announce Type: new Abstract: AI agents deployed into SRE workflows currently derive their understanding of environment state from raw observability telemetry at query time, paying a

CrossView Suite: Harnessing Cross-view Spatial Intelligence of MLLMs with Dataset, Model and Benchmark

arXiv:2605.18621v1 Announce Type: cross Abstract: Spatial intelligence requires multimodal large language models (MLLMs) to move beyond single-view perception and reason consistently about objects, vi

DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs

arXiv:2605.18498v1 Announce Type: cross Abstract: Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balanci

iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learning

arXiv:2605.17179v1 Announce Type: new Abstract: Emotion understanding is a fundamental challenge in affective computing and artificial intelligence. While existing approaches predominantly focus on fa

LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injectio

arXiv:2605.17986v1 Announce Type: cross Abstract: AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools. This creates indirect prompt-injection (IPI) ri

PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments

arXiv:2603.23231v2 Announce Type: replace Abstract: Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of t

SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors

arXiv:2605.16626v1 Announce Type: cross Abstract: Since autonomous coding agents generate complex behaviors at high-volume, we may want to use other LLMs to monitor actions to reduce the risk from dan

Time Series Foundation Models as Strong Baselines in Transportation Forecasting: A Large-Scale Benchmark Analysis

arXiv:2602.24238v2 Announce Type: replace Abstract: Accurate forecasting of transportation dynamics is essential for urban mobility and infrastructure planning. Although recent work has achieved stron

TriALS: Triphasic-Aided Liver Lesion Segmentation Benchmark in Non-Contrast CT

arXiv:2605.16572v1 Announce Type: new Abstract: Automated segmentation of liver lesions on non-contrast computed tomography (NCCT) is clinically important but fundamentally challenging, particularly i

Validate Your Authority: Benchmarking LLMs on Multi-Label Precedent Treatment Classification

arXiv:2605.17691v1 Announce Type: cross Abstract: Automating the classification of negative treatment in legal precedent is a critical yet nuanced NLP task where misclassification carries significant

AGOP-IxG: A Gradient Covariance Filter for Local Feature Attribution on Tabular Data, with a Controlled Benchmark

arXiv:2605.15700v1 Announce Type: new Abstract: Automated machine learning pipelines increasingly produce models whose predictions must be explained to end users, auditors, and downstream decision sys

IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

arXiv:2603.17915v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages rem

Research

Mind the Motions: Benchmarking Theory-of-Mind in Everyday Body Language

arXiv:2511.15887v2 Announce Type: replace Abstract: Our ability to interpret others' mental states through nonverbal cues (NVCs) is fundamental to our survival and social cohesion. While existing Theo

researcharxiv-cs-cl

MyoChallenge 2025: A New Benchmark for Human Athletic Intelligence

model-releasesarxiv-cs-ro

arXiv:2605.15650v1 Announce Type: new Abstract: Athletic performance represents the pinnacle of human motor intelligence, demanding rapid choices, precise control, agility, and coordinated physical ex

VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing

arXiv:2605.15677v1 Announce Type: new Abstract: Despite the rapid advancements in Vision-Language Models (VLMs), a critical gap remains in their ability to handle structured, controllable diagrammatic

Enhancing Few-Shot Classification of Benchmark and Disaster Imagery with ABHFA-Net

arXiv:2510.18326v3 Announce Type: replace Abstract: The rising incidence of natural and human-induced disasters necessitates robust visual recognition systems capable of operating under limited labele

NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces

arXiv:2605.14698v1 Announce Type: cross Abstract: Foundation models (FMs) promise to extract unified representations that generalize across downstream tasks. They have emerged across fields, including

OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling

arXiv:2601.19924v2 Announce Type: replace-cross Abstract: We investigate the capabilities and scalability of Large Language Models (LLMs) in optimization modeling, a domain requiring structured reason

PROCESS-2: A Benchmark Speech Corpus for Early Cognitive Impairment Detection

arXiv:2605.14888v1 Announce Type: cross Abstract: Speech-based analysis offers a scalable and non-invasive approach for detecting cognitive decline, yet progress has been constrained by the limited av

Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning

arXiv:2605.13936v1 Announce Type: cross Abstract: The recent success of large language models (LLMs) has been largely driven by vast public datasets. However, the next frontier for LLM development lie

ViMU: Benchmarking Video Metaphorical Understanding

arXiv:2605.14607v1 Announce Type: new Abstract: Any new medium, once it emerges, is used for more than the transmission of overt content alone. The information it carries typically operates on two lev

Local Ai

AIvaluateXR: An Evaluation Framework for on-Device AI in XR with Benchmarking Results

arXiv:2502.15761v3 Announce Type: replace-cross Abstract: The deployment of large language models (LLMs) on extended reality (XR) devices has great potential to advance the field of human-AI interacti

local-aiarxiv-cs-ai

Benchmarking Attribute Discrimination in Infant-Scale Vision-Language Models

arXiv:2512.18951v3 Announce Type: replace Abstract: Infants learn not only object categories but also fine-grained visual attributes such as color, size, and texture from limited experience. Prior inf

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence

arXiv:2605.12882v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have significantly advanced document understanding, yet current Doc-VQA evaluations score only the final answ

SpikeProphecy: A Large-Scale Benchmark for Autoregressive Neural Population Forecasting

arXiv:2605.12992v1 Announce Type: cross Abstract: Neural population models, which predict the joint firing of many simultaneously recorded neurons forward in time, are typically evaluated by a single

Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?

arXiv:2605.12684v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are now routinely deployed for visual understanding, generation, and curation. A substantial fraction of thes

Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters

arXiv:2605.11960v1 Announce Type: new Abstract: Vision Large Language Models (VLLMs) have achieved remarkable success in modern text-rich visual understanding. However, their perceptual robustness in

13 May 2026

Hardware

MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces

arXiv:2605.11333v1 Announce Type: cross Abstract: The fast pace of artificial intelligence~(AI) innovation demands an agile methodology for observation, reproduction and optimization of distributed ma

hardwarearxiv-cs-lg

13 May 2026

SafeManip: A Property-Driven Benchmark for Temporal Safety Evaluation in Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2605.12386v1 Announce Type: new Abstract: Robotic manipulation is typically evaluated by task success, but successful completion does not guarantee safe execution. Many safety failures are tempo

13 May 2026

A Stability Benchmark of Generative Regularizers for Inverse Problems

arXiv:2605.10076v1 Announce Type: cross Abstract: Generative (diffusion) priors demonstrate remarkable performance in addressing inverse problems in imaging. Yet, for scientific and medical imaging, i

AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks

arXiv:2605.10286v1 Announce Type: new Abstract: Building effective clinical decision support systems requires the synthesis of complex heterogeneous multimodal data. Such modalities include temporal e

BenchHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition

arXiv:2605.08296v1 Announce Type: new Abstract: Human Activity Recognition (HAR) from wearable sensors supports broad healthcare and behavior science applications. However, data heterogeneity and the

Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials

arXiv:2605.08988v1 Announce Type: cross Abstract: Machine Learning Interatomic Potentials play a fundamental role in computational chemistry and materials science, enabling applications from molecular

Benchmarking Sensor-Fault Robustness in Forecasting

arXiv:2605.10822v1 Announce Type: new Abstract: Cyber-physical system (CPS) forecasting models depend on sensor streams with noisy, biased, missing, or temporally misaligned readings, yet standard for

Beyond Isolation: A Unified Benchmark for General-Purpose Navigation

model-releasesarxiv-cs-ro

arXiv:2605.09441v1 Announce Type: new Abstract: The pursuit of general-purpose embodied agents is hindered by fragmented evaluation protocols that isolate navigation skills and fixate on specific robo