Search: “benchmark”

Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method

arXiv:2604.11209v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable success across a wide range of applications especially when augmented by external knowledge thro

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agents

DGX agent

arXiv:2603.24329v2 Announce Type: replace-cross Abstract: Multimodal LLMs are increasingly deployed as perceptual backbones for autonomous agents in 3D environments, from robotics to virtual worlds. T

agentsarxiv-cs-ai

14 Apr 2026

GazeVaLM: A Multi-Observer Eye-Tracking Benchmark for Evaluating Clinical Realism in AI-Generated X-Rays

arXiv:2604.11653v1 Announce Type: new Abstract: We introduce GazeVaLM, a public eye-tracking dataset for studying clinical perception during chest radiograph authenticity assessment. The dataset compr

LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment

arXiv:2604.11689v1 Announce Type: new Abstract: While the shortage of explicit action data limits Vision-Language-Action (VLA) models, human action videos offer a scalable yet unlabeled data source. A

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

arXiv:2604.10024v1 Announce Type: cross Abstract: Long video summarization presents significant challenges for current multimodal large language models (MLLMs), particularly in maintaining temporal fi

M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistency

arXiv:2604.01306v2 Announce Type: replace Abstract: Evaluating scientific arguments requires assessing the strict consistency between a claim and its underlying multimodal evidence. However, existing

MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models

arXiv:2604.10971v1 Announce Type: cross Abstract: In the progress of industrial anomaly detection, general anomaly detection (GAD) is an emerging trend and also the ultimate goal. Unlike the conventio

NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration

arXiv:2604.09678v1 Announce Type: cross Abstract: As agentic network management gains popularity, there is a critical need for evaluation frameworks that transcend static, one-shot testing. To address

PepBenchmark: A Standardized Benchmark for Peptide Machine Learning

arXiv:2604.10531v1 Announce Type: cross Abstract: Peptide therapeutics are widely regarded as the 'third generation' of drugs, yet progress in peptide Machine Learning (ML) are hindered by the absence

Seeing Through the Tool: A Controlled Benchmark for Occlusion Robustness in Foundation Segmentation Models

arXiv:2604.11711v1 Announce Type: new Abstract: Occlusion, where target structures are partially hidden by surgical instruments or overlapping tissues, remains a critical yet underexplored challenge f

SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors

arXiv:2510.17516v4 Announce Type: replace-cross Abstract: Large language model (LLM) simulations of human behavior have the potential to revolutionize the social and behavioral sciences, if and only i

Spatial Competence Benchmark

arXiv:2604.09594v1 Announce Type: new Abstract: Spatial competence is the quality of maintaining a consistent internal representation of an environment and using it to infer discrete structure and pla

Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization

arXiv:2604.09574v1 Announce Type: new Abstract: The rise of autonomous GUI agents has triggered adversarial countermeasures from digital platforms, yet existing research prioritizes utility and robust

BEDTime: A Unified Benchmark for Automatically Describing Time Series

arXiv:2509.05215v3 Announce Type: replace Abstract: Recent works propose complex multi-modal models that handle both time series and language, ultimately claiming high performance on complex tasks lik

Benchmarking CNN- and Transformer-Based Models for Surgical Instrument Segmentation in Robotic-Assisted Surgery

arXiv:2604.09151v1 Announce Type: new Abstract: Accurate segmentation of surgical instruments in robotic-assisted surgery is critical for enabling context-aware computer-assisted interventions, such a

Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understanding

arXiv:2511.23071v2 Announce Type: replace-cross Abstract: Reading scene text, that is, text appearing in images, has numerous application areas, including assistive technology, search, and e-commerce.

PilotBench: A Benchmark for General Aviation Agents with Safety Constraints

arXiv:2604.08987v1 Announce Type: new Abstract: As Large Language Models (LLMs) advance toward embodied AI agents operating in physical environments, a fundamental question emerges: can models trained

QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation

arXiv:2604.08570v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for code generation, yet quantum code generation is still evaluated mostly within single frameworks

Benchmark Your Local LLMs in 3 Commands

This r/ollama post describes a streamlined method for performance-testing locally-running large language models using the Ollama framework, achievable with just three terminal commands. It likely intr

model-releasesr-ollama

12 Apr 2026

A Benchmark of Classical and Deep Learning Models for Agricultural Commodity Price Forecasting on A Novel Bangladeshi Market Price Dataset

arXiv:2604.06227v1 Announce Type: new Abstract: Accurate short-term forecasting of agricultural commodity prices is critical for food security planning and smallholder income stabilisation in developi

10 Apr 2026

ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detection

arXiv:2510.07927v2 Announce Type: replace Abstract: Anomaly detection plays a pivotal role in manufacturing quality control, yet its application is constrained by limited abnormal samples and high man

10 Apr 2026

LLM Spirals of Delusion: A Benchmarking Audit Study of AI Chatbot Interfaces

arXiv:2604.06188v1 Announce Type: cross Abstract: People increasingly hold sustained, open-ended conversations with large language models (LLMs). Public reports and early studies suggest that, in such

10 Apr 2026

When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark

arXiv:2610.21555v1 Announce Type: cross Abstract: Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a

16 Oct 2026

Tutorials

An Explainable System for Offensive Content Detection in Nastaliq Urdu: Corpus Curation, Transformer Benchmarking, and Word-Level Attribution

arXiv:2610.15795v1 Announce Type: cross Abstract: Automated moderation systems are deployed at scale, yet they remain opaque to the users and moderators affected by their decisions, and their coverage

tutorialsarxiv-cs-ai

15 Oct 2026

Agents

ASIWorld: Benchmarking Autonomous Agents in a Persistent Survival Sandbox

arXiv:2610.14807v1 Announce Type: new Abstract: Evaluating autonomous agents requires examining whether they can translate goals into coherent action across extended interactions. In persistent enviro

agentsarxiv-cs-ai

15 Oct 2026

BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs

arXiv:2603.16557v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However,

15 Oct 2026

Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks

model-releasesarxiv-cs-ro

arXiv:2610.16238v1 Announce Type: new Abstract: LLM agents increasingly control robots using VLA policies, perception models, and procedural knowledge. These modules are difficult to reuse because the

Safety

Representation Geometry and Inductive Biases in Quantum Classifiers across Classical Visual Benchmarks

arXiv:2610.17722v1 Announce Type: cross Abstract: Quantum Machine Learning (QML) has a theoretical advantage over classical algorithms for specific synthetic data that is computationally hard. However

safetyarxiv-cs-lg

Skin Tone in the Wild: A Dataset and Benchmark for 10-tone MST Classification

arXiv:2603.02475v2 Announce Type: replace Abstract: Skin tone classification is a fundamental task in computer vision, with applications in quantifying medical features, assessing demographic represen

What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study

arXiv:2610.18869v1 Announce Type: cross Abstract: How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We comp

Research

Benchmarking Embedding Geometries and Clustering Algorithms for Unsupervised Patent Claim Classification

arXiv:2610.12997v1 Announce Type: new Abstract: Prior-art retrieval systems are structurally organized around classification codes, making cross-code discovery difficult. This paper asks if an unsuper

researcharxiv-cs-ai

12 Oct 2026

JitterVLA: A Replay Benchmark for Motion Quality in VLA-Driven Human-Robot Manipulation

arXiv:2610.13628v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies issue sparse targets whose arrival at the robot controller is delayed and irregular, so a reasonable predicted a

12 Oct 2026

FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech

arXiv:2609.32396v2 Announce Type: replace Abstract: Forced alignment estimates the timestamps of each word, phone or character in speech given its transcript. Published comparisons normalize transcrip

How Hackable Is Your Speech Quality Metric? A Corrected Protocol, a Benchmark, and What Patching Buys

arXiv:2610.10899v1 Announce Type: new Abstract: Speech quality predictors are increasingly used as rewards, yet no agreed measure of their hackability exists. The usual measurement has two flaws. Firs

Large-Scale Benchmarking of Quantum Neural Network Configurations for Financial Time Series Forecasting

arXiv:2610.12148v1 Announce Type: new Abstract: Quantum machine learning, and quantum neural networks (QNNs) in particular, are advancing fields with growing potential. Although systematic comparisons

Safety

The Review Lottery: Benchmarking an Observational Estimator of Peer-Review Noise (ICLR 2017-2025)

arXiv:2610.06591v2 Announce Type: replace Abstract: How much of a conference accept/reject decision would change if the same paper were reviewed by a different set of reviewers? Running a second indep

safetyarxiv-cs-ai

Research

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

arXiv:2610.10409v1 Announce Type: cross Abstract: General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carr

researcharxiv-cs-lg

8 Oct 2026

Agents

Wiki-Talkie: Multilingual Benchmarking of Persona-Based Agents on Real-World Discussions

arXiv:2610.08513v1 Announce Type: cross Abstract: LLMs are increasingly deployed as autonomous agents in social environments, making it critical to study their ability to faithfully simulate human int

agentsarxiv-cs-ai

7 Oct 2026

Safety

Benchmarking CLIP for Zero-Shot Face and Periocular Gender Estimation

arXiv:2610.06102v1 Announce Type: new Abstract: We investigate CLIP for zero-shot gender estimation from full-face and periocular images. Three CLIP backbones are evaluated on 11,299 frontal images fr

safetyarxiv-cs-cv

6 Oct 2026

Applications

Here are some pictures of a robot costume wearing high-specularity edge-case mirror suit, a dataset (425 RAW/JPEGs) for benchmarking CV & depth-estimation algorithms against extreme mirror reflections [D]

applicationsr-machinelearning

Purpose-built to stress-test computer vision models, depth cameras, and spatial AI against severe specular glare and geometric reflections. This 425-asset production archive features a custom faceted

4 Oct 2026

Nonobench: an open benchmark of 49 LLMs on nonogram puzzles, public and open source [P]

model-releasesr-machinelearning

Nonobench measures how well LLMs solve nonograms (picross). Each model gets the row and column clues once and returns the full grid. No tools, one attempt per puzzle. Method: - Standard mode: 30 puzzl

4 Oct 2026

VERITYGATE: A Four-Gate Schema-Level Faithfulness Framework and Paired Benchmark for Grounded LLM Narrations over Structured Evidence

arXiv:2610.00833v1 Announce Type: cross Abstract: Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, en

2 Oct 2026

Applications

From Benchmarks to Production: Transferring Time Series Anomaly Detection Methods for Electricity Production Monitoring

arXiv:2609.39257v1 Announce Type: cross Abstract: Accurate forecasting of electricity production is essential for maintaining the operational efficiency and strategic planning of energy utilities. In

applicationsarxiv-cs-ai

1 Oct 2026

Research

Hyperspectral Image Dataset for Benchmarking on Salient Object Detection

arXiv:1806.11314v3 Announce Type: replace Abstract: Many works have been done on salient object detection using supervised or unsupervised approaches on colour images. Recently, a few studies demonstr

researcharxiv-cs-cv

1 Oct 2026

NAQD Env: A benchmark for selective withdrawal in language agents

arXiv:2609.38460v1 Announce Type: new Abstract: Language agents must revise planned actions when evidence changes, permission is revoked, or a stop instruction arrives. A useful response is selective:

1 Oct 2026

Agents

AnthroDial: Benchmarking LLM Anthropomorphism in Autonomous Social Interaction

arXiv:2609.37853v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as social agents, yet credible human-like interaction requires more than fluent responses or pers

agentsarxiv-cs-cl

30 Sept 2026

Evaluating and Benchmarking the System One Model Jev

arXiv:2609.37647v1 Announce Type: cross Abstract: Jev is a commercial System One model from TypeSafe AI that does not generate text: given a state and typed questions, it returns a choice from fixed o

30 Sept 2026

Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method

arXiv:2609.35965v1 Announce Type: cross Abstract: Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require te