Search: “benchmark”

Benchmarking Automatic Speech Recognition Tools for Iberian Languages

arXiv:2609.36920v1 Announce Type: new Abstract: Comprehensive evaluations of automatic speech recognition (ASR) for Iberian languages remain limited, and low-resource languages, biases, and efficiency

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Tutorials

Benchmarking graph-based models for in-silico toxicity prediction in drug discovery

DGX agent

arXiv:2609.37555v1 Announce Type: new Abstract: Drug discovery is a costly and high-risk process, where toxicity-related failures remain a major cause of attrition in both preclinical and clinical sta

tutorialsarxiv-cs-lg

30 Sept 2026

Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark

arXiv:2609.37938v1 Announce Type: cross Abstract: Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggre

Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study

arXiv:2609.37848v1 Announce Type: new Abstract: Biomedical machine learning papers often compress model performance into one headline number. That number can look like a property of the model even whe

Event-Only Wingbeat Counting under Camera Motion: A Controlled MuJoCo Benchmark

arXiv:2609.37465v1 Announce Type: new Abstract: Counting completed wingbeats requires identifying individual cycles, including during frequency changes and pauses; estimating a dominant frequency alon

From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identification

arXiv:2503.12232v3 Announce Type: replace Abstract: Aiming to match pedestrian images captured under varying lighting conditions, visible-infrared person re-identification (VI-ReID) has drawn intensiv

OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells

arXiv:2609.37773v1 Announce Type: new Abstract: Artificial Intelligence Virtual Cells (AIVCs) are envisioned as scientific agents that simulate cellular responses, explain underlying mechanisms, and s

PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets

arXiv:2609.37321v1 Announce Type: cross Abstract: Power markets are a natural testbed for multi-agent reinforcement learning (MARL), where multiple self-interested participants repeatedly submit bids.

PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learning

arXiv:2609.36052v1 Announce Type: new Abstract: Power system operation is a safety-critical sequential decision-making problem, making it a natural testbed for reinforcement learning (RL). However, ex

RoboChrono: A Real Robot Benchmark for Streaming Task Understanding

model-releasesarxiv-cs-ro

arXiv:2609.36605v1 Announce Type: new Abstract: Understanding ongoing robot manipulation requires models to interpret visual observations in relation to interaction history and task progress. We intro

VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics

arXiv:2609.37287v1 Announce Type: cross Abstract: Image translation is a fundamental capability of multimodal models for multilingual applications, requiring visual understanding and meaning preservat

VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training

arXiv:2609.36648v1 Announce Type: new Abstract: Vision-language pre-training has reshaped image clustering, giving rise to language-assisted image clustering (LaIC), which leverages textual semantics

A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services

arXiv:2609.31877v1 Announce Type: cross Abstract: Cloud-based Large language model (LLM) services create a network-level traffic side channel that can expose model, prompt, and task behavior despite e

A Multi-Dataset Benchmark of YOLO-Based Weed Detection in Precision Agriculture

arXiv:2609.33991v1 Announce Type: new Abstract: Weed detection is an important component of precision agriculture, enabling site-specific weed management and reducing unnecessary herbicide use. Althou

AevaScenes: An FMCW LiDAR Dataset and Benchmark for Long-Range Perception

arXiv:2609.33230v1 Announce Type: new Abstract: FMCW LiDAR measures per-point radial Doppler velocity alongside range, providing a motion cue unavailable in conventional time-of-flight sensors. Exploi

Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study

arXiv:2605.06643v2 Announce Type: replace Abstract: Despite the growing popularity of Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported perfor

Benchmarking EEG Foundation Models at Scale: Lessons from 20,000 Evaluations

arXiv:2609.32743v1 Announce Type: new Abstract: Electroencephalography (EEG) foundation models (FMs) promise transferable neural representations, yet their advantages over strong supervised baselines

CLIMB: A Clinical Multimorbidity Benchmark for Diagnosing Co-occurring Conditions through Multiturn Conversations

arXiv:2609.35462v1 Announce Type: new Abstract: Patients often have several co-occurring clinical conditions, and the findings needed to identify and disambiguate them emerge over the course of a cons

ECG-Scroll: A Long-Horizon, Streaming Benchmark and Agent Environment for Interpretation of Ambulatory Electrocardiograms

arXiv:2609.33117v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can now interpret a standard ten-second, twelve-lead electrocardiogram (ECG) with clinically grounded, reward-v

ForVis: An In-Field Dataset and Benchmark for VIO Using Under-Canopy UAV Flights in Forests

model-releasesarxiv-cs-ro

arXiv:2609.35482v1 Announce Type: new Abstract: Visual-inertial Simultaneous Localization and Mapping (VI-SLAM) for UAVs remains difficult to evaluate in real forest environments, where motion, illumi

Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos

arXiv:2609.34630v1 Announce Type: new Abstract: Real-world AI systems must reason about objects that are no longer visible: an AR assistant guiding a user back to an object used earlier, a household r

MixBench-TS: A Multivariate Time Series Forecasting Benchmark Where Channel Mixing Pays Off

arXiv:2609.32656v1 Announce Type: cross Abstract: Multivariate Time Series Forecasting (MTSF) models that mix information across channels assume that the past of one channel carries information about

OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents

arXiv:2609.32569v1 Announce Type: new Abstract: Smart-home assistants are expected to handle diverse, realistic requests that arise in daily life. In such interactions, users often rely on the surroun

PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery

arXiv:2609.32856v1 Announce Type: new Abstract: Vector polygon generation converts visual inputs, e.g., remote sensing (RS) images, into vectorized polygonal geometries, supporting applications such a

SleuthBench: Benchmarking Statistical LLM Evaluation Using Tabular Hidden Signals

arXiv:2609.34228v1 Announce Type: new Abstract: Evaluating statistical discovery by large language model (LLM) agents requires verifiable analytical ground truth. Establishing such ground truth for re

SolveEdit: Benchmarking Visual Problem Solving in Generative Models

arXiv:2609.35504v1 Announce Type: new Abstract: Machine intelligence is often evaluated through abstract reasoning problems, yet many real-world problems are visual, such as arranging objects, repairi

TRACE: Expert-Aligned ECG Representation Learning with Rigorous Benchmarking and Real-World Validation in Acute Cardiac Care

arXiv:2609.34088v1 Announce Type: new Abstract: TRACE (Text-Reinforced Analysis of Cardio ECGs) is a multimodal electrocardiogram (ECG) representation model that learns clinically grounded signal embe

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

arXiv:2609.35028v1 Announce Type: new Abstract: Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosyste

ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Models

arXiv:2609.33314v1 Announce Type: new Abstract: Zero-shot graph models (ZGMs), which learn transferable knowledge from source graphs and directly apply to unseen target graphs without any adaptation,

Research

A Benchmarking Framework for Context-aware XR Interfaces

arXiv:2609.30466v1 Announce Type: cross Abstract: Everyday Extended Reality (XR) systems aim to provide context-aware access to the right functionalities at the right time and place, with minimal manu

researcharxiv-cs-ai

Research

Auditing System-1 Models on Biosecurity-Relevant Benchmarks: Calibration, Selective Prediction, and Permutation Instability in a Non-Generative Model

arXiv:2609.30454v1 Announce Type: new Abstract: Non-generative 'System-1' models return structured probabilistic decisions in a single forward pass, without autoregressive decoding, at a small fractio

researcharxiv-cs-lg

Benchmarking the Connectomes of Caenorhabditis elegans within the Reservoir Computing Framework

arXiv:2609.30508v1 Announce Type: new Abstract: The aim of this work is to examine the connectomes of Caenorhabditis elegans through a computational lens using the reservoir computing framework. Conne

JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models

arXiv:2609.31142v1 Announce Type: cross Abstract: Models trained with reinforcement learning for calibrated decisions (RLCD), such as Jev, answer a typed question about an input, the state, with a pro

RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objects

arXiv:2505.05848v3 Announce Type: replace Abstract: Modern 3D reconstruction and novel view synthesis approaches have demonstrated strong performance on scenes with opaque, non-refractive objects. How

UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound

arXiv:2609.30928v1 Announce Type: cross Abstract: Ultrasound is one of the most widely used medical imaging modalities, and recent large vision-language models(VLMs) have shown increasing capabilities

Benchmarking and Domain Adaptation of Automatic Speech Recognition (ASR) for Adolescent Health Communication in Ghanaian Languages

arXiv:2609.29798v1 Announce Type: cross Abstract: This paper presents an end-to-end study of automatic speech recognition (ASR) for adolescent health communication in three Ghanaian languages (Twi, Da

COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages

arXiv:2609.28826v1 Announce Type: new Abstract: Machine translation (MT) for Indian languages remains constrained by the limited availability of high-quality, Indic-centric parallel corpora and evalua

Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgets

arXiv:2609.29509v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as multi-turn agents that must sustain goals, use tools, and adapt to other agents over extended

EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation

arXiv:2609.29511v1 Announce Type: new Abstract: Machine Translation (MT) for low-resource languages remains far behind that of high-resource languages, and the gap is widest in specialised domains, wh

GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning

arXiv:2609.29541v1 Announce Type: cross Abstract: Referring segmentation in overhead imagery is inherently relational: a query may ask for the buildings north of the road or the pond closest to a resi

IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents

arXiv:2601.06676v3 Announce Type: replace-cross Abstract: Large Language Model (LLM)-based deep research agents perform multi-step reasoning, web exploration, and long-form report generation. In these

Limited Structural Reliability in Public Educational Prediction Benchmarks: A Four-Dimension Audit of Seven Datasets

arXiv:2609.29625v1 Announce Type: new Abstract: Across seven public educational prediction datasets, three passed all four pre-modeling reliability checks; the remaining four either failed group-aware

OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding

arXiv:2603.24876v2 Announce Type: replace Abstract: Remote sensing visual grounding (RSVG) aims to localize specific targets in remote sensing images using natural language expressions. However, exist

PFArena: Benchmarking Language Models for Protein Modification

arXiv:2609.28921v1 Announce Type: new Abstract: Protein modification requires navigating an immense sequence space, yet wet-lab validation remains low-throughput and costly. Although computational par

STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Models

arXiv:2609.29607v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We arg

A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems

arXiv:2609.27201v1 Announce Type: new Abstract: Sequential RecSys are central to modern personalization, exploiting user's historical interaction sequences to drive next-step decisions. Deep learning

24 Sept 2026

Beyond Balanced Accuracy: A Resolution and Parity-Controlled Benchmark for Vision-Language and Vision-Only Defect Assessment in UAV Power-Line Inspection

arXiv:2609.27457v1 Announce Type: new Abstract: Vision-language models (VLMs) are often reported to outperform task-specific vision backbones for unmanned aerial vehicle (UAV) power-line defect assess

24 Sept 2026

Does Graph Structure Earn Its Place in Microservice Root-Cause Analysis? A Controlled Study on RCAEval, and What the Benchmark Was Really Measuring

arXiv:2609.27069v1 Announce Type: cross Abstract: Graph neural networks dominate recent work on microservice root-cause analysis, yet recent results question whether the graph contributes. Those resul