Search: “benchmark”

OmniRaindrop: A Comprehensive Benchmark for Dual-Focused Raindrop Removal under Day and Night Scenes

arXiv:2510.17719v3 Announce Type: replace Abstract: Adherent raindrops on camera lenses are a pervasive source of real-world image degradation, undermining outdoor visual perception for downstream app

14 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Sanzaru: Benchmarking Protocol Formation under Epistemic & Communication Asymmetry

DGX agent

arXiv:2610.15163v1 Announce Type: cross Abstract: A language model's ability to solve a task individually does not ensure that a team can solve it under unequal access to information and communication

model-releasesarxiv-cs-lg

14 Oct 2026

SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks

model-releasesarxiv-cs-ro

arXiv:2606.16953v2 Announce Type: replace Abstract: Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While

14 Oct 2026

From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?

arXiv:2610.06411v2 Announce Type: replace Abstract: Agentic systems increasingly coordinate molecular-design tools, but it is unclear which layer of the stack limits outcomes on real projects. We deve

12 Oct 2026

PPG-Bench: A Unified Benchmark for PPG Foundation Models

arXiv:2610.13435v1 Announce Type: new Abstract: Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (F

12 Oct 2026

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2610.12696v1 Announce Type: new Abstract: Vision-language-action models (VLAs) offer a promising route toward general-purpose robotics by transferring semantic knowledge from large-scale vision-

12 Oct 2026

Deep Learning vs. Statistical Models for Multi-Horizon Price Forecasting of Second-Hand Electronics: A Systematic Benchmark

arXiv:2610.10727v1 Announce Type: cross Abstract: Forecasting resale prices of used electronics is critical for subscription-based platforms where pricing errors translate directly into risk. Unlike s

False Claims, Credible Images: A Red-Teaming Benchmark for Commercial Image Generators

arXiv:2610.11112v1 Announce Type: cross Abstract: Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as new

GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary

arXiv:2610.11129v1 Announce Type: new Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated G

How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices

arXiv:2610.11877v1 Announce Type: new Abstract: Automated research systems support literature synthesis, ideation, experiments, writing, and peer review, but their evaluation is dispersed across tasks

researcharxiv-cs-ai

Agents

MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare

arXiv:2605.11814v2 Announce Type: replace Abstract: The large-scale deployment of personalized healthcare agents demands memory mechanisms that are exceptionally precise, safe, and capable of long-ter

agentsarxiv-cs-ai

Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild

arXiv:2610.11328v1 Announce Type: new Abstract: Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intell

PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology

arXiv:2610.11329v1 Announce Type: new Abstract: Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characte

RH-Detect: A Unified Benchmark for Reward Hacking Detection

arXiv:2610.10947v1 Announce Type: new Abstract: Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model s

SAVU-BENCH: A Real-World Benchmark for Spatial Audio-Visual Understanding

arXiv:2610.10624v1 Announce Type: cross Abstract: Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modal

Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks

arXiv:2610.11978v1 Announce Type: new Abstract: Jev is a 'System One' model that returns a choice among given options instead of generating text. We study how such a specialized decision model compare

researcharxiv-cs-cl

UXBench Pro: Benchmarking Personalized User Experience in Multi-Turn Dialogue Interactions

arXiv:2610.11638v1 Announce Type: new Abstract: Evaluating user experience (UX) with automated computational methods has gained increasing attention, supported by empirical evidence from UXBench. Howe

researcharxiv-cs-cl

When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection

arXiv:2610.10971v1 Announce Type: cross Abstract: Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supp

A Multi-Source Ultrasound Benchmark Revealing the Limits of Contemporary Self-Supervised Anomaly Detection Methods

arXiv:2610.09677v1 Announce Type: new Abstract: Self-supervised anomaly detection is a promising paradigm for medical ultrasound, as normal images are often easier to obtain than exhaustive annotation

Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science

model-releasesarxiv-cs-cl

arXiv:2610.09446v1 Announce Type: new Abstract: Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not de

DISRQAD: Diffusion Image Super-Resolution Quality Assessment Dataset and Benchmark

arXiv:2610.09077v1 Announce Type: new Abstract: Diffusion-based image super-resolution (SR) can create visually plausible detail that is not supported by the low-resolution input. We introduce DISRQAD

Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

arXiv:2606.13994v2 Announce Type: replace-cross Abstract: LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world.

Scaling Legal AI: Benchmarking Mamba and Transformers for Statutory Classification and Case Law Retrieval

arXiv:2509.00141v2 Announce Type: replace-cross Abstract: Statutory corpora and judicial decisions are growing faster than legal professionals can read them, while individual judgments often exceed th

ScribbleEdit: A Benchmark for Scribble-Only Image Editing

arXiv:2610.09382v1 Announce Type: cross Abstract: Scribble-based interaction provides a lightweight and intuitive way for users to specify image editing intents in interactive editing tools. However,

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

arXiv:2610.07967v1 Announce Type: new Abstract: As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their re

HARL-A: An Extensible Benchmark Framework for Heterogeneous Multi-Agent Adversarial Reinforcement Learning in IsaacLab

arXiv:2510.01264v2 Announce Type: replace Abstract: Progress in adversarial multi-agent reinforcement learning (MARL) for robotics has been hampered by a lack of shared, extensible infrastructure that

The Failure Is in the Readout: Fine-Grained Emotion Recognition Benchmarks Measure Elicitation, Not Perception

model-releasesarxiv-cs-cl

arXiv:2610.08162v1 Announce Type: cross Abstract: Fine-grained emotion recognition supports therapy tools and social robots, but it needs facial data, which raises privacy and data-protection concerns

Towards Explainable Benchmarking for Data-driven Post-Wildfire Debris Flow Prediction

arXiv:2610.07358v1 Announce Type: new Abstract: Post-wildfire debris flows (PFDFs) are destructive sediment-laden hazards triggered when intense rainfall strikes recently burned terrain, destabilizing

DABACO: A Multi-Camera Dataset and Benchmark for Screen Localization and Pointing Estimation

arXiv:2610.03928v1 Announce Type: new Abstract: Screen localization and pointing estimation are key to low-cost interactive devices. Yet developing and evaluating these algorithms requires realistic d

Does Neural Complexity Improve Health Misinformation Detection? A Leakage-Controlled Cross-Corpus Benchmark

arXiv:2610.04636v1 Announce Type: cross Abstract: Increasing architectural complexity is often assumed to improve health misinformation detection, yet reported gains are difficult to interpret when st

EXHIB: A Benchmark for Realistic and Diverse Evaluation of Function Similarity in the Wild

arXiv:2604.01554v2 Announce Type: replace-cross Abstract: Binary Function Similarity Detection (BFSD) is a core problem in software security, supporting tasks such as vulnerability analysis, malware c

RMMBench: A Comprehensive Benchmark for Robotic Mobile Manipulation

model-releasesarxiv-cs-ro

arXiv:2610.05414v1 Announce Type: new Abstract: Although the advancement of vision-language models (VLMs) has endowed robots with enhanced environmental understanding and task reasoning, a comprehensi

Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

arXiv:2603.02348v2 Announce Type: replace Abstract: An agent meets a model as served: through an endpoint with a price card, a shared cache and other tenants, or on whatever hardware a self-hosted mod

SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown

arXiv:2610.04008v1 Announce Type: cross Abstract: Executable Agent Skills combine natural-language instructions and scripts into reusable packages for LLM agents, and revising them requires fixing err

A Benchmark for Spatially Grounded Gesture Generation

arXiv:2610.03105v1 Announce Type: new Abstract: Communication in shared space interweaves verbal and non-verbal signals, and pointing gestures anchor language to the environment: 'put the cup on that

Benchmarking Candidate Coverage in Typed Decision Models

arXiv:2610.03387v1 Announce Type: new Abstract: Typed decision models return choices or distributions over answer options supplied at request time. Accuracy with complete options does not establish wh

RNADyn: A Benchmark for Generating and Understanding RNA Dynamics

arXiv:2610.03712v1 Announce Type: new Abstract: Ribonucleic acid (RNA) functions through conformational changes that are not fully captured by static structures. However, large-scale standardized RNA

TPBench: A Turning-Point Benchmark for Dialogue Compression

arXiv:2610.02736v1 Announce Type: cross Abstract: A compressor can keep the facts of a dialogue and still drop the turn that changed them. A user corrects a price, reverses a choice, or adds a constra

Benchmarking System One decision models against trained classifiers and language models for automated decision gates

arXiv:2610.00346v1 Announce Type: new Abstract: Software that hands branching decisions to a model needs a declared option and a probability it can threshold. Typed decision models, also called System

researcharxiv-cs-lg

Bounded-Fidelity Sim-as-Demo-Stage: Mocap Handoff for Governance Benchmarks

arXiv:2610.00008v1 Announce Type: cross Abstract: Sim-to-real research pursues physics fidelity as a primary objective: simulators are judged by how closely they reproduce real-world contact dynamics.

DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents

arXiv:2610.00316v1 Announce Type: cross Abstract: Voice agents enable low-latency, natural interaction, yet their ability to faithfully ground responses in external documents remains underexplored. We

HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution

arXiv:2610.02089v1 Announce Type: cross Abstract: As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use req

TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models

arXiv:2610.00835v1 Announce Type: new Abstract: Text-to-music generation models are trained on massive music collections, creating a growing need for data attribution methods that can quantify the con

VoxelSynth3D: Interpretable Volumetric Image-Domain Metal Artifact Reduction with a Paired Synthetic CLINIC-Metal Benchmark

arXiv:2610.01512v1 Announce Type: new Abstract: Metal artifacts in postoperative musculoskeletal CT obscure bone-implant and adjacent soft-tissue interfaces. Many metal artifact reduction (MAR) method

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

arXiv:2609.39810v1 Announce Type: new Abstract: Source-Free Universal Domain Adaptation (SF-UniDA) extends Universal Domain Adaptation by removing access to source data at adaptation time while still

1 Oct 2026

Trustworthy Runtime Error Healing in Real-World Repositories: A Benchmark and Guardrail

arXiv:2609.39086v1 Announce Type: cross Abstract: Runtime error healing lets a crashed program continue by generating code that repairs its live runtime state. Recent work shows that LLMs can generate

1 Oct 2026

A Benchmark & Dataset for Detecting AI-Manipulated Visual Evidence in the Court System

arXiv:2609.37783v1 Announce Type: cross Abstract: Photographic evidence is becoming increasingly vulnerable to forms of alteration and fabrication that existing legal and technical workflows are not w

30 Sept 2026

A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding

arXiv:2603.14733v2 Announce Type: replace Abstract: Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos