Search: “benchmark”

BlenderWorld Benchmark from Alibaba

model-releasesr-localllama

Alibaba's Qwen team released a new repository on Wednesday, October 14, 2026. Its goal is to measure how well visual agents can create 3D scenes in Blender based on real-world references. https://gith

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

ChaosBench-Logic: A Benchmark for Logical and Symbolic Reasoning on Chaotic Dynamical Systems

DGX agent

arXiv:2601.01982v2 Announce Type: replace Abstract: Large language models (LLMs) excel at natural language tasks but remain brittle in domains requiring precise logical and symbolic reasoning. Chaotic

model-releasesarxiv-cs-ai

16 Oct 2026

Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation

arXiv:2610.21180v1 Announce Type: new Abstract: General-purpose dexterous manipulation requires policies to transfer knowledge from related tasks to new tasks with only a few target demonstrations. Ex

From Adults to Children: Benchmarking Foundation Models for Chest X-Ray Classification under Domain Shift

arXiv:2610.21809v1 Announce Type: new Abstract: Foundation models have shown significant potential across diverse fields and have been applied to medical imaging, but their robustness across patient p

RDL-Bench: A Benchmark for Task-Aware Question Answering over Relational Databases

arXiv:2610.21382v1 Announce Type: new Abstract: Relational databases (RDBs) remain the primary infrastructure for storing and managing structured data across enterprise and scientific domains. Natural

Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout

arXiv:2610.21397v1 Announce Type: new Abstract: An AI data center is committed years before it draws a watt: a few discrete choices (grid or on-site gas, evaporative or dry cooling, which county to fi

Benchmarking 3D Reconstruction for Under-Ice Robotic Perception in Arctic Environments

arXiv:2610.19097v1 Announce Type: new Abstract: As the Arctic transitions toward a seasonally ice-free state, capturing the complex morphology of submerged sea ice is crucial for advancing models of o

Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents

arXiv:2610.18052v1 Announce Type: new Abstract: Long-term memory has become a core component of large language model (LLM) agents. As agents accumulate personal information over long-term use, users i

Cross-Lingual TTS for Indian Languages: A Benchmark and Inference-Time Duration Estimation Strategies

arXiv:2610.14826v1 Announce Type: cross Abstract: Recent alignment-free Text-to-Speech (TTS) models achieve high synthesis quality by removing explicit alignment modules and simplifying training. Howe

HM3D-M3: A Benchmark for Multi-Agent, Multi-Modal, Multi-Object Goal Navigation

arXiv:2610.20121v1 Announce Type: new Abstract: Rapid progress in AI agents and robotics makes it a matter of the near future that humans are served by a team of robots. Such a team will receive sever

IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks

arXiv:2610.19481v1 Announce Type: new Abstract: Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimatio

LED-BENCH: A Benchmark for Large-format Engineering Drawing Understanding

arXiv:2610.19571v1 Announce Type: new Abstract: Real-world engineering drawings are often large-format and information-dense. Answering practical engineering questions therefore requires both fine-gra

MagicGarment: Benchmarking Single-Image Standalone 3D Garment Reconstruction

arXiv:2610.19623v1 Announce Type: new Abstract: Single-image 3D reconstruction has advanced rapidly, yet standalone garments remain challenging due to thin-shell geometry, open boundaries, and layered

OLiVES: An Outdoor Low-Light Video Benchmark for Enhancement and Segmentation

arXiv:2610.20501v1 Announce Type: new Abstract: Low-light video processing remains an underexplored problem in computer vision, primarily due to the scarcity of high-quality ground-truth data. Existin

SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs

arXiv:2508.08742v3 Announce Type: replace-cross Abstract: Scientific literature question answering is a pivotal step towards new scientific discoveries. Recently, extit{two-stage} retrieval-augmented

Benchmarking Interaction-Oriented Aerial Mobility across Heterogeneous Environments

arXiv:2610.18381v1 Announce Type: new Abstract: Aerial mobile manipulation requires unmanned aerial vehicles (UAVs) to traverse heterogeneous environments such as indoor and outdoor domains, and conne

BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views

arXiv:2610.17457v1 Announce Type: cross Abstract: Browser agents need to make decisions and execute actions on behalf of users while consuming untrusted webpages, creating opportunities for malicious

DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments

arXiv:2607.17291v2 Announce Type: replace Abstract: Deep research agents increasingly operate over the open web, where relevant records coexist with redundant summaries, outdated reports, and misleadi

I2O2I-Nav: A Benchmark with a Modular Agent for Long-Horizon Indoor-to-Outdoor-to-Indoor Navigation

arXiv:2610.18307v1 Announce Type: new Abstract: Service and delivery robots often need to complete long-horizon navigation missions spanning indoor and outdoor environments. Existing embodied navigati

OmniRaindrop: A Comprehensive Benchmark for Dual-Focused Raindrop Removal under Day and Night Scenes

arXiv:2510.17719v3 Announce Type: replace Abstract: Adherent raindrops on camera lenses are a pervasive source of real-world image degradation, undermining outdoor visual perception for downstream app

Sanzaru: Benchmarking Protocol Formation under Epistemic & Communication Asymmetry

arXiv:2610.15163v1 Announce Type: cross Abstract: A language model's ability to solve a task individually does not ensure that a team can solve it under unequal access to information and communication

SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks

arXiv:2606.16953v2 Announce Type: replace Abstract: Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While

From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?

arXiv:2610.06411v2 Announce Type: replace Abstract: Agentic systems increasingly coordinate molecular-design tools, but it is unclear which layer of the stack limits outcomes on real projects. We deve

12 Oct 2026

PPG-Bench: A Unified Benchmark for PPG Foundation Models

arXiv:2610.13435v1 Announce Type: new Abstract: Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (F

12 Oct 2026

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models

arXiv:2610.12696v1 Announce Type: new Abstract: Vision-language-action models (VLAs) offer a promising route toward general-purpose robotics by transferring semantic knowledge from large-scale vision-

12 Oct 2026

Deep Learning vs. Statistical Models for Multi-Horizon Price Forecasting of Second-Hand Electronics: A Systematic Benchmark

arXiv:2610.10727v1 Announce Type: cross Abstract: Forecasting resale prices of used electronics is critical for subscription-based platforms where pricing errors translate directly into risk. Unlike s

False Claims, Credible Images: A Red-Teaming Benchmark for Commercial Image Generators

arXiv:2610.11112v1 Announce Type: cross Abstract: Image-generation models can now produce text-rich, natural-looking visual artifacts that are hard to distinguish from real-world evidence, such as new

GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary

arXiv:2610.11129v1 Announce Type: new Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated G

Research

How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices

arXiv:2610.11877v1 Announce Type: new Abstract: Automated research systems support literature synthesis, ideation, experiments, writing, and peer review, but their evaluation is dispersed across tasks

researcharxiv-cs-ai

Agents

MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare

arXiv:2605.11814v2 Announce Type: replace Abstract: The large-scale deployment of personalized healthcare agents demands memory mechanisms that are exceptionally precise, safe, and capable of long-ter

agentsarxiv-cs-ai

Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild

arXiv:2610.11328v1 Announce Type: new Abstract: Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intell

PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology

arXiv:2610.11329v1 Announce Type: new Abstract: Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characte

RH-Detect: A Unified Benchmark for Reward Hacking Detection

arXiv:2610.10947v1 Announce Type: new Abstract: Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model s

SAVU-BENCH: A Real-World Benchmark for Spatial Audio-Visual Understanding

arXiv:2610.10624v1 Announce Type: cross Abstract: Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modal

Research

Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks

arXiv:2610.11978v1 Announce Type: new Abstract: Jev is a 'System One' model that returns a choice among given options instead of generating text. We study how such a specialized decision model compare

researcharxiv-cs-cl

Research

UXBench Pro: Benchmarking Personalized User Experience in Multi-Turn Dialogue Interactions

arXiv:2610.11638v1 Announce Type: new Abstract: Evaluating user experience (UX) with automated computational methods has gained increasing attention, supported by empirical evidence from UXBench. Howe

researcharxiv-cs-cl

When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection

arXiv:2610.10971v1 Announce Type: cross Abstract: Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supp

A Multi-Source Ultrasound Benchmark Revealing the Limits of Contemporary Self-Supervised Anomaly Detection Methods

arXiv:2610.09677v1 Announce Type: new Abstract: Self-supervised anomaly detection is a promising paradigm for medical ultrasound, as normal images are often easier to obtain than exhaustive annotation

Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science

model-releasesarxiv-cs-cl

arXiv:2610.09446v1 Announce Type: new Abstract: Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not de

DISRQAD: Diffusion Image Super-Resolution Quality Assessment Dataset and Benchmark

arXiv:2610.09077v1 Announce Type: new Abstract: Diffusion-based image super-resolution (SR) can create visually plausible detail that is not supported by the low-resolution input. We introduce DISRQAD

Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

arXiv:2606.13994v2 Announce Type: replace-cross Abstract: LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world.

Scaling Legal AI: Benchmarking Mamba and Transformers for Statutory Classification and Case Law Retrieval

arXiv:2509.00141v2 Announce Type: replace-cross Abstract: Statutory corpora and judicial decisions are growing faster than legal professionals can read them, while individual judgments often exceed th

ScribbleEdit: A Benchmark for Scribble-Only Image Editing

arXiv:2610.09382v1 Announce Type: cross Abstract: Scribble-based interaction provides a lightweight and intuitive way for users to specify image editing intents in interactive editing tools. However,

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

arXiv:2610.07967v1 Announce Type: new Abstract: As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their re

Decision model benchmark: Jev, Kev, Liquid d1, and more

On 15 September, TypeSafe released Jev, and within a few weeks about 10 companies had shipped something that works the same way. OpenAI, Liquid, Cloudflare, AWS and PostHog all launched one, and an op

model-releasesarize-ai

HARL-A: An Extensible Benchmark Framework for Heterogeneous Multi-Agent Adversarial Reinforcement Learning in IsaacLab

arXiv:2510.01264v2 Announce Type: replace Abstract: Progress in adversarial multi-agent reinforcement learning (MARL) for robotics has been hampered by a lack of shared, extensible infrastructure that

The Failure Is in the Readout: Fine-Grained Emotion Recognition Benchmarks Measure Elicitation, Not Perception

model-releasesarxiv-cs-cl

arXiv:2610.08162v1 Announce Type: cross Abstract: Fine-grained emotion recognition supports therapy tools and social robots, but it needs facial data, which raises privacy and data-protection concerns

Towards Explainable Benchmarking for Data-driven Post-Wildfire Debris Flow Prediction

arXiv:2610.07358v1 Announce Type: new Abstract: Post-wildfire debris flows (PFDFs) are destructive sediment-laden hazards triggered when intense rainfall strikes recently burned terrain, destabilizing