Search: “benchmark”

SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning

model-releasesarxiv-cs-ro

arXiv:2608.08070v1 Announce Type: new Abstract: Reliable surgical planning requires models that move beyond recognizing the current surgical step or imitating expert demonstrations, and instead antici

11 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SurveyReview: A Reviewer-Aligned Benchmark for Survey Evaluators

DGX agent

arXiv:2608.07641v1 Announce Type: new Abstract: The rapid advancement of large language models has transformed survey writing from a months-long manual effort into an automated process. As generation

model-releasesarxiv-cs-cl

11 Aug 2026

TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair

arXiv:2608.07617v1 Announce Type: new Abstract: Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatche

11 Aug 2026

WuYuEval: A Multi-Level Benchmark for Large Language Models in Solid Waste Management

arXiv:2608.07529v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as technical assistants, but their competence in solid waste management (SWM) remains difficult to

11 Aug 2026

Explanation Stability of Test-Time Adaptation in Computational Pathology: A Large-Scale Benchmark

arXiv:2608.07062v1 Announce Type: new Abstract: Test-time adaptation (TTA) has become a practical way to adapt deployed models to unlabeled target data, a setting that is especially relevant in comput

10 Aug 2026

FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings

arXiv:2608.07400v1 Announce Type: new Abstract: Financial question answering is typically evaluated by answer correctness, yet in SEC filings a plausible and even numerically correct answer can be gro

10 Aug 2026

UAV3DCrop: Benchmarking 3D Reconstruction in Repeated Multi-Angle UAV Crop Surveys

arXiv:2608.06404v1 Announce Type: new Abstract: Accurate 3D crop monitoring underpins data-driven precision agriculture by enabling field-scale analysis of plant structure, growth dynamics, and manage

10 Aug 2026

An Axiomatic Benchmark for Evaluation of Scientific Novelty Metrics

arXiv:2604.15145v2 Announce Type: replace Abstract: The rigorous evaluation of the novelty of a scientific paper is, even for human scientists, a challenging task. With the increasing interest in AI s

Benchmarking and Enhancing LLMs for Rule-Intensive Review of National Standard Documents

arXiv:2608.06312v1 Announce Type: new Abstract: Large language models (LLMs) increasingly support complex professional tasks, yet their capabilities in rule-intensive document review remain insufficie

GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models

arXiv:2608.05948v1 Announce Type: new Abstract: Physics engines facilitate large-scale training and evaluation for embodied intelligence, while generative video world models are emerging as implicit s

IPV-Bench: Benchmarking Image Protection Methods under Diverse Image-to-Video Generation Scenarios

arXiv:2603.26154v2 Announce Type: replace Abstract: Image-to-video (I2V) generation models can be misused to animate a single image into a convincing fake video, motivating perturbation-based image pr

M^3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

arXiv:2608.05817v1 Announce Type: new Abstract: Metaphor enables the understanding of abstract concepts through cross-domain mappings while conveying affective attitudes. In multimodal scenarios, visu

Matching Matters: A Fair Quality-Efficiency Benchmark for Command-Line Agents

arXiv:2606.21140v2 Announce Type: replace-cross Abstract: Rapid advances in large language models have improved the task-solving capabilities of command-line-interface (CLI)-based agents, whose CLIs d

NavTrust: Benchmarking Trustworthiness for Embodied Navigation

arXiv:2603.19229v2 Announce Type: replace-cross Abstract: There are two major categories of embodied navigation: Vision-Language Navigation (VLN), where agents navigate by following natural language i

SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution

arXiv:2608.05573v1 Announce Type: new Abstract: LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to veri

Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports

arXiv:2608.04682v1 Announce Type: cross Abstract: Coding agents powered by large language models (LLMs) are increasingly adopted in software engineering (SWE) scenarios, capable of fixing a specific b

RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists

arXiv:2608.04783v1 Announce Type: cross Abstract: The integration of Large Language Models (LLMs) into software engineering has shifted the focus from function-level generation to repository-scale ass

RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care

arXiv:2608.04514v1 Announce Type: new Abstract: Background: Respiratory specialty care requires multimodal interpretation, longitudinal risk assessment, guideline-concordant intervention, and whole-co

SciCode-Verified: How Benchmark Defects Underestimated the Scientific-Coding Ability of Language Models

arXiv:2608.04975v1 Announce Type: cross Abstract: SciCode is the standard measure of the scientific-coding ability of language models: research-level problems that demand both frontier scientific theo

Text2GraphQuery-Bench: A Text to Graph Query Benchmark

arXiv:2602.11745v2 Announce Type: replace Abstract: Graph models are fundamental to data analysis in domains rich with complex relationships. Unlike SQL, which benefits from a rel- atively unified sta

TIDE: A Physically Diverse 3D Turbulence Benchmark Dataset for Advancing Scientific Machine Learning

arXiv:2608.04222v1 Announce Type: cross Abstract: Turbulence is a central testbed for machine learning on physical dynamics because its governing laws are known exactly. However, most existing studies

AI World Cup 2026: Benchmarking Large Language Models for End-to-End Football Tournament Prediction

arXiv:2608.03416v1 Announce Type: new Abstract: Large language models (LLMs) are now regularly asked to forecast real-world events, but comparisons are often difficult because models receive different

DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction

arXiv:2608.03591v1 Announce Type: cross Abstract: Large Language Model (LLM) agents offer a promising approach to attack chain reconstruction by retrieving and interpreting heterogeneous telemetry to

Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks

arXiv:2608.03297v1 Announce Type: new Abstract: A standard claim in the literature on retrieval-augmented and memory-augmented language models is that shorter context is better when the relevant infor

Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluation

arXiv:2608.03791v1 Announce Type: new Abstract: Vision-Language Models (VLMs), like Large Language Models (LLMs), may memorize sensitive, copyrighted, or harmful knowledge from their pretraining corpo

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

arXiv:2608.03206v1 Announce Type: cross Abstract: Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only re

Evaluating LLMs in Database Scenarios: A Lifecycle Benchmark for Assessing Their Potential in Core Database Tasks

arXiv:2608.03794v1 Announce Type: cross Abstract: Large Language Models (LLMs) are transforming database interaction paradigms, evolving from simple query translators to autonomous database administra

HomeSafeBench: A Benchmark for Embodied Vision-Language Models in Free-Exploration Home Safety Inspection

arXiv:2509.23690v2 Announce Type: replace-cross Abstract: Safety hazards in the home are a leading cause of preventable domestic injuries, motivating an automated inspector that actively explores a ho

Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation

arXiv:2608.02639v1 Announce Type: cross Abstract: Production prompts rarely carry a single instruction. One system message may require valid JSON, a word limit, three citations, and a fixed tone at th

MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning

arXiv:2608.03882v1 Announce Type: cross Abstract: Geospatial reasoning, i.e., computing distances, containment, and other spatial relations over real-world entities, is central to navigation and logis

VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs

arXiv:2608.03810v1 Announce Type: cross Abstract: Large language models routinely describe socially salient targets, including political figures, countries, religions, organizations, historical events

When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills

arXiv:2608.03700v1 Announce Type: cross Abstract: Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personaliz

Research

A reproducible and extensible framework for benchmarking competing risks survival models

arXiv:2608.00271v1 Announce Type: cross Abstract: A wide range of statistical and machine learning methods have been proposed for survival analysis with competing risks, where the occurrence of one ev

researcharxiv-cs-lg

Research

Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking

arXiv:2608.01021v1 Announce Type: cross Abstract: In an age of rapid model turnover, how do we make hallucination evaluation more perennial? We explore whether human-written hallucination samples coul

researcharxiv-cs-cl

Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

arXiv:2608.02396v1 Announce Type: new Abstract: Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural network

EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next

arXiv:2603.12147v2 Announce Type: replace Abstract: Egocentric video provides a natural modality for studying human behavior, but conventional visual understanding captures mainly observable scenes, o

First Deployable Dynamic-CoM: A Unified Policy and Method-Agnostic Benchmark for Humanoid Single-Leg Balance

model-releasesarxiv-cs-ro

arXiv:2608.00500v1 Announce Type: new Abstract: Unified humanoid policies handle agile whole-body motion, yet stumble on a simple demand: staying balanced on one leg. On our single-leg-balance benchma

How Benchmarks and Evaluation Protocols Shape Conclusions in Provenance-Based Intrusion Detection

arXiv:2608.01454v1 Announce Type: cross Abstract: Provenance-based intrusion detection systems (PIDS) frequently report strong performance, but the conclusions drawn from these results can be highly s

Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark

arXiv:2608.00106v1 Announce Type: new Abstract: Agentic systems must decide not only what answer to produce, but which reasoning and execution operations should precede it. A controller may answer dir

SoM-1K: A Thousand-Problem Benchmark Dataset for Strength of Materials

arXiv:2509.21079v2 Announce Type: replace Abstract: Foundation models have shown remarkable capabilities in various domains, but their performance on complex, multimodal engineering problems remains l

SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models

arXiv:2608.01709v1 Announce Type: new Abstract: Vision-language models (VLMs) achieve strong semantic understanding but remain unreliable in metric spatial reasoning, particularly when queries require

Agents

SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

arXiv:2608.02499v1 Announce Type: cross Abstract: Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task,

agentsarxiv-cs-cl

What Carries the Signal in Pathology Foundation-Model Atlases? A Patient-Level Controlled Benchmark in Breast Cancer

arXiv:2608.00105v1 Announce Type: new Abstract: Pathology foundation models are reported to encode molecular programmes in tissue morphology, but the evidence is usually a cohort-wide ranked gene list

WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity

arXiv:2608.02603v1 Announce Type: new Abstract: Controllable video generation models are increasingly being developed as world models. Accordingly, evaluating them in this role extends beyond the appa

CBCT-IQ: A Publicly Available Annotated Cone-Beam CT Dataset for Image Quality Assessment and Benchmarking

arXiv:2607.29253v1 Announce Type: cross Abstract: Medical image quality plays a critical role in diagnostic accuracy, especially in X-ray-based imaging modalities such as cone-beam computed tomography

EgoEverything: A Benchmark for Human Behavior Inspired Long Context Egocentric Video Understanding in AR Environment

arXiv:2604.08342v2 Announce Type: replace Abstract: Long context egocentric video understanding has recently attracted significant research attention, with augmented reality (AR) highlighted as one of

LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data

arXiv:2607.28880v1 Announce Type: cross Abstract: Modern multimedia machine learning workloads increasingly store large-scale datasets in cloud object storage services such as AWS S3. How these sample

So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection

arXiv:2505.18660v5 Announce Type: replace Abstract: Recent advances in AI-powered generative models have enabled the creation of increasingly realistic synthetic images, posing significant risks to in