Search: “benchmark”

When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor State

arXiv:2605.18580v1 Announce Type: new Abstract: Outcome-only evaluation can certify economically unsafe agents: a policy can hit a business KPI while violating deployable behavioral discipline. In hot

19 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Who Generated This 3D Asset? Learning Source Attribution for Generative 3D Models

DGX agent

arXiv:2605.18132v1 Announce Type: cross Abstract: Generative 3D models are deployed in gaming, robotics, and immersive creation, making source attribution critical: given a 3D asset, can we identify w

model-releasesarxiv-cs-ai

19 May 2026

3D Segmentation Using Viewpoint-Dependent Spatial Relationships

arXiv:2605.15708v1 Announce Type: new Abstract: Recent advances in 3D datasets and multimodal models have greatly improved natural language 3D scene understanding. However, most 3D referring segmentat

A Causally Grounded Taxonomy for Image Degradation Robustness Evaluation

arXiv:2605.15906v1 Announce Type: new Abstract: Image degradations can occur during acquisition, processing, and transmission, altering visual appearance and affecting downstream vision tasks. They ar

A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation

arXiv:2605.15761v1 Announce Type: new Abstract: Evaluation leaderboards such as LMArena play a central role in benchmarking large language models by aggregating pairwise human preferences into model r

researcharxiv-cs-lg

DetectRL-X: Towards Reliable Multilingual and Real-World LLM-Generated Text Detection

arXiv:2605.15518v1 Announce Type: new Abstract: The effective detection and governance of Large Language Model (LLM) generated content has become increasingly critical due to the growing risk of misus

EduVQA: Towards Concept-Aware Assessment of Educational AI-Generated Videos

arXiv:2603.03066v2 Announce Type: replace Abstract: Existing AI-generated video quality assessment (AIGVQA) methods mainly focus on global perceptual realism and coarse text-video alignment, while ove

From Guidelines to Guarantees: A Graph-Based Evaluation Harness for Domain-Specific Evaluation of LLMs

arXiv:2508.20810v3 Announce Type: replace Abstract: Rigorous evaluation of domain-specific language models requires benchmarks that are comprehensive, contamination-resistant, and maintainable. Static

researcharxiv-cs-ai

Local Ai

Is Agentic AI Ready for Real-World Hardware Engineering? A Deep Dive with Phoenix-bench

arXiv:2605.15226v1 Announce Type: cross Abstract: We ask whether agentic AI systems built for software engineering transfer to realistic hardware engineering. Existing hardware LLM benchmarks isolate

local-aiarxiv-cs-ai

MuteBench: Modality Unavailability Tolerance Evaluation for Incomplete Multimodal Fusion

arXiv:2605.15235v1 Announce Type: new Abstract: Multimodal physiological data powers clinical AI systems from intensive care units to wearable devices, but sensors routinely fail in practice. Two fail

Safety

parallelcbf: A composable safety-filter and auditability framework for tensor-parallel reinforcement learning

arXiv:2605.15509v1 Announce Type: new Abstract: While Isaac Lab provides massive parallel UAV simulation, OmniSafe and safe-control-gym provide constrained-RL benchmarks, and CBFKit provides control-b

safetyarxiv-cs-lg

RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades

arXiv:2605.15846v1 Announce Type: cross Abstract: Coding agents are increasingly deployed in real software development, where a single version iteration requires months of coordinated work across many

SOLAR: Self-supervised Joint Learning for Symmetric Multimodal Retrieval

arXiv:2605.15868v1 Announce Type: new Abstract: In this work, we address the critical yet underexplored challenge of symmetric multimodal-to-multimodal (MM2MM) retrieval, where queries and contexts ar

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language

arXiv:2605.15607v1 Announce Type: new Abstract: Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from

researcharxiv-cs-cl

Local Ai

VLMs Trace Without Tracking: Diagnosing Failures in Visual Path Following

arXiv:2605.15672v1 Announce Type: cross Abstract: Vision-language models (VLMs) achieve strong performance on multimodal benchmarks, but may still lack robust control over basic visual operations. We

local-aiarxiv-cs-ai

XSearch: Explainable Code Search via Concept-to-Code Alignment

arXiv:2605.16046v1 Announce Type: cross Abstract: Semantic code search has been widely adopted in both academia and industry. These approaches embed natural-language queries and code snippets into a s

Agents

Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents

arXiv:2510.02837v2 Announce Type: replace Abstract: Although recent tool-augmented benchmarks involve complex requests, evaluation remains limited to answer matching, neglecting critical trajectory as

agentsarxiv-cs-ai

Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia

arXiv:2509.23023v3 Announce Type: replace Abstract: Large language models are increasingly deployed in multi-agent settings whose outcomes hinge on social intelligence, motivating evaluations of their

Descriptor: Distance-Annotated Traffic Perception Question Answering (DTPQA)

arXiv:2511.13397v2 Announce Type: replace-cross Abstract: The remarkable progress of Vision-Language Models (VLMs) on a variety of tasks has raised interest in their application to automated driving.

LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling

arXiv:2605.14186v1 Announce Type: new Abstract: Large language models (LLMs) often expose useful signals of self-monitoring: before solving a problem, they can estimate whether they are likely to succ

On the Cultural Anachronism and Temporal Reasoning in Vision Language Models

arXiv:2605.15071v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied to cultural heritage materials, from digital archives to educational platforms. This work ident

Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study

arXiv:2605.14890v1 Announce Type: new Abstract: Foundation models tokenize Ukrainian legal text with vastly different efficiency, yet no systematic comparison exists for this domain. We benchmark seve

Local Ai

Towards Real-Time Autonomous Navigation: Transformer-Based Catheter Tip Tracking in Fluoroscopy

arXiv:2605.14253v1 Announce Type: new Abstract: Purpose: Mechanical thrombectomy (MT) improves stroke outcomes, but is limited by a lack of local treatment access. Widespread distribution of reinforce

local-aiarxiv-cs-cv

AGOP as Explanation: From Feature Learning to Per-Sample Attribution in Image Classifiers

arXiv:2605.12816v1 Announce Type: new Abstract: The Average Gradient Outer Product (AGOP) governs feature learning in neural networks: the Neural Feature Ansatz states that weight Gram matrices at eac

LeanSearch v2: Global Premise Retrieval for Lean 4 Theorem Proving

arXiv:2605.13137v1 Announce Type: cross Abstract: Proving theorems in Lean 4 often requires identifying a scattered set of library lemmas whose joint use enables a concise proof -- a task we call glob

MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence

arXiv:2605.12703v1 Announce Type: cross Abstract: We introduce MMCL-Bench, a benchmark for multimodal context learning: learning task-local rules, procedures, and empirical patterns from visual or mix

RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning

arXiv:2605.13695v1 Announce Type: cross Abstract: LLM-as-a-judge is now the default measurement instrument for open-ended generation, but on the public JudgeBench benchmark even strong instruction-tun

A Proof-of-Concept Simulation-Driven Digital Twin Framework for Decision-Aware Diabetes Modeling

arXiv:2605.11247v1 Announce Type: new Abstract: This paper presents a proof-of-concept digital twin framework for simulation-driven diabetes modeling using benchmark clinical data, synthetic temporal

AgentDisCo: Towards Disentanglement and Collaboration in Open-ended Deep Research Agents

arXiv:2605.11732v1 Announce Type: cross Abstract: In this paper, we present AgentDisCo, a novel Disentangled and Collaborative agentic architecture that formulates deep research as an adversarial opti

AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor

arXiv:2601.05752v3 Announce Type: replace Abstract: We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across div

Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation

arXiv:2605.11533v1 Announce Type: new Abstract: Clinical check-up reports are multimodal documents that combine page layouts, tables, numerical biomarkers, abnormality flags, imaging findings, and dom

Evaluating the Pre-Consultation Ability of LLMs using Diagnostic Guidelines

arXiv:2601.03627v3 Announce Type: replace Abstract: We introduce EPAG, a benchmark dataset and framework designed for Evaluating the Pre-consultation Ability of LLMs using diagnostic Guidelines. LLMs

HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-bench

arXiv:2601.20255v2 Announce Type: replace-cross Abstract: SWE-bench has emerged as the premier benchmark for evaluating Large Language Models on complex software engineering tasks. While these capabil

PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments

model-releasesarxiv-cs-ro

arXiv:2605.11534v1 Announce Type: new Abstract: When an LLM-based embodied agent fails at a household task, the culprit could be misidentified objects, forgotten sub-goals, or poor action sequencing -

Robust Promptable Video Object Segmentation

arXiv:2605.12006v1 Announce Type: new Abstract: The performance of promptable video object segmentation (PVOS) models substantially degrades under input corruptions, which prevents PVOS deployment in

STRUM: A Spectral Transcription and Rhythm Understanding Model for End-to-End Generation of Playable Rhythm-Game Charts

arXiv:2605.12135v1 Announce Type: cross Abstract: We present STRUM (Spectral Transcription and Rhythm Understanding Model), an audio-to-chart pipeline that converts raw recordings into playable Clone

Targeted Tests for LLM Reasoning: An Audit-Constrained Protocol

arXiv:2605.11599v1 Announce Type: new Abstract: Fixed reasoning benchmarks evaluate canonical prompts, but semantically valid changes in presentation can still change model behavior. Studies of prompt

researcharxiv-cs-lg

Agentic MIP Research: Accelerated Constraint Handler Generation

arXiv:2605.09186v1 Announce Type: new Abstract: Mixed-integer programming (MIP) research is both mathematically sophisticated and engineering-intensive: testing an algorithmic hypothesis within a bran

AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

arXiv:2605.08704v1 Announce Type: new Abstract: Multi-agent reasoning has shown promise for improving the problem-solving ability of large language models by allowing multiple agents to explore divers

Applications

Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solvers

arXiv:2602.04189v2 Announce Type: replace Abstract: Uncertainty evaluation is critical in scientific and engineering inverse problems. However, existing benchmarks on Diffusion Inverse Solvers (DIS) p

applicationsarxiv-cs-lg

Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomies

arXiv:2601.12369v3 Announce Type: replace Abstract: Deep Research Agents increasingly automate survey generation, yet whether they match human experts at retrieving essential papers and organizing the

Computer Use at the Edge of the Statistical Precipice

arXiv:2605.08261v1 Announce Type: cross Abstract: Evaluating Computer Use Agents (CUAs) on interactive environments is fraught with methodological pitfalls that the field has yet to systematically add

CREATE: Testing LLMs for Associative Creativity

arXiv:2603.09970v2 Announce Type: replace Abstract: A key component of creativity is associative reasoning: the ability to draw novel yet meaningful connections between concepts. We introduce CREATE,

Tutorials

Developing a foundation model for high-resolution remote sensing data of the Netherlands

arXiv:2605.10184v1 Announce Type: cross Abstract: We develop a foundation model using 1.2m high resolution satellite images of the Netherlands. By combining a Convolutional Neural Network and a Vision

tutorialsarxiv-cs-ai

EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints

arXiv:2605.08636v1 Announce Type: new Abstract: Federated fine-tuning offers a promising paradigm for adapting large language models (LLMs) on edge devices by leveraging the rich, diverse, and continu

Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness

arXiv:2509.13332v2 Announce Type: replace Abstract: As Large Language Models (LLMs) are increasingly adopted as automated judges in benchmarking and reward modeling, ensuring their reliability, effici

Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?

arXiv:2603.00166v2 Announce Type: replace-cross Abstract: Recent advances in generative AI have shown human-level performance in complex content creation. However, we identify a 'Paradox of Simplicity

Applications

From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

arXiv:2605.10834v1 Announce Type: new Abstract: AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perfor

applicationsarxiv-cs-ai