Search: “evaluation”

arXiv:2610.18088v1 Announce Type: cross Abstract: Autoresearch systems increasingly undertake entire research projects. In practice, scientific research requires systems to pursue multiple interdepend

agentsarxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

RFWeave: Learning Wireless Propagation Across Scenes and Transceiver Configurations

DGX agent

arXiv:2610.16656v1 Announce Type: cross Abstract: Wireless propagation modeling is foundational to the evaluation and design of modern communication systems. To achieve accurate wireless propagation p

applicationsarxiv-cs-lg

14 Oct 2026

Same Goal, Different Words: Spatial Instruction Sensitivity in World-Action Models

arXiv:2610.16242v1 Announce Type: new Abstract: World-action models should preserve the intended manipulation across equivalent descriptions of a spatial goal. We evaluate this consistency in 864 roll

researcharxiv-cs-ro

SoK: Rethinking Membership Inference in Tabular Machine Learning

arXiv:2610.18486v1 Announce Type: new Abstract: Membership Inference Attacks (MIAs) are currently the dominant approach to evaluating privacy in machine learning applications. Despite their significan

researcharxiv-cs-lg

Spatiotemporal Recognition of Abdominal Wall Layers During Optical Trocar Entry

arXiv:2610.14512v1 Announce Type: new Abstract: Objective. To evaluate whether spatial context and temporal modeling improve tissue recognition during optical trocar entry, where multiple abdominal-wa

researcharxiv-cs-cv

SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation

arXiv:2610.16338v1 Announce Type: new Abstract: Recent years have witnessed rapid progress in joint audio--visual generation. However, audio--visual synchronization remains a challenge, as generated s

Towards a Systematic Study of Embodied System 2 over Full Task Execution Cycles

arXiv:2610.17657v1 Announce Type: new Abstract: An embodied brain, or System 2, must turn an abstract goal into actions while discovering relevant scene conditions and checking whether executed skills

Video-Arena: A Scalable Agentic Video Benchmark for Spatiotemporal Capability

arXiv:2610.14158v1 Announce Type: new Abstract: Understanding and tracking a dynamic world are fundamental to visual intelligence. Existing video benchmarks provide limited insight into whether models

Agreement Without Alignment: Label-Free Rubric Clarification Increases LLM-Judge Consensus but Not Agreement with Humans

arXiv:2610.13303v1 Announce Type: cross Abstract: LLM judges are increasingly deployed as panels, and disagreement among panel members is often read as a symptom of an unclear rubric. We test a label-

safetyarxiv-cs-ai

Approximation of differential entropy in Bayesian optimal experimental design

arXiv:2510.00734v2 Announce Type: replace-cross Abstract: Bayesian optimal experimental design provides a principled framework for selecting experimental configurations that maximize obtained informat

researcharxiv-cs-lg

AutoBio+: Benchmarking Dexterous Manipulation with Dynamic Experimental Phenomena in Realistic Laboratories

arXiv:2610.13643v1 Announce Type: new Abstract: Automating biological laboratories requires robots to perform dexterous tool interactions under realistic visual conditions while handling dynamic exper

Localized Generative Design of Ship Hulls with Diffusion Models and High-Fidelity Assessment

arXiv:2610.12870v1 Announce Type: cross Abstract: Practical ship design requires localized refinement and performance evaluation. This paper presents a reward-directed diffusion framework for localize

Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning

arXiv:2610.13330v1 Announce Type: new Abstract: Designing tool-integrated reasoning agent systems for complex chemical tasks remains a fundamental challenge. Although recent chemical agents have demon

MC-EvoBench: Auditing Experience-Driven Self-Improvement in Minecraft Agents

arXiv:2610.13829v1 Announce Type: cross Abstract: A central challenge in embodied recursive self-improvement is enabling agents to convert interaction history into capabilities that persist beyond one

Measuring Artifact Reliance in Machine-Generated Text Detectors

arXiv:2610.13583v1 Announce Type: new Abstract: Detectors of machine-generated text score almost perfectly on scientific-literature benchmarks, implying we can reliably distinguish with these detector

researcharxiv-cs-cl

MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph

arXiv:2610.12961v1 Announce Type: cross Abstract: We introduce MVGraph, a benchmark for evaluating large vision-language models (LVLMs) on multi-graph reasoning where the evidence required by a query

Partially Informed Stochastic Optimization via Zeroth-Order Residual Tracking

arXiv:2610.13871v1 Announce Type: new Abstract: Many optimization problems provide direct access to only part of the gradient, while the remaining component must be inferred from function evaluations.

researcharxiv-cs-lg

perturber: A Stateless Service for Semantics-Preserving Text Perturbations

arXiv:2610.12667v1 Announce Type: new Abstract: Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving varian

Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM

arXiv:2610.13660v1 Announce Type: cross Abstract: The Abstraction and Reasoning Corpus (ARC) is increasingly used to evaluate whether neural networks can infer task-specific transformations from few d

researcharxiv-cs-ai

All Verdicts are Not Equal: Rethinking LLM Judge Reliability

arXiv:2610.12083v1 Announce Type: cross Abstract: LLM-as-a-Judge is the standard paradigm for NLP evaluation, yet its systemic reliability remains poorly understood despite being widely treated as a d

safetyarxiv-cs-ai

Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review

arXiv:2610.11087v1 Announce Type: new Abstract: The rapid growth of scientific publishing has strained peer review, particularly in machine learning, raising concerns about declining review quality an

From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents

arXiv:2610.12463v1 Announce Type: cross Abstract: In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google agents reached real systems outside their authorized test scope. The paths

Tutorials

Gaze Estimation for Human-Robot Interaction: Analysis Using the NICO Platform

arXiv:2509.24001v3 Announce Type: replace Abstract: This paper evaluates the current gaze estimation methods within a human-robot interaction (HRI) context of a shared workspace scenario. We introduce

tutorialsarxiv-cs-cv

Language Models for Page-Level Layout Decisions in E-commerce Search

arXiv:2610.10920v1 Announce Type: cross Abstract: E-commerce search pages are critical touchpoints for millions of online shoppers. While traditional search engines return a ranked list of results, mo

researcharxiv-cs-cl

HeiCo-FOCUS: A Clinically Grounded Dataset for Long-Context Video Understanding

arXiv:2610.10156v1 Announce Type: new Abstract: Recent advances in Vision-Language Models (VLMs) have led to rapid progress in video understanding across a wide range of benchmark tasks. However, exis

How Do LLMs Change Predictions Under Negation?

arXiv:2610.09571v1 Announce Type: new Abstract: Negation is an essential feature of human language, yet large language models (LLMs) remain unreliable in processing it. We evaluate recent open-source

How Private is Private? A Comparative Study for Face De-Identification

arXiv:2610.10334v1 Announce Type: new Abstract: Face de-identification (FDeID) has emerged as a critical privacy-preserving technology, yet its evaluation remains fundamentally fragmented. Existing pr

PatchBench: Measuring Collateral Damage in Activation Patching

arXiv:2610.10276v1 Announce Type: cross Abstract: An LLM safety patch can pass a benchmark while still being a poor repair. This risk is especially acute for jailbreak repairs, where the goal is to co

Hardware

PoreML: A Data-Driven Framework for Learning Multiphase Flow in Porous Media

arXiv:2610.10314v1 Announce Type: new Abstract: Multiphase flow in porous microstructures is central to CO_2 storage, fuel-cell operation, and flip-chip packaging. Predicting these flows remains chall

hardwarearxiv-cs-lg

RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment

model-releasesarxiv-cs-lg

arXiv:2610.09294v1 Announce Type: cross Abstract: Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments. As agents move i

Shaer: Controlled Arabic Poetry Generation with Meter Subform and Semantic Conditioning

arXiv:2610.09756v1 Announce Type: new Abstract: Classical Arabic poetry generation requires simultaneously satisfying semantic, linguistic, and fine-grained prosodic constraints. Existing systems typi

researcharxiv-cs-cl

World Models Dream of Success: Diagnosing and Repairing Failure Insensitivity in Robot World Models

arXiv:2610.09134v1 Announce Type: new Abstract: Robot world models support policy evaluation, planning, and synthetic data generation, but these applications require predictions that distinguish succe

Auditable Claims about AI Agents

arXiv:2610.07459v1 Announce Type: new Abstract: Organizations make claims about their AI agents: a person approves every external email, every action is logged, an evaluation shows the agent is safe t

safetyarxiv-cs-ai

CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets

arXiv:2610.07132v1 Announce Type: cross Abstract: Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful read

EnGRICH: Enhancing Generative Reward Modeling with Critiques from Humans

arXiv:2610.05370v2 Announce Type: replace Abstract: Generative reward models (GRMs) are important for LLM optimization. Unlike scalar reward models, GRMs generate natural-language critiques alongside

researcharxiv-cs-ai

EPOCH: Reliable Discovery through Evidence-Governed Search

arXiv:2610.06986v1 Announce Type: new Abstract: AI research agents are increasingly used to search over programs, mathematical constructions, and proofs. However, existing systems typically optimize e

M3SunAgent: Monocular 3D Spatial Understanding Agent for Metric Depth Estimation and 3D Visual Grounding

arXiv:2610.07982v1 Announce Type: new Abstract: Monocular metric depth estimation and 3D visual grounding represent the two complementary cornerstones of monocular 3D spatial understanding (M3Sun), fr

OSGuard: A Benchmark for Safety in Computer-Use Agents

arXiv:2606.15034v2 Announce Type: replace Abstract: Computer-use agents can complete benign user instructions while violating important constraints of the user's environment. We introduce OSGuard, a d

OTel: Open Telco AI Datasets, Benchmarks, and Models

arXiv:2610.07766v1 Announce Type: new Abstract: We present Open Telco (OTel), an open telecom AI resource that releases derived telecom datasets for retrieval, reranking, instruction tuning, and safet

Agents

Partially Observable Zero-shot coordination by Predicting Intention of Partner

arXiv:2610.08142v1 Announce Type: new Abstract: Zero-shot coordination in embodied settings requires acting while the partner is intermittently out of view, leaving existing methods with ambiguous par

agentsarxiv-cs-ai

PERSIST: Who-What-When Memory Across Sessions for Full-Duplex Spoken Dialogue

arXiv:2610.07725v1 Announce Type: new Abstract: Modern voice assistants may be shared by multiple users and should be able to answer questions about earlier conversations such as 'When did I originall

RAG-PIBench: A Leakage-Aware Benchmark for Prompt-Injection Detection in Trustworthy RAG Systems

arXiv:2610.08571v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content. We introduce RAG-PIBench, a ben

RenderBench: Benchmarking Render-to-Real Video Transfer with Reconstructed Digital Twins

arXiv:2610.08684v1 Announce Type: new Abstract: Modern video models can generate realistic videos from real appearance references and proxy renders that specify scene structure, viewpoint changes, and

TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts

arXiv:2610.06824v2 Announce Type: replace Abstract: We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models. We define research taste as the ability to pick i

What the Elevation Map Cannot See: Semantic-Aware Locomotion and Execution-Aware Navigation for Humanoid Robot

arXiv:2610.07396v1 Announce Type: new Abstract: Navigation for humanoid robots is critical, yet large-scale evaluation on physical hardware is often impractical due to cost and safety concerns, making

BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning

arXiv:2610.06571v1 Announce Type: new Abstract: Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences,

6 Oct 2026

EgoExo-Next:Benchmarking Vision-Language Models on Visual-Option Next-State and Cross-View Reasoning

arXiv:2610.04506v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly evaluated for egocentric and cross-view video reasoning, yet existing benchmarks largely focus on semanti

6 Oct 2026

Harmful Content Generation in Text-to-Image Models: Capabilities and Moderation Limitations