Search: “evaluation”

arXiv:2609.36805v1 Announce Type: new Abstract: Large language model (LLM) agents reuse external memory to guide new tasks, but effective retrieval requires learning which memory sets improve executio

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

VeriWeave Govern: Evidence-Gated Deterministic Runtime Governance for Enterprise AI Agents

DGX agent

arXiv:2609.37457v1 Announce Type: new Abstract: Enterprise artificial-intelligence agents increasingly call tools, modify infrastructure, and process protected data, creating a need to separate action

model-releasesarxiv-cs-ai

30 Sept 2026

WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses

model-releasesarxiv-cs-ai

arXiv:2609.36635v1 Announce Type: cross Abstract: Bug validation asks a coding agent to produce an executable witness for a reported bug. The witness combines a concrete input with a testing harness a

WorldLine: Action-Driven Visual Simulation for Robotic Manipulation

arXiv:2609.38059v1 Announce Type: cross Abstract: Real-world robot learning is constrained by the cost of collecting experience and evaluating candidate behaviors. Video generation models offer a scal

You Cannot Recover What Was Never Measured: Quantifying the Information Ceiling of Ultra-Low-Field MRI Super-Resolution

arXiv:2609.36837v1 Announce Type: new Abstract: Generative super-resolution models can turn portable 64 mT MRI into images that look like 3T scans, and the field evaluates them with PSNR, SSIM, and pi

AnesTRACE: Benchmarking Intraoperative Anesthesia from Multimodal Perception to Multi-step Decision-Making

arXiv:2609.32740v1 Announce Type: new Abstract: Intraoperative anesthesia requires systems to interpret evolving multimodal evidence, recommend timely management, and revise decisions as patient state

Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study

arXiv:2605.06643v2 Announce Type: replace Abstract: Despite the growing popularity of Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported perfor

Concept Score Relearning: A Unified Cross-Architecture Attack on Concept Erasure

arXiv:2609.33445v1 Announce Type: new Abstract: Concept erasure aims to suppress undesirable knowledge in text-to-image generative models. However, existing robustness evaluations typically rely on re

ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts

model-releasesarxiv-cs-ro

arXiv:2609.31792v1 Announce Type: cross Abstract: While Vision-Language-Action (VLA) models perform strongly on manipulation tasks, their responses to invalid task premises remain underexplored. Exist

Contamination, Prior, or Evidence? Decomposing and Training Evidence Use in Whole-Slide Vision-Language Models

arXiv:2609.32185v1 Announce Type: new Abstract: Pathology vision-language models (VLMs) are conventionally evaluated by accuracy, but accuracy alone does not measure evidence use: it may conflate data

Convex Optimization Is Free When Accuracy Is Expensive

arXiv:2609.35418v1 Announce Type: cross Abstract: This paper studies convex optimization when the gradient cannot be evaluated exactly, but only approximated by a hierarchy of algorithms whose compute

safetyarxiv-cs-lg

Deep Weighted Bellman Residual Minimization for Q^* Estimation

arXiv:2609.34593v1 Announce Type: new Abstract: Off-policy evaluation is a foundational component of offline reinforcement learning, aiming to assess and optimize policy performance using pre-collecte

safetyarxiv-cs-lg

Disentangle and Drop: Robust Universal Removal of Image Watermarks via Reconstructive Grayscale Residual Decomposition

arXiv:2609.31693v1 Announce Type: new Abstract: Invisible image watermarks are commonly evaluated against benign postprocessing operations such as compression, resizing, blur, and color changes. These

researcharxiv-cs-cv

DS-VLA: A Dendritic-inspired Vision-Language-Action Model for Robust Action Control

arXiv:2609.32253v1 Announce Type: new Abstract: Vision-language-action (VLA) models have achieved strong performance in language-conditioned manipulation, yet success under nominal evaluation does not

researcharxiv-cs-ro

EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis

arXiv:2609.31632v1 Announce Type: new Abstract: Electroencephalography (EEG) analysis is evolving from short-segment classification toward long-horizon interpretation that demands iterative evidence a

EngIntervene: Benchmarking Multimodal Engineering State Understanding and Design Intervention Reasoning

arXiv:2609.33261v1 Announce Type: new Abstract: Multimodal engineering benchmarks largely evaluate static understanding, such as recognizing components, interpreting diagrams, or answering technical q

EyeVQA: Benchmarking Ophthalmic Vision-Language Models from Recognition to Spatial Grounding

arXiv:2609.32352v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown increasing potential for medical image understanding, yet their capabilities in ophthalmic imaging remain insuf

Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models

arXiv:2609.32069v1 Announce Type: new Abstract: Vision--language--action (VLA) models provide strong priors for robotic manipulation but are typically deployed as frozen policies, unable to improve fr

safetyarxiv-cs-ro

From One-Shot Generation to Incremental Music Composition: Adapting a General-Purpose Instruction LLM for Persistent Symbolic Editing

arXiv:2609.34994v1 Announce Type: cross Abstract: Most music-generation systems are still framed and evaluated primarily as producers of complete outputs, whereas composition often proceeds through su

JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments

arXiv:2609.35032v1 Announce Type: cross Abstract: In complex embodied visual reasoning scenarios, an agent often has only a limited field of view, and the evidence needed to answer a question may be d

Learning High-Risk High-Precision Motion Control

arXiv:2609.34851v1 Announce Type: new Abstract: Deep reinforcement learning (DRL) algorithms for movement control are typically evaluated and benchmarked on sequential decision tasks where imprecise a

safetyarxiv-cs-lg

Learning to Act under Visual Interruptions with Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2609.35003v1 Announce Type: new Abstract: Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with a

Agents

M3OS: A Monte Carlo Graph Search-Orchestrated Multi-Agent LLM System for Evidence-Traced Molecular Optimization

arXiv:2609.34491v1 Announce Type: new Abstract: Small-molecule optimization integrates medicinal-chemistry reasoning and computational evidence through iterative, multi-objective decisions. When large

agentsarxiv-cs-lg

Tutorials

Observability-Informed Optimal Sensor Placement for Soft Robots

arXiv:2609.33681v1 Announce Type: new Abstract: This paper presents the application and experimental evaluation of a systematic method for optimal sensor placement in soft robots. Existing methods eit

tutorialsarxiv-cs-ro

Patient-level validation of a foundation-model pipeline for pediatric lung sounds: physician-annotated adventitious events are recognized, disease groups are not reliably predicted

arXiv:2603.15688v2 Announce Type: replace-cross Abstract: Background: Lung-sound classifiers are usually evaluated on recording- or event-level splits, although each child contributes many recordings.

PDMD: Projected Distribution Matching Distillation for Video Diffusion Models

arXiv:2609.35768v1 Announce Type: new Abstract: Modern video diffusion models require tens of denoising evaluations over long spatiotemporal token sequences. Distribution Matching Distillation (DMD) r

researcharxiv-cs-cv

Recovery-Directed Symbolic Distillation of Neural Likelihoods

arXiv:2609.32409v1 Announce Type: new Abstract: Amortized neural likelihoods enable computationally expensive inference for models with analytically intractable or unspecified likelihoods, but their b

Recursive LLM Degradation in Biomedical Question Answering: A Cross-Generation Study

arXiv:2609.34257v1 Announce Type: cross Abstract: Repeatedly training language models on their own generated data may create a synthetic-data feedback loop in which errors and distributional biases ar

Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

arXiv:2609.35441v1 Announce Type: new Abstract: State space models (SSMs) achieve efficient sequence processing because their affine state updates are closed under composition and can therefore be eva

Scaling Versatile 3D Assets Editing with a Million-Scale Dataset

arXiv:2609.34271v1 Announce Type: new Abstract: Although recent 3D generative models produce increasingly realistic assets, controllable 3D asset editing remains challenging. Existing methods are limi

SPACE: Sparse Predictive Attractor via Counterfactual Eviction for Streaming Video Memory

arXiv:2609.32592v1 Announce Type: new Abstract: Fixed-capacity streaming video memory requires repeated eviction decisions whose effects accumulate over time. Yet existing policies are evaluated prima

The Composition Gap in Dataset Distillation

arXiv:2609.34343v1 Announce Type: new Abstract: Dataset distillation compresses a training set into a small synthetic set, usually evaluated one at a time. In federated and data-governance settings, s

The Price of Peeking: Anytime-Valid Leakage Detection on ML-KEM EM Traces

arXiv:2609.33597v1 Announce Type: cross Abstract: Side-channel evaluators routinely inspect leakage tests while acquisition is still running, and extend or stop the campaign based on what they see. Fi

Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?

arXiv:2609.35020v1 Announce Type: new Abstract: Vision Sparse Autoencoders (SAEs) have become a popular tool in Mechanistic Interpretability due to their presumed ability to disentangle complex featur

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

arXiv:2609.35028v1 Announce Type: new Abstract: Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosyste

Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

arXiv:2609.34065v1 Announce Type: cross Abstract: Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such ev

Word Similarity Datasets for Indian Languages: Annotation and Baseline Systems

arXiv:2609.34138v1 Announce Type: cross Abstract: With the advent of word representations, word similarity tasks are becoming increasing popular as an evaluation metric for the quality of the represen

ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Models

arXiv:2609.33314v1 Announce Type: new Abstract: Zero-shot graph models (ZGMs), which learn transferable knowledge from source graphs and directly apply to unseen target graphs without any adaptation,

A Benchmark Framework for Screening Automation in Systematic Reviews

model-releasesarxiv-cs-cl

arXiv:2609.30298v1 Announce Type: new Abstract: Systematic reviews (SR) are essential for evidence-based research, but their screening phase is highly time-consuming and labor-intensive. Large languag

AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth

model-releasesarxiv-cs-cl

arXiv:2609.30483v1 Announce Type: cross Abstract: Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the sig

Aligning One-Step Generative Models with Reward-Weighted Transport Distillation

arXiv:2609.30840v1 Announce Type: cross Abstract: One-step generators enable high-quality visual generation with a single network evaluation, but their post-training is difficult: general implicit gen

Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport

arXiv:2609.31470v1 Announce Type: cross Abstract: Outliers are essential for evaluating and improving the robustness of machine learning systems, especially when future distributions may differ signif

Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer

model-releasesarxiv-cs-ai

arXiv:2609.31587v1 Announce Type: cross Abstract: We investigate whether natural-language documentation helps coding agents resolve software issues, and we build the tools to construct and evaluate it

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

model-releasesarxiv-cs-ai

arXiv:2609.31103v1 Announce Type: cross Abstract: Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language re

DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration

arXiv:2609.31215v1 Announce Type: new Abstract: Large language models (LLMs) as a judge enable scalable evaluation, but their judgments can be sensitive to response order and, even after removing such

safetyarxiv-cs-ai

LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting

arXiv:2609.30393v1 Announce Type: new Abstract: Selecting informative camera views is critical for efficient training and adaptive refinement in 3D Gaussian Splatting, where each observation significa

researcharxiv-cs-cv

Agents

Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms

arXiv:2609.30558v1 Announce Type: cross Abstract: Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often colla

agentsarxiv-cs-ai

Same Text, Different Numbers: The Divergence of LLM-Based Measures

arXiv:2609.31013v1 Announce Type: new Abstract: Researchers increasingly use generative large language models (LLMs) to convert corporate text into empirical variables. We examine the extent to which

researcharxiv-cs-ai