Search: “evaluation”

A Baseline Assessment for the Agentic Web Technology Stack

arXiv:2610.13703v1 Announce Type: new Abstract: The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming t

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Accelerating Linear Bandits with Persistent Candidate Sets

DGX agent

arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of K actions, requiring O(K) score evaluations p

researcharxiv-cs-lg

12 Oct 2026

AI Models Can Provably Hide Arbitrary Capabilities

arXiv:2610.13928v1 Announce Type: new Abstract: Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilit

Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation a

safetyarxiv-cs-lg

Agents

Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests

arXiv:2610.12849v1 Announce Type: cross Abstract: Current software engineering agents are evaluated predominantly inside isolated sandboxes against synchronous local tests. However, deploying features

agentsarxiv-cs-ai

Critical Mass: Structure-Disjoint Splits Drift With the Libraries They Are Built On

arXiv:2610.12694v1 Announce Type: new Abstract: Molecular machine learning evaluates out-of-distribution performance with structure disjoint splits, a construction rule fixed once at the scale its lib

DataCupid: Stable Matching Repair for Training Data Acquisition with Delayed Feedback

arXiv:2610.12960v1 Announce Type: new Abstract: Matching repair after utility changes can reuse valid observations, but new evaluations must respect matching capacity and may return out of order. We s

DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design

arXiv:2610.13116v1 Announce Type: new Abstract: As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce s

GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks

arXiv:2610.12669v1 Announce Type: cross Abstract: Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. Th

How Reliable are AMR Parser Outputs? Uncertainty Estimation for AMR Parsing

arXiv:2610.13236v1 Announce Type: cross Abstract: Recent advances have significantly improved the performance of Abstract Meaning Representation (AMR) parsers, yet determining when their outputs can b

researcharxiv-cs-ai

Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing

arXiv:2610.12881v1 Announce Type: new Abstract: Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world model

OccludeBench: Can VLMs Find What's Missing?

arXiv:2610.13790v1 Announce Type: cross Abstract: How well can vision-language models (VLMs) reason about what is missing from an image? We study this question with OccludeBench, a manually curated be

Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear?

arXiv:2610.13550v1 Announce Type: new Abstract: Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions. However, little work investigates how thi

SWE-Duel: Towards a Self-Scaling Arena for Coding Agents

arXiv:2610.12569v1 Announce Type: new Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue tha

Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf

arXiv:2610.13036v1 Announce Type: new Abstract: Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings

arXiv:2610.13247v1 Announce Type: cross Abstract: Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validat

safetyarxiv-cs-cl

AdaptEvo: Adaptive Agent Learning with Evolving Supervision

arXiv:2610.11354v1 Announce Type: new Abstract: Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in d

safetyarxiv-cs-cl

Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data

arXiv:2610.11362v1 Announce Type: cross Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference c

researcharxiv-cs-cv

DataSense-Bench: The First Step Toward an AI Scientist

arXiv:2610.12190v1 Announce Type: new Abstract: As claims about recursive self-improvement (RSI) and artificial general intelligence (AGI) proliferate, we ask a simple question: do frontier AI models

Design Creativity Bench: Measuring creativity in LLM-Generated UI

arXiv:2610.11539v1 Announce Type: cross Abstract: As leading LLMs improve on capability evaluations, their limitations in producing creative outputs on design tasks remain insufficiently characterised

Distillation for Incrimination and Distillation for Capabilities

arXiv:2610.11012v1 Announce Type: new Abstract: Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. Howeve

safetyarxiv-cs-ai

Tutorials

Exploiting Gradients in Bayesian Inference of Expensive Simulators

arXiv:2610.12076v1 Announce Type: new Abstract: Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation-based inference to evaluate the

tutorialsarxiv-cs-lg

Hardware

Language Models as AI Research World Models

arXiv:2610.12235v1 Announce Type: new Abstract: AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet thei

hardwarearxiv-cs-cl

Marformer: A Transformer for Predicting Missing Data Distributions

arXiv:2610.12379v1 Announce Type: new Abstract: Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The extbf{cond

MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

arXiv:2610.09484v2 Announce Type: replace Abstract: Training and evaluating interactive language agents typically requires rich user interactions, yet collecting human feedback is expensive and diffic

Agents

MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovation

arXiv:2610.11966v1 Announce Type: new Abstract: Research idea innovation is a fundamental engine of scientific progress, yet it remains difficult to generate and evaluate in a scalable and controllabl

agentsarxiv-cs-ai

OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences

arXiv:2610.11118v1 Announce Type: new Abstract: The next frontier for artificial general intelligence is tackling unresolved scientific problems, calling for benchmarks that assess progress beyond est

Agents

Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulators

arXiv:2610.11015v1 Announce Type: new Abstract: As voice agents gain more popularity commercially, the user simulators used to evaluate the deployed agents are also being developed to include more rea

agentsarxiv-cs-cl

RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation

arXiv:2610.11183v1 Announce Type: cross Abstract: Following retrieved evidence does not guarantee factual correctness: misleading evidence can induce a model to replace an answer it previously gave co

researcharxiv-cs-ai

READ-Bench: Benchmarking Historical Instance Retrieval for Time-Series Diagnosis

arXiv:2609.32123v2 Announce Type: replace Abstract: Time-series diagnostic systems rarely rely on retrieving relevant historical cases, and when they do, retrieval is evaluated only indirectly through

Reasoning-Informed Visual Editing

model-releasesarxiv-cs-cv

arXiv:2610.12343v1 Announce Type: new Abstract: Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, p

SAVU-BENCH: A Real-World Benchmark for Spatial Audio-Visual Understanding

arXiv:2610.10624v1 Announce Type: cross Abstract: Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modal

Strategic Governance of AI Models in Earth Science

arXiv:2610.10560v1 Announce Type: cross Abstract: AI foundation models pretrained on weather and climate data are increasingly fine-tuned to Earth science tasks well beyond weather forecasting. Their

TacHair: Tactile Contact-Distribution Guided Online Correction for Robotic Hair Stroking and Perception

arXiv:2610.10637v1 Announce Type: new Abstract: Hair stroking is common in daily grooming and personal care, and is also widely used in hair-product evaluation, motivating robots with similar physical

safetyarxiv-cs-ro

TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models

arXiv:2610.11392v1 Announce Type: new Abstract: System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generati

VESSI - VLM-Enhanced Support for Surveillance and Investigations

arXiv:2610.11674v1 Announce Type: new Abstract: Automated video surveillance analysis has become a critical component of intelligence infrastructures and Law Enforcement agencies. Traditional systems

researcharxiv-cs-cv

BEACON-SP: Ontology-Grounded GraphRAG Framework for Clinical Suicide Risk Assessment

arXiv:2610.09026v1 Announce Type: cross Abstract: We present BEACON-SP, an ontology-grounded Graph Retrieval-Augmented Generation (GraphRAG) framework for clinician-facing decision support in behavior

CARE: Certifying Acceleration for Vision-Language-Action Inference

arXiv:2610.08917v1 Announce Type: new Abstract: While vision-language-action (VLA) models have advanced rapidly, running them at every control step remains expensive. Prior work accelerates VLA infere

Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy

arXiv:2610.09906v1 Announce Type: cross Abstract: Security Operations Centers (SOCs) for information technology and operational technology share one incident-response problem: a flood of correlated al

safetyarxiv-cs-cl

Few-Shot Learning for Personalised Automated Pain Assessment

arXiv:2610.09692v1 Announce Type: new Abstract: Pain perception varies substantially across individuals, making it difficult for population-based classifiers to generalise across all subjects in a dat

FinVector-Market-4B: A Controlled Study of LoRA Adaptation for Structured Financial Tasks

arXiv:2610.08882v1 Announce Type: cross Abstract: FinVector-Market-4B adapts Qwen/Qwen3.5-4B with rank-16 LoRA on a 22,000-example corpus for structured financial tasks. We evaluate the base and adapt

InstanceBench: Diagnosing Referential Reasoning and Target Identity in Referring Expression Segmentation

model-releasesarxiv-cs-cv

arXiv:2610.09478v1 Announce Type: new Abstract: Referring Expression Segmentation (RES) links natural-language descriptions to pixel-level object masks. Yet standard evaluation provides limited insigh

Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurements

arXiv:2610.10366v1 Announce Type: new Abstract: Feature caching accelerates diffusion sampling by replacing expensive network evaluations with predictions from previously computed activations. However

Lower Bounds for Parallel Diffusion Sampling

arXiv:2610.09166v1 Announce Type: cross Abstract: Standard diffusion samplers generate samples through repeated evaluations of a learned score function. Parallel sampling methods seek to accelerate ge

OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified Sim-and-Real Framework

model-releasesarxiv-cs-ro

arXiv:2610.10384v1 Announce Type: new Abstract: Tactile feedback provides embodied agents with physical information beyond visual observations, enabling more reliable interaction with the real world.

The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules

arXiv:2610.09239v1 Announce Type: cross Abstract: Self-improving LLM systems propose changes to themselves and keep those that score better on a small evaluation set. We treat this keep-if-better step

Applications

Towards Financial World Modeling

arXiv:2610.09048v1 Announce Type: new Abstract: Building a world model requires a state representation useful for planning and decision-making---potentially over tasks unknown at training time. In the

applicationsarxiv-cs-lg

Adaptive Gait Biofeedback With Participant-Held-Out Modeling and Participant-Specific Updating in Chronic Ankle Instability