Search: “evaluation”

Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents

arXiv:2609.27353v1 Announce Type: new Abstract: Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reprod

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

How Sensitive Are LLM Leaderboard Claims to Hidden Model Selection?

DGX agent

arXiv:2609.28177v1 Announce Type: cross Abstract: LLM leaderboard gains can reflect selection among privately evaluated model variants, yet neither the number of variants nor their dependence is publi

researcharxiv-cs-lg

24 Sept 2026

LEGO: Synergizing Expert GraphRAG and Expert Chain-of-Thought for Legal Reasoning

arXiv:2609.27009v1 Announce Type: new Abstract: Large language models are increasingly applied to high-risk domains such as law, yet complex legal reasoning remains limited by two structural challenge

applicationsarxiv-cs-cl

Tutorials

LLM-Powered Socially Assistive Robot-Delivered Cognitive Behavioral Therapy Exercises: an Exploratory Study with University Students

arXiv:2402.17937v2 Announce Type: replace Abstract: Mental health is a significant healthcare challenge, and cognitive behavioral therapy (CBT) is a widely used therapeutic method for treating anxiety

tutorialsarxiv-cs-ro

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

arXiv:2608.24707v2 Announce Type: replace Abstract: While text-based hallucination detection is well studied, reference-free detection of factual alterations in speech remains underexplored, especiall

LWCal: Loss-Weighted Calibration for Tabular Classifiers with Noisy Calibration Labels

arXiv:2609.26839v1 Announce Type: cross Abstract: Post-hoc probability calibration is usually evaluated under an optimistic assumption: the held-out calibration labels are clean. In many AI deployment

researcharxiv-cs-ai

Local Ai

M3D-Net: Hierarchical Coordination of Spatial Context, Feature Reuse, and Differential Attention for Mammography Classification

arXiv:2609.27523v1 Announce Type: new Abstract: Breast image classification requires local detail and global tissue context, yet these cues can weaken as representations deepen. We present M3D-Net, a

local-aiarxiv-cs-cv

Memory That Changes Action Is Not Memory That Guides It: Counterfactual Auditing of History-Conditioned Robot Policies

arXiv:2609.27247v1 Announce Type: new Abstract: A robot returning a block to its origin tray may encounter two task-consistent pasts that reconverge to the same current input but warrant different act

MSK-Bench: Benchmarking Full-Body Musculoskeletal Motor Control Across Tasks, Control Paradigms, and Physiological Metrics

arXiv:2609.26872v1 Announce Type: new Abstract: Musculoskeletal (MSK) humanoids provide a physiologically grounded embodiment for studying full-body motor control, but their high-dimensional muscle ac

Local Ai

nnFoundation: 3D Foundation Models for Radiology

arXiv:2609.26924v1 Announce Type: new Abstract: Radiological artificial intelligence has advanced rapidly, yet most systems remain narrowly task-specific, data-intensive, and fragile under domain shif

local-aiarxiv-cs-cv

PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety

arXiv:2609.28197v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents that alter real-world states, ensuring operational safety across multi-step workflows has

Quantum score matching with applications to learning thermal states

model-releasesarxiv-cs-lg

arXiv:2609.28391v1 Announce Type: cross Abstract: Score matching has driven major advances in classical generative learning by enabling models to learn from data without evaluating intractable normali

Reference-Based Analysis of Coherence and Diversity in Open-Ended Text Generation

arXiv:2609.28080v1 Announce Type: new Abstract: Evaluating open-ended text generation involves understanding how different properties of a continuation relate to its perceived quality. We present a re

safetyarxiv-cs-cl

Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets

arXiv:2609.27981v1 Announce Type: new Abstract: KV-cache eviction is typically evaluated through average quality-memory trade-offs, yet a small average loss can hide requests whose utility degrades ma

State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State

arXiv:2609.27606v1 Announce Type: new Abstract: We introduce State-Grounded Conditioning (SGC), a design principle for user-facing LLM agents that must condition on live user state (game state, sessio

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

model-releasesarxiv-cs-cv

arXiv:2512.11490v2 Announce Type: replace Abstract: Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both regio

What Changed? Drift Detection with Real, Virtual, and Incomparable Diagnosis

arXiv:2609.27865v1 Announce Type: cross Abstract: Sharing a deep encoder does not, by itself, fix the central confound of task-comparison scores. We show that cross-evaluated heads on a frozen shared

researcharxiv-cs-ai

Agents

What Makes a Terminal-Bench Task Hard? Separating Genuine Hardness from Fake-Hardness on an Adjudicated Agentic Corpus

arXiv:2609.26826v1 Announce Type: cross Abstract: Frontier benchmarks need tasks that current models cannot solve. But a task that no model solves is not automatically a hard task. The same zero pass

agentsarxiv-cs-ai

When Visual Quality Misleads: Intent Recognition under Rendered Avatar Distortions

arXiv:2609.27560v1 Announce Type: cross Abstract: Avatar-streaming systems are commonly evaluated with image and video quality assessment (IQA/VQA) metrics, implicitly treating visual fidelity as a pr

safetyarxiv-cs-cv

Benchmarking Robots for Everyday Environments: From Lab Experiments to Real-World Operations

arXiv:2609.26490v1 Announce Type: new Abstract: This study introduces an interdisciplinary framework for benchmarking robots deployed in public environments, addressing the gap between traditional lab

Beyond End-Task Success: How to Audit Visual Experience Retrieval in Robotics

arXiv:2609.26567v1 Announce Type: new Abstract: Robots that store past experiences must select which one to reuse in a new scene. Most systems select by visual similarity, and most evaluations report

Building Trustworthy Mental Health Benchmarks on Bluesky: A Validation-Aware Weak-Supervision Framework

arXiv:2609.22696v1 Announce Type: new Abstract: Decentralized social media platforms create new opportunities and challenges for computational mental health research because data access, moderation, l

CableVLA: Simulation-Privileged Global-Local Representation Learning for Cable Routing

arXiv:2609.25606v1 Announce Type: new Abstract: Cable routing requires coordinated control of global cable topology and changing local contacts. We present CableVLA, an end-to-end multimodal vision-la

Local Ai

Can We Predict Anomaly Detection Performance from Embedding-Space Geometry?

arXiv:2609.26460v1 Announce Type: new Abstract: Anomaly detection systems are often trained using normal data alone, while model selection and evaluation typically require labeled anomalies. We study

local-aiarxiv-cs-lg

Concept Drift from a Causal Perspective

arXiv:2609.25340v1 Announce Type: new Abstract: Concept drift is a common phenomenon in real-world data streams, in which changes in the data-generating distribution can degrade predictive model perfo

applicationsarxiv-cs-lg

Designing an Efficient Excavator Bucket for Lunar ISRU: A Comparative Study with Vision-Based Fill and Displacement Analysis

arXiv:2609.25724v1 Announce Type: new Abstract: This paper present a spiral-cavity wheel for lunar regolith excavation and a sensor-light evaluation stack that jointly estimates fill ratio (vision), s

researcharxiv-cs-ro

Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs

arXiv:2609.22851v2 Announce Type: cross Abstract: Large Audio Language Models (LALMs) utilize either continuous features or discrete tokens, yet the optimal representation paradigm for general audio u

Agents

Divergent strategies and convergent outcomes in autonomous materials discovery

arXiv:2609.23957v1 Announce Type: new Abstract: Scientific agents are mostly evaluated on whether they complete tasks or recover known results; we instead study variation across repeated open-ended ca

agentsarxiv-cs-ai

DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving

arXiv:2609.26792v1 Announce Type: new Abstract: Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene featur

EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation

arXiv:2609.24115v1 Announce Type: new Abstract: Tool-calling LLM agents are increasingly deployed in enterprise applications. However, effective evaluation and optimization require high-quality, diver

Fairly Compensated Distributed Information Retrieval and Augmentation for AI Agents

arXiv:2609.22601v1 Announce Type: cross Abstract: The increasing reliance of autonomous AI agents on external and distributed knowledge sources introduces a fundamental challenge for decentralized inf

safetyarxiv-cs-ai

FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability

arXiv:2609.25192v1 Announce Type: new Abstract: Financial search is a highly demanding task for LLM agents, requiring not only a correct final answer but also temporally valid information retrieval, a

FMMD: A multimodal multidisciplinary dataset of open peer reviews from F1000Research

arXiv:2602.14285v2 Announce Type: replace-cross Abstract: Automated scholarly paper review (ASPR) has entered the coexistence phase with traditional peer review, where artificial intelligence (AI) sys

safetyarxiv-cs-cl

Higher-Order Approximation of Exit Functionals in Sampling-Based Stochastic Model Predictive Control

arXiv:2609.25257v1 Announce Type: cross Abstract: Safety evaluation in sampling-based stochastic model predictive control often requires numerical estimation of exit functionals. The approximation of

Tutorials

In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization

arXiv:2609.25836v1 Announce Type: new Abstract: Multi-task optimization (MTO) addresses a set of optimization tasks simultaneously, often suffering from inaccurate inter-task relationship estimation u

tutorialsarxiv-cs-lg

Interpretable AI plus Handheld, Portable Retinal Photographs: A Low-Cost Glaucoma Screening Solution for West Africa

arXiv:2609.25697v1 Announce Type: new Abstract: Purpose: To develop and evaluate an interpretable artificial intelligence (AI) framework for glaucoma screening from low-cost portable, handheld retinal

researcharxiv-cs-cv

MCPGen: Benchmarking LLMs on Executable MCPWorkflow Development

arXiv:2609.23925v1 Announce Type: cross Abstract: We study whether LLMs can produce executable workflow artifacts that remain consistent across graph structure, tool implementation, schema bindings, a

MOLA LiDAR-Inertial Odometry (MOLA-LIO) on the COMFORT Localization Benchmark

arXiv:2609.25813v1 Announce Type: new Abstract: This short report documents our entry to the COMFORT Localization Benchmark (IROS 2026), evaluated on the GrandTour dataset recorded with the Boxi paylo

One Domain, Many Tongues: Composing Domain and Language LoRAs for Cross-Lingual Remote-Sensing MLLMs without Paired Data

arXiv:2609.26097v1 Announce Type: new Abstract: Remote-sensing (RS) multimodal large language models (MLLMs) are trained and evaluated only in English, while text-only instruction data covers over 100

researcharxiv-cs-cl

PINNForge: Execution-Grounded Evolutionary Design of Physics-Informed Neural Networks for PDE Solving via Large Language Models

arXiv:2609.23023v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) require coordinated choices over network representation, sampling, loss construction, and optimization, while e

researcharxiv-cs-ai

Predictive Uncertainty for Neural CAE Surrogates

arXiv:2609.25430v1 Announce Type: new Abstract: Neural surrogates can substantially accelerate computer-aided engineering (CAE) workflows, but their use in design requires uncertainty estimates that r

researcharxiv-cs-lg

Same Quantity, Different Answer: Numerical Representation Invariance in Language Models

arXiv:2609.25009v1 Announce Type: new Abstract: Numerically equivalent word problems should yield the same canonical answer whether a quantity is written as a decimal, fraction, percentage, number wor

SelfOp: An Optimization Algorithm for Self-Improving Security Agents

arXiv:2609.22792v1 Announce Type: cross Abstract: LLM agents are increasingly used for security tasks: vulnerability discovery, exploit reproduction, and patch generation. Improving them at the model

Agents

Social Influence and the Allocation of Scientific Attention in AI Populations

arXiv:2609.22408v1 Announce Type: new Abstract: AI systems are becoming participants in the evaluation and use of scientific research. They encounter citation counts, download statistics and lists of

agentsarxiv-cs-ai

The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents

arXiv:2609.22818v1 Announce Type: cross Abstract: Memory-poisoning defenses for LLM agents are typically evaluated by their ability to prevent attacks. However, the traffic they process is rarely adve

safetyarxiv-cs-ai

Truth for Believable AI: Expressed Doubt, Provenance, and Belief Revision as an Engineerable Stance

arXiv:2609.26035v1 Announce Type: new Abstract: Conversational agents often express answers in a uniformly confident register. We test whether expressed uncertainty, provenance-aware assertion, and ex

You Should Be Properly Scoring Your Odometry

arXiv:2609.25900v1 Announce Type: new Abstract: When we evaluate the performance of our odometry, it is common practice to score the estimated track against a ground truth. Unfortunately, scoring uses

applicationsarxiv-cs-ro

A Hybrid Quantum Neural Network to Analyse Big Experimental Powder X-ray Diffraction Data