Search: “evaluation”

BranchDrive: A Branch-Structured Dataset for Action-Conditioned Driving Prediction

arXiv:2609.27275v1 Announce Type: new Abstract: Most autonomous-driving datasets record only the action executed by a behavior policy and the single future that followed, providing limited supervision

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

BrickCraft-Duo: Efficient Dual-Arm Skill Learning and Refinement for Compositional Long-Horizon Assembly

DGX agent

arXiv:2609.28281v1 Announce Type: new Abstract: Interlocking brick assembly provides a representative testbed for evaluating real-world robotic manipulation capabilities, where diverse structural desi

safetyarxiv-cs-ro

24 Sept 2026

Calibration and transfer in indicator-based assessments of artificial consciousness

arXiv:2603.27597v2 Announce Type: replace Abstract: Research on artificial consciousness increasingly shifts evaluation from behaviour to internal architecture. Theory-based indicators are used to upd

ChipMEM: Verification-Grounded Memory for EDA Agents

arXiv:2609.27067v1 Announce Type: cross Abstract: Large language model (LLM)-based agents use Electronic Design Automation (EDA) tools to generate and revise register-transfer-level (RTL) designs unde

Controlled Attribute-Specific Summarization of Interrogative Dialogues

arXiv:2609.28004v1 Announce Type: cross Abstract: Effective summarization of interrogative dialogues is a critical task in forensic and investigative settings, requiring high factual accuracy, coheren

Curriculum Learning with GNN-based Reinforcement Learning for Job Shop Scheduling

arXiv:2609.28085v1 Announce Type: cross Abstract: The job shop scheduling problem is a challenging combinatorial optimization problem, and recent reinforcement learning approaches using graph neural n

DCRL: Decoupling and Coupling Reinforcement Learning via Policy-Reward Manifold Alignment

arXiv:2609.27572v1 Announce Type: cross Abstract: Reinforcement learning (RL) has emerged as a key paradigm for improving the reasoning capabilities of large language models (LLMs). However, existing

Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Training

arXiv:2609.27197v1 Announce Type: new Abstract: Minimum Risk Training (MRT) enables neural machine translation models to directly optimize sequence-level evaluation metrics instead of relying only on

FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation

arXiv:2609.27657v1 Announce Type: cross Abstract: Solutions based on large language models (LLMs) often rely on temperature sampling to improve accuracy and stability by aggregating multiple samples f

Hardware

Folks, have you purchased the Mac M5 Ultra with 256GB yet? We need serious benchmarks, because we only get YouTube clowns influencers results

Ok, so the Mac M5 Ultra (256GB) hit the market, but the only publicly available benchmarks material are flashy YouTube 'clown influencers' videos. We need serious numbers to evaluate whether Apple’s s

hardwarer-localllama

Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents

arXiv:2609.27353v1 Announce Type: new Abstract: Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reprod

How Sensitive Are LLM Leaderboard Claims to Hidden Model Selection?

arXiv:2609.28177v1 Announce Type: cross Abstract: LLM leaderboard gains can reflect selection among privately evaluated model variants, yet neither the number of variants nor their dependence is publi

researcharxiv-cs-lg

Applications

LEGO: Synergizing Expert GraphRAG and Expert Chain-of-Thought for Legal Reasoning

arXiv:2609.27009v1 Announce Type: new Abstract: Large language models are increasingly applied to high-risk domains such as law, yet complex legal reasoning remains limited by two structural challenge

applicationsarxiv-cs-cl

Tutorials

LLM-Powered Socially Assistive Robot-Delivered Cognitive Behavioral Therapy Exercises: an Exploratory Study with University Students

arXiv:2402.17937v2 Announce Type: replace Abstract: Mental health is a significant healthcare challenge, and cognitive behavioral therapy (CBT) is a widely used therapeutic method for treating anxiety

tutorialsarxiv-cs-ro

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

arXiv:2608.24707v2 Announce Type: replace Abstract: While text-based hallucination detection is well studied, reference-free detection of factual alterations in speech remains underexplored, especiall

LWCal: Loss-Weighted Calibration for Tabular Classifiers with Noisy Calibration Labels

arXiv:2609.26839v1 Announce Type: cross Abstract: Post-hoc probability calibration is usually evaluated under an optimistic assumption: the held-out calibration labels are clean. In many AI deployment

M3D-Net: Hierarchical Coordination of Spatial Context, Feature Reuse, and Differential Attention for Mammography Classification

arXiv:2609.27523v1 Announce Type: new Abstract: Breast image classification requires local detail and global tissue context, yet these cues can weaken as representations deepen. We present M3D-Net, a

local-aiarxiv-cs-cv

Memory That Changes Action Is Not Memory That Guides It: Counterfactual Auditing of History-Conditioned Robot Policies

arXiv:2609.27247v1 Announce Type: new Abstract: A robot returning a block to its origin tray may encounter two task-consistent pasts that reconverge to the same current input but warrant different act

MSK-Bench: Benchmarking Full-Body Musculoskeletal Motor Control Across Tasks, Control Paradigms, and Physiological Metrics

arXiv:2609.26872v1 Announce Type: new Abstract: Musculoskeletal (MSK) humanoids provide a physiologically grounded embodiment for studying full-body motor control, but their high-dimensional muscle ac

nnFoundation: 3D Foundation Models for Radiology

arXiv:2609.26924v1 Announce Type: new Abstract: Radiological artificial intelligence has advanced rapidly, yet most systems remain narrowly task-specific, data-intensive, and fragile under domain shif

local-aiarxiv-cs-cv

PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety

arXiv:2609.28197v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents that alter real-world states, ensuring operational safety across multi-step workflows has

Quantum score matching with applications to learning thermal states

model-releasesarxiv-cs-lg

arXiv:2609.28391v1 Announce Type: cross Abstract: Score matching has driven major advances in classical generative learning by enabling models to learn from data without evaluating intractable normali

Reference-Based Analysis of Coherence and Diversity in Open-Ended Text Generation

arXiv:2609.28080v1 Announce Type: new Abstract: Evaluating open-ended text generation involves understanding how different properties of a continuation relate to its perceived quality. We present a re

safetyarxiv-cs-cl

Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets

arXiv:2609.27981v1 Announce Type: new Abstract: KV-cache eviction is typically evaluated through average quality-memory trade-offs, yet a small average loss can hide requests whose utility degrades ma

State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State

arXiv:2609.27606v1 Announce Type: new Abstract: We introduce State-Grounded Conditioning (SGC), a design principle for user-facing LLM agents that must condition on live user state (game state, sessio

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

model-releasesarxiv-cs-cv

arXiv:2512.11490v2 Announce Type: replace Abstract: Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both regio

What Changed? Drift Detection with Real, Virtual, and Incomparable Diagnosis

arXiv:2609.27865v1 Announce Type: cross Abstract: Sharing a deep encoder does not, by itself, fix the central confound of task-comparison scores. We show that cross-evaluated heads on a frozen shared

Agents

What Makes a Terminal-Bench Task Hard? Separating Genuine Hardness from Fake-Hardness on an Adjudicated Agentic Corpus

arXiv:2609.26826v1 Announce Type: cross Abstract: Frontier benchmarks need tasks that current models cannot solve. But a task that no model solves is not automatically a hard task. The same zero pass

agentsarxiv-cs-ai

When Visual Quality Misleads: Intent Recognition under Rendered Avatar Distortions

arXiv:2609.27560v1 Announce Type: cross Abstract: Avatar-streaming systems are commonly evaluated with image and video quality assessment (IQA/VQA) metrics, implicitly treating visual fidelity as a pr

safetyarxiv-cs-cv

Benchmarking Robots for Everyday Environments: From Lab Experiments to Real-World Operations

arXiv:2609.26490v1 Announce Type: new Abstract: This study introduces an interdisciplinary framework for benchmarking robots deployed in public environments, addressing the gap between traditional lab

Beyond End-Task Success: How to Audit Visual Experience Retrieval in Robotics

arXiv:2609.26567v1 Announce Type: new Abstract: Robots that store past experiences must select which one to reuse in a new scene. Most systems select by visual similarity, and most evaluations report

Building Trustworthy Mental Health Benchmarks on Bluesky: A Validation-Aware Weak-Supervision Framework

arXiv:2609.22696v1 Announce Type: new Abstract: Decentralized social media platforms create new opportunities and challenges for computational mental health research because data access, moderation, l

CableVLA: Simulation-Privileged Global-Local Representation Learning for Cable Routing

arXiv:2609.25606v1 Announce Type: new Abstract: Cable routing requires coordinated control of global cable topology and changing local contacts. We present CableVLA, an end-to-end multimodal vision-la

Can We Predict Anomaly Detection Performance from Embedding-Space Geometry?

arXiv:2609.26460v1 Announce Type: new Abstract: Anomaly detection systems are often trained using normal data alone, while model selection and evaluation typically require labeled anomalies. We study

local-aiarxiv-cs-lg

Applications

Concept Drift from a Causal Perspective

arXiv:2609.25340v1 Announce Type: new Abstract: Concept drift is a common phenomenon in real-world data streams, in which changes in the data-generating distribution can degrade predictive model perfo

applicationsarxiv-cs-lg

Designing an Efficient Excavator Bucket for Lunar ISRU: A Comparative Study with Vision-Based Fill and Displacement Analysis

arXiv:2609.25724v1 Announce Type: new Abstract: This paper present a spiral-cavity wheel for lunar regolith excavation and a sensor-light evaluation stack that jointly estimates fill ratio (vision), s

researcharxiv-cs-ro

Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs

arXiv:2609.22851v2 Announce Type: cross Abstract: Large Audio Language Models (LALMs) utilize either continuous features or discrete tokens, yet the optimal representation paradigm for general audio u

Agents

Divergent strategies and convergent outcomes in autonomous materials discovery

arXiv:2609.23957v1 Announce Type: new Abstract: Scientific agents are mostly evaluated on whether they complete tasks or recover known results; we instead study variation across repeated open-ended ca

agentsarxiv-cs-ai

DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving

arXiv:2609.26792v1 Announce Type: new Abstract: Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene featur

EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation

arXiv:2609.24115v1 Announce Type: new Abstract: Tool-calling LLM agents are increasingly deployed in enterprise applications. However, effective evaluation and optimization require high-quality, diver

Fairly Compensated Distributed Information Retrieval and Augmentation for AI Agents

arXiv:2609.22601v1 Announce Type: cross Abstract: The increasing reliance of autonomous AI agents on external and distributed knowledge sources introduces a fundamental challenge for decentralized inf

FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability

arXiv:2609.25192v1 Announce Type: new Abstract: Financial search is a highly demanding task for LLM agents, requiring not only a correct final answer but also temporally valid information retrieval, a

FMMD: A multimodal multidisciplinary dataset of open peer reviews from F1000Research

arXiv:2602.14285v2 Announce Type: replace-cross Abstract: Automated scholarly paper review (ASPR) has entered the coexistence phase with traditional peer review, where artificial intelligence (AI) sys

safetyarxiv-cs-cl

Higher-Order Approximation of Exit Functionals in Sampling-Based Stochastic Model Predictive Control

arXiv:2609.25257v1 Announce Type: cross Abstract: Safety evaluation in sampling-based stochastic model predictive control often requires numerical estimation of exit functionals. The approximation of

How SWE-Serve Exposes the Gap Between Local Tests and Live Serving

SWE‑Serve evaluates AI coding agents on 53 inference‑engineering tasks derived from SGLang pull requests across six functional domains. Among 19 tasks that launch a live server, patches that pass loca

local-ainvidia-developer

Tutorials

In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization

arXiv:2609.25836v1 Announce Type: new Abstract: Multi-task optimization (MTO) addresses a set of optimization tasks simultaneously, often suffering from inaccurate inter-task relationship estimation u

tutorialsarxiv-cs-lg

Interpretable AI plus Handheld, Portable Retinal Photographs: A Low-Cost Glaucoma Screening Solution for West Africa

arXiv:2609.25697v1 Announce Type: new Abstract: Purpose: To develop and evaluate an interpretable artificial intelligence (AI) framework for glaucoma screening from low-cost portable, handheld retinal

researcharxiv-cs-cv

MCPGen: Benchmarking LLMs on Executable MCPWorkflow Development

arXiv:2609.23925v1 Announce Type: cross Abstract: We study whether LLMs can produce executable workflow artifacts that remain consistent across graph structure, tool implementation, schema bindings, a