Search: “evaluation”

Strategic Governance of AI Models in Earth Science

arXiv:2610.10560v1 Announce Type: cross Abstract: AI foundation models pretrained on weather and climate data are increasingly fine-tuned to Earth science tasks well beyond weather forecasting. Their

9 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

TacHair: Tactile Contact-Distribution Guided Online Correction for Robotic Hair Stroking and Perception

DGX agent

arXiv:2610.10637v1 Announce Type: new Abstract: Hair stroking is common in daily grooming and personal care, and is also widely used in hair-product evaluation, motivating robots with similar physical

safetyarxiv-cs-ro

9 Oct 2026

TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models

arXiv:2610.11392v1 Announce Type: new Abstract: System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generati

9 Oct 2026

VESSI - VLM-Enhanced Support for Surveillance and Investigations

arXiv:2610.11674v1 Announce Type: new Abstract: Automated video surveillance analysis has become a critical component of intelligence infrastructures and Law Enforcement agencies. Traditional systems

researcharxiv-cs-cv

9 Oct 2026

BEACON-SP: Ontology-Grounded GraphRAG Framework for Clinical Suicide Risk Assessment

arXiv:2610.09026v1 Announce Type: cross Abstract: We present BEACON-SP, an ontology-grounded Graph Retrieval-Augmented Generation (GraphRAG) framework for clinician-facing decision support in behavior

CARE: Certifying Acceleration for Vision-Language-Action Inference

arXiv:2610.08917v1 Announce Type: new Abstract: While vision-language-action (VLA) models have advanced rapidly, running them at every control step remains expensive. Prior work accelerates VLA infere

Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy

arXiv:2610.09906v1 Announce Type: cross Abstract: Security Operations Centers (SOCs) for information technology and operational technology share one incident-response problem: a flood of correlated al

safetyarxiv-cs-cl

Few-Shot Learning for Personalised Automated Pain Assessment

model-releasesarxiv-cs-lg

arXiv:2610.09692v1 Announce Type: new Abstract: Pain perception varies substantially across individuals, making it difficult for population-based classifiers to generalise across all subjects in a dat

FinVector-Market-4B: A Controlled Study of LoRA Adaptation for Structured Financial Tasks

arXiv:2610.08882v1 Announce Type: cross Abstract: FinVector-Market-4B adapts Qwen/Qwen3.5-4B with rank-16 LoRA on a 22,000-example corpus for structured financial tasks. We evaluate the base and adapt

InstanceBench: Diagnosing Referential Reasoning and Target Identity in Referring Expression Segmentation

model-releasesarxiv-cs-cv

arXiv:2610.09478v1 Announce Type: new Abstract: Referring Expression Segmentation (RES) links natural-language descriptions to pixel-level object masks. Yet standard evaluation provides limited insigh

Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurements

arXiv:2610.10366v1 Announce Type: new Abstract: Feature caching accelerates diffusion sampling by replacing expensive network evaluations with predictions from previously computed activations. However

researcharxiv-cs-lg

Lower Bounds for Parallel Diffusion Sampling

arXiv:2610.09166v1 Announce Type: cross Abstract: Standard diffusion samplers generate samples through repeated evaluations of a learned score function. Parallel sampling methods seek to accelerate ge

researcharxiv-cs-lg

OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified Sim-and-Real Framework

model-releasesarxiv-cs-ro

arXiv:2610.10384v1 Announce Type: new Abstract: Tactile feedback provides embodied agents with physical information beyond visual observations, enabling more reliable interaction with the real world.

The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules

model-releasesarxiv-cs-lg

arXiv:2610.09239v1 Announce Type: cross Abstract: Self-improving LLM systems propose changes to themselves and keep those that score better on a small evaluation set. We treat this keep-if-better step

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the …

model-releasesclem-delangue--x

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the platform trains Qwen-3.8-27B on them, we evaluate the models

Applications

Towards Financial World Modeling

arXiv:2610.09048v1 Announce Type: new Abstract: Building a world model requires a state representation useful for planning and decision-making---potentially over tasks unknown at training time. In the

applicationsarxiv-cs-lg

Adaptive Gait Biofeedback With Participant-Held-Out Modeling and Participant-Specific Updating in Chronic Ankle Instability

arXiv:2610.07428v1 Announce Type: new Abstract: Adaptive gait biofeedback may support repeated practice in chronic ankle instability, but its evaluation must address model performance and human respon

Benchmarking Label-Revealed Online Updates for EEG BCI Decoding

model-releasesarxiv-cs-lg

arXiv:2610.07420v1 Announce Type: new Abstract: Electroencephalography (EEG) signals drift over time, which can cause static brain-computer interface (BCI) models to degrade in practice. We present a

Calibration Is Not Control: Intervention Value for LLM-Agent Oversight

arXiv:2606.21399v2 Announce Type: replace Abstract: Runtime oversight often intervenes when an LLM agent's calibrated failure score crosses a threshold. Yet states with the same failure risk can diffe

safetyarxiv-cs-ai

Agents

CuratorMAS: Automating Dataset Curation via Multi-Agent Orchestration

arXiv:2610.07075v1 Announce Type: new Abstract: High-quality datasets are essential for reliable machine learning, but dataset curation remains costly and hard to generalize across domains. Existing m

agentsarxiv-cs-ai

EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation

arXiv:2610.07969v1 Announce Type: new Abstract: Scaling robotic foundation models requires diverse training data and reliable evaluation environments. Simulation offers a scalable solution, yet existi

safetyarxiv-cs-cv

Tutorials

Evidence Before Sampling: Interpretable Implicit Negative Candidate Discovery for Recommendation

arXiv:2610.07708v1 Announce Type: new Abstract: Recommender systems learn from observed user-item interactions, but explicit negative feedback is often unavailable. Since deep learning models require

tutorialsarxiv-cs-ai

Evidence-Bound Reasoning: Neuro-Semantic Verification of Biomedical AI in Glioblastoma Radiogenomics

arXiv:2610.08660v1 Announce Type: new Abstract: Background: Biomedical AI can generate plausible explanations without reliably verifying whether each statement is supported by patient-specific evidenc

Tutorials

Generalizable Robustness Testing of DNN-Based Robotic Navigation Systems via XAI-Guided Search

arXiv:2610.06862v1 Announce Type: new Abstract: **Context:** Deep Neural Networks (DNNs) increasingly control Cyber-Physical Systems (CPSs), yet small input perturbations can cause unsafe system-level

tutorialsarxiv-cs-ro

Generative AI translations in high-stakes emergency messaging

arXiv:2610.08601v1 Announce Type: new Abstract: Emergency messaging such as extreme-weather reports and earthquake instructions can involve high stakes, to the extent that translation errors can lead

researcharxiv-cs-cl

HygieneRoboBench: Benchmarking Hygiene-Aware Planning for Household Robots

arXiv:2610.08642v1 Announce Type: cross Abstract: Contact with contaminated objects can spread hazards through a household robot's grippers, tools, and shared surfaces, while new contacts can make an

Applications

HyperNSDE: Personalized Neural SDEs for Joint Static-Longitudinal Clinical Data Generation

arXiv:2610.07383v1 Announce Type: cross Abstract: Synthetic patient data generation is a promising solution to the dual challenge of data scarcity and privacy constraints in healthcare machine learnin

applicationsarxiv-cs-lg

Illusory Pattern Perception Drives Spurious Inference in Large Language Models

arXiv:2610.07791v1 Announce Type: new Abstract: Illusory pattern perception is a well-documented human cognitive tendency to infer meaningful relationships in data that is actually random. Such a tend

LOGIC: An LLM Benchmark for Intent-Grounded Change Impact in Aerospace Electrical Systems

arXiv:2610.07580v1 Announce Type: new Abstract: Aerospace electrical-design revisions can contain multiple genuine changes, although an engineering request may authorize only a subset. Propagating eve

OMIT the Action: Measuring Framing-Invariant Omission Bias under Philosophical Disagreement

arXiv:2610.07847v1 Announce Type: new Abstract: As LLMs increasingly assist in moral reasoning, omission bias, the tendency to prefer inaction even when equivalent framings reverse substantive outcome

Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices

arXiv:2610.08153v1 Announce Type: cross Abstract: Multiple-choice question answering (MCQA) is commonly used to evaluate large language models under the assumption that one of the provided options is

Personal-Agent Mediated Recommendation with Cross-Platform User History

arXiv:2610.07588v1 Announce Type: new Abstract: Modern recommendation is shifting from platform-centric personalization toward user-governed personalization, where a personal LLM agent can act on the

PuzzleJAX: A Benchmark for Reasoning and Learning

arXiv:2508.16821v2 Announce Type: replace Abstract: We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinfor

Quality-Aware Self-Correcting Speech Translation on an Edge Device

arXiv:2610.07545v1 Announce Type: new Abstract: We present a fully offline speech-to-speech translation pipeline that runs on a Jetson Nano (4 GB) and corrects its own weak translations without retrai

researcharxiv-cs-cl

Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective

arXiv:2610.07894v1 Announce Type: new Abstract: Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is insuf

Sample-Optimal Estimation of the Frechet Inception Distance

arXiv:2610.07114v1 Announce Type: cross Abstract: The Frechet Inception Distance (FID) is widely used to evaluate generative models, but its empirical plug-in estimator suffers from finite-sample bias

safetyarxiv-cs-cv

Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning

arXiv:2610.07385v1 Announce Type: new Abstract: Fine-tuning vision-language models (VLMs) is typically evaluated at a single downstream checkpoint, obscuring whether a semantic capability was never ac

researcharxiv-cs-cv

SkillPoison: Progressive Skill Poisoning via Successful Experiences

arXiv:2610.07645v1 Announce Type: cross Abstract: Self-improving LLM agents increasingly distill successful experiences into persistent, reusable skills. Existing skill attack methods corrupt this lea

SpeedrunBench: Challenging LLM Agents with Video Game Speedrunning

arXiv:2610.08076v1 Announce Type: new Abstract: Frontier LLM agents have been shown to be capable of solving increasingly complex tasks for which humans have measurable solutions. This begs the pertin

Symphony for Text Generation: Benchmarking Clinical Note Generation

arXiv:2610.08161v1 Announce Type: cross Abstract: Ambient documentation systems are rapidly gaining adoption, yet their impact on clinical note quality remains poorly characterized. We introduce MedCo

Systemization of Knowledge (SoK): Human-Centered AI Safety for Youth

arXiv:2610.08554v1 Announce Type: cross Abstract: While HCI increasingly examines AI-safety for youth, the literature lacks a comprehensive view of what risks have been identified, how they are addres

safetyarxiv-cs-ai

The Robot Is Not Its Description: GaugeBench for Representation Robustness in Morphology-Aware Policies

arXiv:2610.07597v1 Announce Type: cross Abstract: A robot description does more than specify a physical mechanism: it also encodes arbitrary conventions, such as joint-axis direction, joint-angle zero

safetyarxiv-cs-lg

Towards a Unified Misuse Monitoring Benchmark

arXiv:2610.07089v1 Announce Type: cross Abstract: LLM agents increasingly act in multi-actor environments, exposing them to misuse from multiple sources: decomposition attacks, where a harmful request

Trustworthy Method Comparison with AI Judges: Estimation and Design under Order, Batch, and Aggregation Effects

arXiv:2610.07755v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges for automated AI evaluation. A common practice is to randomize prompt sequences and avera

researcharxiv-cs-lg

When Plans Change Answers: Formalizing Cost-Accuracy Optimization for Semantic Queries

arXiv:2610.08089v1 Announce Type: cross Abstract: In semantic query engines, predicates are evaluated by machine-learned models, and the choice of a query plan affects not only the cost of a query but

safetyarxiv-cs-ai

Agents

When to Remember, When to Abstain: Category-Conditioned Retention for Reliable Agent Memory

arXiv:2610.07100v1 Announce Type: new Abstract: Persistent agent memory is only as reliable as its retention decision: an assertion weakly supported by its source can be stored and later reused as est

agentsarxiv-cs-ai

Agentic Resource Allocation for Batch Multi-Objective Bayesian Optimization in Autonomous Materials Discovery

arXiv:2610.04134v1 Announce Type: cross Abstract: The discovery and development of advanced materials is a challenging process constrained by the high time and monetary costs of synthesis, processing,

safetyarxiv-cs-lg

6 Oct 2026

AgroGround: Multi-Granularity Grounded Recognition in Agriculture