Search: “evaluation”

From Articles to Publishers: Aggregating Language Model Predictions for News Source Reliability Inference

arXiv:2609.24219v1 Announce Type: new Abstract: Traditionally, the reliability of news publishers is assessed by expert organisations that evaluate editorial practices, transparency and factual standa

researcharxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

GameReplica: A Benchmark for Black-Box Visual Game Replication by Vision-Language Agents

DGX agent

arXiv:2609.22308v1 Announce Type: new Abstract: Coding-agent benchmarks usually evaluate implementation after the target behavior has been specified in text, code, or demonstrations. Existing research

model-releasesarxiv-cs-cv

22 Sept 2026

Imagine-RL: Residual-Confidence-Guided Cross-Attention for World-Model-Augmented VLA Reinforcement Learning

arXiv:2609.24033v1 Announce Type: new Abstract: Reliable action evaluation in contact-rich manipulation requires looking beyond the current observation to future visual and contact consequences. Exist

safetyarxiv-cs-ro

LIBERO-VPro: Benchmarking Closed-Loop Visual Robustness of Robotic Foundation Models

arXiv:2609.24350v1 Announce Type: cross Abstract: Robotic foundation models achieve impressive performance on standard manipulation benchmarks, yet these evaluations typically assume clean, timely, an

LLMs Anchor on Chief Complaint and Fail to Integrate Evidence in Sequential Clinical Triage

arXiv:2609.22904v1 Announce Type: new Abstract: Triage in the emergency department (ED) is a sequential decision process that unfolds turn by turn. Existing evaluations of large language models (LLMs)

researcharxiv-cs-cl

RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling

arXiv:2609.22947v1 Announce Type: new Abstract: Reinforcement learning (RL) is vital for optimizing video generation models, with a robust reward model (RM) serving as the cornerstone. However, existi

XYEval: Agents say yes to bad advice

arXiv:2609.23939v1 Announce Type: new Abstract: Effective communication between users and AI agents is essential for human-AI collaboration. The XY problem is a well-known communication pitfall where

ZoAQ: Adaptive Zeroth-Order Querying via Query-Reuse Coupling

arXiv:2609.22115v1 Announce Type: new Abstract: Zeroth-order optimization (ZOO) estimates updates from function evaluations, making perturbation queries a primary cost. Fixed budgets spend the same nu

researcharxiv-cs-lg

Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognition

arXiv:2609.21879v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have recently been proposed as promising tools for face recognition, as they can produce natural language explanations a

Beyond Benchmark Scores: Auditing Medical Vision-Language Models for Chest X-Ray Tuberculosis Screening

arXiv:2609.21763v1 Announce Type: new Abstract: A medical model's benchmark score does not establish that the same conclusion holds under a different evaluation. This study tests whether claims about

BrainWideBench: Benchmarking large-scale pretraining and across-animal transfer in multi-region neural recordings

model-releasesarxiv-cs-lg

arXiv:2609.22064v1 Announce Type: new Abstract: Advances in large-scale neural recording have made it possible to collect data across many animals and distributed brain regions, raising the question o

CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents

arXiv:2609.21686v1 Announce Type: cross Abstract: Privacy leakage in LLM agents is commonly evaluated within individual components such as memory, retrieval, or tool-use pipelines, which makes it diff

safetyarxiv-cs-ai

From Memory to Behavior: A Behavior-Aware Role-Playing Framework for Social Media Influencers

arXiv:2609.21349v1 Announce Type: cross Abstract: Large language models have shown strong potential as role-playing agents for real individuals, yet faithful impersonating remains challenging. Existin

researcharxiv-cs-ai

A Policy Profile for Croissant: Refusal as a Property of the Dataset

arXiv:2609.19640v1 Announce Type: new Abstract: Croissant is the de facto machine-readable descriptor for ML datasets: JSON-LD over schema.org. Since version 1.1 it also carries data use conditions, r

safetyarxiv-cs-lg

F^{2}DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows

arXiv:2609.19827v1 Announce Type: new Abstract: With the widespread industrial deployment of Large Language Models (LLMs), DeepSearch has emerged as the dominant paradigm for resolving complex user qu

Seeing Abnormal from Normal: Glomerular Abnormality in Representations of Normal Renal Morphology

arXiv:2609.19444v1 Announce Type: new Abstract: Fine-grained evaluation of glomerular pathology must distinguish normal glomeruli from abnormalities such as global and segmental glomerulosclerosis, ob

safetyarxiv-cs-cv

When Does Retrieval Help Time-Series Forecasting?

arXiv:2609.20193v1 Announce Type: new Abstract: Retrieval plug-ins supply a deep forecaster with information its lookback window cannot carry. Published evaluations report consistent gains, and each c

researcharxiv-cs-lg

'Your Robot Was Trained on a Lie': Collision Mesh Poisoning Attacks on Robotic Manipulation

arXiv:2609.18122v1 Announce Type: cross Abstract: Learning-enabled robotic manipulation increasingly relies on robot simulators for policy training and evaluation before real-world deployment. Inside

safetyarxiv-cs-ro

Compositional Policy Violations: When Step-Level Compliance Fails In Agentic AI Workflows

arXiv:2609.18820v1 Announce Type: new Abstract: Agentic workflows now make consequential decisions in regulated settings, and the governance placed around them is almost entirely step-scoped: input-ou

safetyarxiv-cs-ai

Do Frontier Models Seek Safety Evidence Before Acting?

arXiv:2609.17865v1 Announce Type: new Abstract: Frontier models are often evaluated on how they respond to safety information once it is already in context. We study an earlier decision point: whether

FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback

model-releasesarxiv-cs-ro

arXiv:2609.18651v1 Announce Type: new Abstract: Generalist robot policies offer useful initialization, but refining compact specialists through limited physical interaction requires informative learni

MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents

arXiv:2609.19059v1 Announce Type: cross Abstract: Multimodal large language model (MLLM) agents are increasingly used as personal assistants for long-running tasks. Their utility depends on continuity

Modular Deep Learning Mechanisms for Auditable Next-Day Wildfire Spread Prediction

model-releasesarxiv-cs-lg

arXiv:2609.17763v1 Announce Type: new Abstract: Next-day wildfire prediction requires models whose forecasts can be evaluated alongside the assumptions and historical evidence used in their computatio

MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education

arXiv:2609.19088v1 Announce Type: new Abstract: Large vision-language models have achieved remarkable progress in multi-modal understanding, yet their capabilities in educational settings remain insuf

PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics

arXiv:2505.23126v5 Announce Type: replace Abstract: Although many benchmarks evaluate the reasoning abilities of Large Language Models (LLMs) within domains such as mathematics, coding, or data wrangl

REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff

arXiv:2609.17745v1 Announce Type: cross Abstract: A central goal of autonomous reinforcement learning is continuous policy training without external resets. However, existing paradigms largely depend

Tutorials

TACTICS: Taxonomy-Aware Intelligent Corpus Sampling for Machine Translation

arXiv:2609.17956v1 Announce Type: new Abstract: Large-scale machine-translation (MT) systems are typically evaluated on random samples from a corpus whose distributional composition is an artifact of

tutorialsarxiv-cs-cl

What Counts as Strategic Reasoning? A Systematic Mapping of Chess Research on Humans, Engines, and Language Models

arXiv:2609.18286v1 Announce Type: new Abstract: Chess has long served as a model domain for studying search, expertise, decision-making, and artificial intelligence. The emergence of large language mo

A Framework for Generating Valid Context-Specific Benchmarks through Expert Guidance

arXiv:2609.16592v1 Announce Type: new Abstract: This paper presents an end-to-end approach for generating context-specific large language model (LLM) benchmark datasets by combining expert input with

CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design

arXiv:2609.16251v1 Announce Type: new Abstract: Computer-use agents are increasingly evaluated in realistic desktop environments, but existing benchmarks provide limited coverage of professional engin

Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering

arXiv:2609.17043v1 Announce Type: cross Abstract: Multi-hop question answering requires combining information from multiple documents to answer complex questions. These systems have grown increasingly

researcharxiv-cs-ai

DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflow

arXiv:2509.12626v4 Announce Type: replace-cross Abstract: Aligning agentic AI with user intent is critical for delegating complex, socially embedded tasks, yet user preferences are often implicit, evo

safetyarxiv-cs-ai

EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models

arXiv:2609.17081v1 Announce Type: new Abstract: Grounded language-model systems are often evaluated by final answer accuracy, yet a correct answer can be unsupported, drawn from the wrong source, or p

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

arXiv:2607.18063v2 Announce Type: replace-cross Abstract: LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. We present a 21-scenario benchmark f

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

arXiv:2609.15478v1 Announce Type: new Abstract: Multimodal agents can create complex videos in software such as Blender by coding without relying on diffusion models. Yet video understanding benchmark

CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models

arXiv:2609.14956v1 Announce Type: cross Abstract: Speech Quality Assessment (SQA) is essential for modern speech technologies, and recent non-intrusive SQA predictors increasingly rely on Speech Found

Failure-Guided Co-Evolution of Prompts and Training Data

arXiv:2609.15209v1 Announce Type: cross Abstract: Automatic prompt optimization (APO) improves language-model programs by revising prompts from task feedback, yet it typically holds its training data

researcharxiv-cs-ai

FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learning

arXiv:2609.15625v1 Announce Type: cross Abstract: Driven by the escalating demand for privacy-preserving computing, Federated Learning (FL) has witnessed remarkable progress, becoming a cornerstone te

GroundBench: A Factorized, Counterfactual Benchmark for Locating VLM Affordance Failures

arXiv:2609.13308v1 Announce Type: new Abstract: A companion evaluation found that naming the target part in a manipulation prompt increased action accuracy by 0.32-0.63 across eight vision-language mo

Grounded Adjudication of Variations across Extracted TimeLines (GAVEL): Comparing Clinical Timelines Against Their Case Reports

arXiv:2609.13475v1 Announce Type: new Abstract: Existing pipelines for clinical timeline extraction from case reports are evaluated using an expert reference and are limited by imperfect reference ann

Merging the Knowledge of LLMs for Automatic Speech Recognition

arXiv:2609.15743v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems, trained on paired speech-text data, have been improved by leveraging language models (LMs) trained on text-o

researcharxiv-cs-cl

MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding

arXiv:2609.14992v1 Announce Type: new Abstract: Recently, the rapid development of large language models (LLMs) has reshaped software engineering by enabling autonomous code agents that plan, execute,

NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities

arXiv:2609.15695v1 Announce Type: new Abstract: Vision-language models (VLMs) increasingly power consumer-facing AI search, yet evaluating them on the diversity of everyday visual questions remains ch

PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents

arXiv:2606.17467v3 Announce Type: replace-cross Abstract: Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and inte

Post-Reasoning: Improving the Performance of Non-Thinking Models at No Cost

arXiv:2605.06165v2 Announce Type: replace Abstract: As the widespread adoption of Large Language Models (LLMs) accelerates, token consumption from intermediate reasoning traces increasingly contribute

Sanity Checking Causal Representation Learning on a Simple Real-World System

arXiv:2502.20099v3 Announce Type: replace-cross Abstract: We evaluate methods for causal representation learning (CRL) on a simple, real-world system that satisfies the basic problem setup of CRL. The

SkillLift: Learning Dense Rubrics from Sparse Oracles for Efficient Skill Evolution

arXiv:2609.15396v1 Announce Type: new Abstract: LLM-based agents increasingly rely on persistent skills, i.e., reusable procedural prompts, to adapt without weight updates. Existing skill self-evoluti

TotalSynth: Robust Whole-Body Synthetic CT from MRI and CBCT

arXiv:2609.13838v1 Announce Type: new Abstract: Purpose: To develop and evaluate TotalSynth, a reusable pretrained framework for whole-body synthetic CT (sCT) generation from MRI and cone-beam CT (CBC