Search: “evaluation”

arXiv:2603.23806v2 Announce Type: replace-cross Abstract: AI agents are increasingly embedded in real software systems, where they execute multi-step workflows through multi-turn dialogue, tool invoca

agentsarxiv-cs-ai

12 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

A Resilience Framework for Bi-Criteria Combinatorial Optimization with Bandit Feedback

DGX agent

arXiv:2503.12285v2 Announce Type: replace-cross Abstract: We study bi-criteria combinatorial optimization under noisy function evaluations. While resilience and black-box offline-to-online reductions

researcharxiv-cs-ai

11 May 2026

Adaptive auditing of AI systems with anytime-valid guarantees

arXiv:2605.07002v1 Announce Type: new Abstract: A major bottleneck in characterizing the failure modes of generative AI systems is the cost and time of annotation and evaluation. Consequently, adaptiv

researcharxiv-cs-ai

Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge

arXiv:2605.06912v1 Announce Type: new Abstract: The proliferation of generative video technologies has intensified the need for reliable methods to detect and characterize synthetic media. To address

researcharxiv-cs-cv

Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

arXiv:2605.06869v1 Announce Type: new Abstract: AI agent research spans a wide spectrum: from RL agents that learn from scratch to foundation model agents that leverage pre-trained knowledge, yet no u

Amortized Multi-Objective Optimization Across Tasks with Generative Solution Modeling

arXiv:2511.09598v5 Announce Type: replace Abstract: Many real-world applications require solving families of expensive multi-objective optimization problems~(EMOPs) under varying operational condition

Beyond Retrieval: A Multitask Benchmark and Model for Code Search

arXiv:2605.04615v2 Announce Type: replace-cross Abstract: Code search has usually been evaluated as first-stage retrieval, even though production systems rely on broader pipelines with reranking and d

Delulu: A Verified Multi-Lingual Benchmark for Code Hallucination Detection in Fill-in-the-Middle Tasks

arXiv:2605.07024v1 Announce Type: new Abstract: Large Language Models for code generation frequently produce hallucinations in Fill-in-the-Middle (FIM) tasks -- plausible but incorrect completions suc

Do Joint Audio-Video Generation Models Understand Physics?

arXiv:2605.07061v1 Announce Type: cross Abstract: Joint audio-video generation models are rapidly approaching professional production quality, raising a central question: do they understand audio-visu

Hardware

Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation

arXiv:2605.07985v1 Announce Type: cross Abstract: Selecting the optimal LLM inference configuration requires evaluation across hardware, serving engines, attention backends, and model architectures, s

hardwarearxiv-cs-ai

Local Ai

GESR: Graph-Based Edge Semantic Reconstruction for Stealthy Communication Detection with Benign-Only Training

arXiv:2605.07536v1 Announce Type: cross Abstract: Detecting stealthy malicious communications from flow logs under benign-only training remains a critical challenge in network security. Malicious comm

local-aiarxiv-cs-lg

Local Ai

HMACE: Heterogeneous Multi-Agent Collaborative Evolution for Combinatorial Optimization

arXiv:2605.07214v1 Announce Type: new Abstract: Large Language Models have recently emerged as a promising paradigm for automated heuristic design for NP-hard combinatorial optimization problems. Desp

local-aiarxiv-cs-ai

IntentGrasp: A Comprehensive Benchmark for Intent Understanding

arXiv:2605.06832v1 Announce Type: cross Abstract: Accurately understanding the intent behind speech, conversation, and writing is crucial to the development of helpful Large Language Model (LLM) assis

LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation

arXiv:2605.07640v1 Announce Type: cross Abstract: Remote sensing lithology interpretation is fundamental to geological surveys, mineral exploration, and regional geological mapping. Unlike general lan

MAS-Algorithm: A Workflow for Solving Algorithmic Programming Problems with a Multi-Agent System

arXiv:2605.05949v2 Announce Type: replace Abstract: Algorithmic problem solving serves as a rigorous testbed for evaluating structured reasoning in AI coding systems, as it directly reflects a model's

MathlibPR: Pull Request Merge-Readiness Benchmark for Formal Mathematical Libraries

arXiv:2605.07147v1 Announce Type: cross Abstract: The ecosystem of Lean and Mathlib has become the de facto standard for large language model (LLM) assisted formal reasoning with remarkable successes

Agents

mathsf{VISTA}: Decentralized Machine Learning in Adversary Dominated Environments

arXiv:2605.07841v1 Announce Type: cross Abstract: Decentralized machine learning often relies on outsourcing computations, such as gradient evaluations, to untrusted worker nodes. Existing robust aggr

agentsarxiv-cs-ai

MedAction: Towards Active Multi-turn Clinical Diagnostic LLMs

arXiv:2605.07305v1 Announce Type: cross Abstract: Most existing LLM diagnoses are evaluated on static, single-turn settings where complete patient information is provided upfront, an oversimplificatio

Applications

Multimodal synthesis of MRI and tabular data with diffusion in a joint latent space via cross-attention

arXiv:2605.06699v1 Announce Type: cross Abstract: We propose a multimodal latent diffusion model that jointly synthesizes volumetric magnetic resonance imaging (MRI) and tabular clinical data within a

applicationsarxiv-cs-ai

Overview of the TREC 2025 RAGTIME Track

arXiv:2602.10024v2 Announce Type: replace-cross Abstract: The principal goal of the RAG TREC Instrument for Multilingual Evaluation (RAGTIME) track at TREC is to study report generation from multiling

researcharxiv-cs-cl

Physics-Informed Reduced-Order Operator Learning for Hyperelasticity in Continuum Micromechanics

arXiv:2605.07738v1 Announce Type: cross Abstract: Physics-informed operator learning is an attractive candidate for surrogate modeling of microstructures, especially in multiscale finite-element simul

researcharxiv-cs-lg

ProcObject-10K: Benchmarking Object-Centric Procedural Understanding in Instructional Videos

arXiv:2512.03479v2 Announce Type: replace Abstract: Procedural activities are fundamentally driven by object state transitions, yet existing instructional video benchmarks remain action-centric and ca

R-GTD: A Geometric Analysis of Gradient Temporal-Difference Learning in Singular Regimes

arXiv:2601.20599v2 Announce Type: replace-cross Abstract: Gradient temporal-difference (GTD) learning algorithms are widely used for off-policy policy evaluation with function approximation. However,

safetyarxiv-cs-ai

Self-Programmed Execution for Language-Model Agents

arXiv:2605.06898v1 Announce Type: new Abstract: At the heart of existing language model agents is a fixed orchestrator program responsible for the state transition between consecutive turns. This pape

safetyarxiv-cs-ai

ShellfishNet: A Domain-Specific Benchmark for Visual Recognition of Marine Molluscs

arXiv:2605.07338v1 Announce Type: new Abstract: The decline of global shellfish biodiversity poses a severe threat to coastal ecosystems. Although artificial intelligence (AI) technologies show potent

TAVIS: A Benchmark for Egocentric Active Vision and Anticipatory Gaze in Imitation Learning

arXiv:2605.07943v1 Announce Type: cross Abstract: Active vision -- where a policy controls its own gaze during manipulation -- has emerged as a key capability for imitation learning, with multiple ind

The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval

arXiv:2605.07186v1 Announce Type: cross Abstract: Existing Large Language Model (LLM) benchmarks primarily focus on syntactically correct inputs, leaving a significant gap in evaluation on imperfect t

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation

arXiv:2605.07924v1 Announce Type: cross Abstract: Discrete flow matching generates text by iteratively transforming noise tokens into coherent language, but may require hundreds of forward passes. Dis

TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models

arXiv:2601.18744v2 Announce Type: replace Abstract: Time series are ubiquitous in real-world scenarios and crucial for applications ranging from energy management to traffic control. Consequently, the

Zero-Shot Imagined Speech Decoding via Imagined-to-Listened MEG Mapping

arXiv:2605.08075v1 Announce Type: new Abstract: Decoding imagined speech from non-invasive brain recordings is challenging because imagined datasets are scarce and difficult to align temporally across

safetyarxiv-cs-lg

Adaptive Policy Selection and Fine-Tuning under Interaction Budgets for Offline-to-Online Reinforcement Learning

arXiv:2605.05123v1 Announce Type: new Abstract: In offline-to-online reinforcement learning (O2O-RL), policies are first safely trained offline using previously collected datasets and then further fin

safetyarxiv-cs-lg

Cognitive Twins: Investigating Personalized Thinking Model Building and Its Performance Enhancement with Human-in-the-Loop

arXiv:2605.04761v1 Announce Type: new Abstract: This paper presents the Personalized Thinking Model (PTM), a hierarchical and interpretable learner representation designed for AI supported education.

Agents

Collision-Aware Object-Goal Visual Navigation via Two-Stage Deep Reinforcement Learning

arXiv:2502.13498v2 Announce Type: replace-cross Abstract: Object-goal visual navigation aims to reach a specific target object using egocentric visual observations. Recent deep reinforcement learning

agentsarxiv-cs-cv

Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation

arXiv:2512.14954v2 Announce Type: replace Abstract: Computing next-token likelihood ratios between two language models (LMs) is a standard task in training paradigms such as knowledge distillation. Si

researcharxiv-cs-cl

Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction

arXiv:2605.04770v1 Announce Type: new Abstract: While zero-shot appearance-based 3D gaze estimation offers significant cost-efficiency by directly mapping RGB images to gaze vectors, its reliability i

ICPR 2026 Competition on Privacy-Preserving Person Re-Identification from Top-View RGB-Depth Camera (TVRID)

arXiv:2605.04977v1 Announce Type: new Abstract: This companion paper reports the ICPR 2026 TVRID competition on privacy-aware top-view person re-identification. We present the competition setting, the

Misaligned by Reward: Socially Undesirable Preferences in LLMs

arXiv:2605.05003v1 Announce Type: new Abstract: Reward models are a key component of large language model alignment, serving as proxies for human preferences during training. However, existing evaluat

safetyarxiv-cs-cl

MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents

arXiv:2605.03952v1 Announce Type: cross Abstract: Coding agents often pass per-prompt safety review yet ship exploitable code when their tasks are decomposed into routine engineering tickets. The chal

Probing Structural Mathematical Reasoning in Language Models with Algebraic Trapdoors

arXiv:2605.04352v1 Announce Type: new Abstract: We introduce a benchmark suite for evaluating structural mathematical reasoning in language models, built on subgroup-construction problems in SL(3, Z)

Programmatic Context Augmentation for LLM-based Symbolic Regression

arXiv:2605.03101v1 Announce Type: new Abstract: Symbolic regression (SR), the task of discovering mathematical expressions that best describe a given dataset, remains a fundamental challenge in scient

researcharxiv-cs-ai

Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extraction

model-releasesarxiv-cs-cl

arXiv:2605.04221v1 Announce Type: new Abstract: Clinical named entity recognition from dental progress notes is challenging because documentation is highly unstructured, domain-specific, and often pri

Tutorials

Stream-T1: Test-Time Scaling for Streaming Video Generation

arXiv:2605.04461v1 Announce Type: new Abstract: While Test-Time Scaling (TTS) offers a promising direction to enhance video generation without the surging costs of training, current test-time video ge

tutorialsarxiv-cs-cv

A Benchmark for Interactive World Models with a Unified Action Generation Framework

arXiv:2605.03941v1 Announce Type: new Abstract: Achieving Artificial General Intelligence (AGI) requires agents that learn and interact adaptively, with interactive world models providing scalable env

A Knowledge-Driven LLM-Based Decision-Support System for Explainable Defect Analysis and Mitigation Guidance in Laser Powder Bed Fusion

arXiv:2605.01100v1 Announce Type: new Abstract: This work presents a knowledge-driven decision-support system that integrates structured defect knowledge with LLM-based reasoning to provide explainabl

safetyarxiv-cs-ai

A Paradigm for Interpreting Metrics and Identifying Critical Errors in Automatic Speech Recognition

arXiv:2605.03671v1 Announce Type: new Abstract: The most commonly used metrics for evaluating automatic speech transcriptions, namely Word Error Rate (WER) and Character Error Rate (CER), have been he

researcharxiv-cs-cl

AcademiClaw: When Students Set Challenges for AI Agents

arXiv:2605.02661v1 Announce Type: new Abstract: Benchmarks within the OpenClaw ecosystem have thus far evaluated exclusively assistant-level tasks, leaving the academic-level capabilities of OpenClaw

Boundary-Aware Uncertainty Quantification for Wildfire Spread Prediction

arXiv:2605.03148v1 Announce Type: new Abstract: Reliable wildfire spread prediction is vital for risk-aware emergency planning, yet most deep learning models lack principled uncertainty quantification

researcharxiv-cs-cv

Enhancing AI-Based ECG Delineation with Deep Learning Denoising Techniques

arXiv:2605.03183v1 Announce Type: new Abstract: Evaluating canine electrocardiograms (ECGs) is challenging due to noise that can obscure clinically relevant cardiac electrical activity. Common sources

researcharxiv-cs-lg