Search: “evaluation”

A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

arXiv:2605.28556v1 Announce Type: new Abstract: As agent capabilities advance, existing benchmarks, such as au^2-Bench, are becoming increasingly saturated. Yet constructing new benchmark tasks remain

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

A novel ordinal multi-view aggregation scheme for oak defoliation

DGX agent

arXiv:2605.28151v1 Announce Type: new Abstract: Forest decline driven by climate and biotic stressors threatens ecosystem functioning, making accurate monitoring of tree health essential. In this work

researcharxiv-cs-cv

28 May 2026

Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents

arXiv:2605.22166v2 Announce Type: replace Abstract: LLM agents are shaped not only by their language models, but also by the runtime harness that mediates observation, tool use, action execution, feed

agentsarxiv-cs-ai

Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named

arXiv:2605.27396v1 Announce Type: cross Abstract: Autonomous AI agents now plan, decide, and act on behalf of users across healthcare, financial services, and workplace contexts, often without step-by

agentsarxiv-cs-ai

An Empirical Audit of k-NAF Budget Accounting for Anchored Decoding

arXiv:2605.28001v1 Announce Type: new Abstract: We empirically audit the k-NAF budget-accounting mechanism in Anchored Decoding using (i) a fixed, class-stratified workload (approximately 8,500 random

researcharxiv-cs-ai

Are We Truly Innovating? A Qualitative and Quantitative Study of Originality in AI Research Papers

arXiv:2602.06054v3 Announce Type: replace Abstract: Assessing originality in AI research is arguably the most consequential yet least reliable step in peer review. Reviewer judgments of originality re

researcharxiv-cs-cl

Assessing Factual Music Comprehension in Large Audio Language Models

arXiv:2511.05550v2 Announce Type: replace-cross Abstract: Large audio language models (LALMs) leverage multimodal representations to generate open-ended answers to natural language queries about audio

Benchmarking Inductive Biases for Multivariate Time-Series Anomaly Detection with a Robust Multi-View Channel-Graph Detector

model-releasesarxiv-cs-lg

arXiv:2605.28103v1 Announce Type: new Abstract: We present a unified experiment, analysis, and benchmark study of multivariate time-series (MTS) anomaly detection. Ten family-representative detectors

BuddyBench: A Privacy-Constrained Multi-Task Benchmark for Pediatric Social-Communication Personalization

arXiv:2605.28089v1 Announce Type: new Abstract: BuddyBench introduces a privacy-constrained multi-task benchmark for pediatric social-communication personalization. Unlike existing neurodevelopmental

Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

arXiv:2510.05291v2 Announce Type: replace Abstract: As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly im

COOP^2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems

arXiv:2603.00349v2 Announce Type: replace Abstract: Many complex tasks require extended effort, diverse capabilities, or coordinated actions beyond what a single agent can provide. However, simply add

agentsarxiv-cs-ai

Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies

arXiv:2605.24302v2 Announce Type: replace Abstract: Egocentric action recognition is a challenging task due to erratic camera motion, frequent hand occlusion, and the difficulty of maintaining consist

researcharxiv-cs-cv

Cultural Fidelity in English-to-Hindi Translation: A Preservation-Fluency Frontier for Gender Recoverability

arXiv:2605.27654v1 Announce Type: cross Abstract: Generative translation systems are cultural technologies because they decide how socially meaningful cues are rendered within culturally specific gram

Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models

arXiv:2512.00349v2 Announce Type: replace Abstract: Are frontier AI systems becoming more capable? Certainly. Yet such progress is not an unalloyed blessing but rather a Trojan horse: behind their per

Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024

arXiv:2503.02857v5 Announce Type: replace-cross Abstract: In the age of increasingly realistic generative AI, robust deepfake detection is essential for mitigating fraud and disinformation. While many

Applications

Degradation-Aware Cooperative Multi-Modal GNSS-Denied Localization Leveraging LiDAR-Based Robot Detections

arXiv:2510.20480v2 Announce Type: replace Abstract: Accurate long-term localization using onboard sensors is crucial for robots operating in Global Navigation Satellite System (GNSS)-denied environmen

applicationsarxiv-cs-ro

Diagnosing Live Within-Policy Instruction Conflicts in LLM Agents with Witnessed Resolution Profiles

arXiv:2605.27784v1 Announce Type: new Abstract: LLM agents are governed by long-lived natural-language prompt policies, but individually reasonable standing rules can interact in uninspected ways. We

safetyarxiv-cs-ai

Diffusion-Based Ukrainian Handwritten Text Generation with Cross-Domain Style Transfer

arXiv:2605.27487v1 Announce Type: cross Abstract: Handwritten text generation (HTG) conditioned on writer style has been widely studied for Latin scripts, but remains underexplored for low-resource an

Disentangling Language Roles in Multilingual LLM Task Execution

arXiv:2605.27649v1 Announce Type: new Abstract: Multilingual LLMs are increasingly used when instruction, source content, and required response languages do not coincide. Existing benchmarks have expa

Dr-CiK: A Testbed for Foresight-Driven Agents

arXiv:2605.27904v1 Announce Type: new Abstract: Time series forecasting in real-world settings often depends not only on historical observations, but also on external context that must be actively dis

E^3-Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference

arXiv:2605.27428v1 Announce Type: new Abstract: Edge deployments of generative inference increasingly face two practical realities: per-device per-model performance is often unknown at deployment time

agentsarxiv-cs-lg

Efficient and Scalable Provenance Tracking for LLM-Generated Code Snippets

arXiv:2605.28510v1 Announce Type: cross Abstract: Large language models (LLMs) for code completion and generation are increasingly used in software development, yet they may reproduce training example

EventShiftFlow: Towards Hardware-efficient FPGA-based Flow Estimation

arXiv:2605.28312v1 Announce Type: cross Abstract: Event-based vision sensors offer asynchronous, high-temporal-resolution measurements that are attractive for low-latency robotic perception, but many

Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objects

arXiv:2605.28270v1 Announce Type: new Abstract: Estimating the 9D pose of everyday objects from a single real-world image remains challenging. This is largely due to the lack of large-scale supervisio

Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification

arXiv:2605.28500v1 Announce Type: cross Abstract: Large language models have shown impressive capabilities in code generation, yet they often produce functionally incorrect code. Uncertainty quantific

GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation

arXiv:2605.27491v1 Announce Type: new Abstract: We introduce GE-Sim 2.0 (Genie Envisioner World Simulator 2.0), a closed-loop video world simulator for robotic manipulation. Building on the action-con

safetyarxiv-cs-ro

GeneralThinker: Domain-General Reasoning through Likelihood-Guided Answer-Conditioned Optimization

arXiv:2605.27934v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards improves language model reasoning, but its reliance on domain-specific verifiers, sparse outcome rewards,

safetyarxiv-cs-cl

HELEA: Hard-Negative Benchmark and LLM-based Reranking for Robust Entity Alignment

arXiv:2605.28308v1 Announce Type: new Abstract: Entity Alignment (EA) is essential for knowledge graph (KG) fusion, but existing benchmarks often allow models to exploit name overlap rather than relat

Applications

IGADA-IoT: IoT Sensor Energy Optimization in Wireless Sensor Networks Driven by Automatic Data Augmentation

arXiv:2605.27397v1 Announce Type: new Abstract: In wireless sensor networks (WSNs), data augmentation is a novel method to improve sampling-frequency decision performance, thereby enabling energy opti

applicationsarxiv-cs-lg

Laguna M.1/XS.2 Technical Report

arXiv:2605.27605v1 Announce Type: new Abstract: We present Laguna M.1 and Laguna XS.2, two Mixture-of-Experts foundation models built for long-horizon, agentic coding: M.1 has 225.8B total parameters

LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?

arXiv:2605.28721v1 Announce Type: new Abstract: Are LLM-based search agents genuinely searching, or using the web to verify what they already know? We study this question on BrowseComp with three diag

Measuring Progress Toward AGI: A Cognitive Framework

arXiv:2605.28405v1 Announce Type: new Abstract: Despite widespread discussion of AGI, there is no clear framework for measuring progress toward it. This ambiguity fuels subjective claims, makes it dif

researcharxiv-cs-ai

MeniOmni: A Structured Multimodal Benchmark for Holistic Meniscus Injury Assessment

arXiv:2605.28161v1 Announce Type: new Abstract: Clinical diagnosis of meniscus injuries requires radiologists to integrate volumetric MRI evidence with patient context (e.g., sex, age, BMI) and to pro

Local Ai

MMRad-22K: A Structured Multimodal Evidence Dataset for Chest X-ray Report Generation

arXiv:2602.12843v2 Announce Type: replace Abstract: Chest X-ray (CXR) reporting follows a region-based clinical workflow in which radiologists inspect anatomical regions and integrate localized findin

local-aiarxiv-cs-cv

PAST2HARM: A Simple Adaptive Past Tense Attack for Jailbreaking Multimodal AI

arXiv:2605.27545v1 Announce Type: new Abstract: Jailbreak attacks on multimodal AI systems remain underexplored, even though unsafe image generation can have more severe consequences than unsafe text

Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control

model-releasesarxiv-cs-lg

arXiv:2601.15015v2 Announce Type: replace Abstract: Reinforcement learning (RL) has shown promising results in active flow control (AFC), yet progress in the field remains difficult to assess as exist

POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigation

arXiv:2605.28237v1 Announce Type: cross Abstract: Real-world navigation is fundamentally driven by Points of Interest (POIs), yet reaching a precise POI remains a critical 'final-meters' challenge. Ex

Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups

arXiv:2510.06974v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in user-facing applications, raising concerns that they may reflect and amplify social biases

safetyarxiv-cs-cl

Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation

arXiv:2605.28230v1 Announce Type: new Abstract: Modern video generative models produce visually impressive results, yet frequently violate basic physical principles. We propose Proprio, a training-fre

researcharxiv-cs-cv

ProvMind: Provenance-grounded reasoning for materials synthesis

arXiv:2605.28487v1 Announce Type: new Abstract: Materials process optimization requires reasoning over routes, conditions, tools and causal dependencies, yet most computational formulations flatten sy

Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations

arXiv:2605.28553v1 Announce Type: new Abstract: In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on

safetyarxiv-cs-ai

Relational Semantic Reasoning on 3D Scene Graphs for Open World Interactive Object Search

arXiv:2603.05642v2 Announce Type: replace-cross Abstract: Open-world interactive object search in household environments requires understanding semantic relationships between objects and their surroun

Resource-Constrained Affect Modelling via Variance Regularisation Pruning

arXiv:2605.27479v1 Announce Type: cross Abstract: Affective computing systems are increasingly embedded in pervasive and interactive environments, such as adaptive games, assistive technologies, and r

Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure

arXiv:2605.27996v1 Announce Type: new Abstract: Single-axis mitigations of reward-model biases (e.g., reducing proxy reliance on length, sycophancy, or style) can rotate optimization pressure onto cor

safetyarxiv-cs-ai

Single-Rollout Hidden-State Dynamics for Training-Free RLVR Data Selection

arXiv:2605.28631v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) can yield large reasoning gains from very few training instances, yet its strong sensitivity to wh

researcharxiv-cs-lg

SmartIterator: Visual Analytics Workflows for Supervising Unsupervised Data Grouping

arXiv:2605.28219v1 Announce Type: cross Abstract: Unsupervised learning methods -- topic modeling, partition-based and density-based clustering -- produce data groupings without human guidance, yet ch

Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language

model-releasesarxiv-cs-ro

arXiv:2605.27886v1 Announce Type: new Abstract: Tactile sensing is essential for robots to achieve human-like gentle manipulation. However, existing Vision-Language-Action (VLA) models struggle to exp

The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages

arXiv:2605.27901v1 Announce Type: cross Abstract: Chain-of-thought (CoT) monitoring has been proposed as a promising safety mechanism for detecting misaligned behavior in large language models. Howeve