Search: “evaluation”

LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injectio

arXiv:2605.17986v1 Announce Type: cross Abstract: AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools. This creates indirect prompt-injection (IPI) ri

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Matrix-Decoupled Concentration for Autoregressive Sequences: Dimension-Free Guarantees for Sparse Long-Context Rewards

DGX agent

arXiv:2605.06017v2 Announce Type: replace Abstract: Sequence-level evaluations in autoregressive Large Language Models (LLMs) rely on highly dependent token generation. Establishing tight concentratio

researcharxiv-cs-lg

19 May 2026

PaliBench: A Multi-Reference Blueprint for Classical Language Translation Benchmarks

model-releasesarxiv-cs-cl

arXiv:2605.16881v1 Announce Type: new Abstract: Digital humanities projects increasingly rely on machine translation and large language models to widen access to classical, religious, and otherwise un

Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehicles

arXiv:2605.16858v1 Announce Type: cross Abstract: Autonomous Vehicles (AVs) must make reliable decisions in dense urban environments where pedestrian behavior is variable, sometimes abnormal, and ofte

Perceptual implications of automatic anonymization in pathological speech

arXiv:2505.00409v3 Announce Type: replace-cross Abstract: Automatic anonymization is increasingly used to enable ethical sharing of clinical speech, yet its perceptual and clinical consequences remain

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

arXiv:2603.04639v2 Announce Type: replace-cross Abstract: Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipula

Speak Your Mind: The Speech Continuation Task as a Probe of Voice-Based Model Bias

arXiv:2509.22061v2 Announce Type: replace-cross Abstract: Speech Continuation (SC) is the task of generating a coherent extension of a spoken prompt while preserving both semantic context and speaker

safetyarxiv-cs-cl

State-of-the-Art Claims Require State-of-the-Art Evidence

arXiv:2605.17273v1 Announce Type: cross Abstract: State-of-the-Art (SOTA) claims pervade Artificial Intelligence (AI) and Machine Learning (ML) research. These claims rest on benchmark evaluations, wh

SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

arXiv:2605.17946v1 Announce Type: new Abstract: Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external too

Agents

The Journal of Prompt-Engineered (Moral) Philosophy Or: Why AI-Assisted Ethics Research Requires Process Transparency

arXiv:2511.08639v3 Announce Type: replace-cross Abstract: Existing AI disclosure mandates in scholarship require that AI assistance be reported but leave transparency philosophically unspecified: they

agentsarxiv-cs-ai

Time Series Foundation Models as Strong Baselines in Transportation Forecasting: A Large-Scale Benchmark Analysis

model-releasesarxiv-cs-lg

arXiv:2602.24238v2 Announce Type: replace Abstract: Accurate forecasting of transportation dynamics is essential for urban mobility and infrastructure planning. Although recent work has achieved stron

Uncertainty Reliability Under Domain Shift: An Investigation for Data-Driven Blood Pressure Estimation in Photoplethysmography

arXiv:2605.18008v1 Announce Type: new Abstract: Uncertainty quantification (UQ) is critical for safety-critical domains like healthcare, yet it is rarely evaluated under realistic out-of-distribution

safetyarxiv-cs-lg

VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?

arXiv:2602.04802v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) have achieved impressive performance in cross-modal understanding across textual and visual inputs, yet existing bench

WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scale

model-releasesarxiv-cs-cl

arXiv:2510.16252v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) for web agents demands environments that are both effective for evaluation and efficient enough for large-scale on

Local Ai

When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors

arXiv:2605.17265v1 Announce Type: new Abstract: Accurate prediction of molecular properties underpins drug discovery and material design, yet even state-of-the-art models remain vulnerable to localize

local-aiarxiv-cs-lg

AGOP-IxG: A Gradient Covariance Filter for Local Feature Attribution on Tabular Data, with a Controlled Benchmark

model-releasesarxiv-cs-lg

arXiv:2605.15700v1 Announce Type: new Abstract: Automated machine learning pipelines increasingly produce models whose predictions must be explained to end users, auditors, and downstream decision sys

Can We Trust AI-Inferred User States. A Psychometric Framework for Validating the Reliability of Users States Classification by LLMs in Operational Environments

arXiv:2605.15734v1 Announce Type: new Abstract: The use of large language models to assess user states in conversational and adaptive systems is based on the assumption that the metrics used for such

DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo

model-releasesarxiv-cs-ro

arXiv:2605.16257v1 Announce Type: new Abstract: Achieving human-level manipulation requires dexterous robotic hands capable of complex object interactions. Advancing such capabilities further demands

Effective Harness Engineering for Algorithm Discovery with Coding Agents

arXiv:2605.15221v1 Announce Type: cross Abstract: AlphaEvolve and FunSearch have demonstrated the potential of combining large language models (LLMs) with evolutionary search for automated algorithm d

Gradient-Based Join Ordering

arXiv:2511.14482v2 Announce Type: replace-cross Abstract: Join ordering is the NP-hard problem of selecting the most efficient order in which to evaluate joins (conjunctive, binary operators) in a dat

researcharxiv-cs-lg

Agents

PRISM: Prompt Reliability via Iterative Simulation and Monitoring for Enterprise Conversational AI

arXiv:2605.15665v1 Announce Type: new Abstract: Deploying large language model (LLM)-driven conversational agents in enterprise settings requires prompts that are simultaneously correct at launch and

agentsarxiv-cs-ai

Local Ai

Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routers

arXiv:2505.12601v2 Announce Type: replace Abstract: As large language models (LLMs) grow in scale and specialization, routing--selecting the best model for a given input--has become essential for effi

local-aiarxiv-cs-lg

STAR: A Stage-attributed Triage and Repair framework for RCA Agents in Microservices

arXiv:2605.15581v1 Announce Type: new Abstract: LLM-based root cause analysis (RCA) agents have recently emerged as a promising paradigm for incident diagnosis in microservice AIOps. However, their re

Tutorials

AaSP: Aliasing-aware Self-Supervised Pre-Training for Audio Spectrogram Transformers

arXiv:2512.03637v2 Announce Type: replace-cross Abstract: Transformer-based audio self-supervised learning (SSL) models commonly use spectrograms, vision-style Transformers, and masked modeling object

tutorialsarxiv-cs-lg

Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability

arXiv:2605.14246v1 Announce Type: cross Abstract: Many safety-critical control problems are modeled as risk-sensitive partially observable Markov decision processes, where the controller must make dec

Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia

arXiv:2509.23023v3 Announce Type: replace Abstract: Large language models are increasingly deployed in multi-agent settings whose outcomes hinge on social intelligence, motivating evaluations of their

Emotion-Attended Stateful Memory (EASM):The Architecture for Hyper-Personalization at Scale

arXiv:2605.14833v1 Announce Type: new Abstract: Current language model systems remain fundamentally stateless across sessions, limiting their ability to personalize interactions over time. While retri

EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation

arXiv:2605.15199v1 Announce Type: cross Abstract: Multi-shot video generation extends single-shot generation to coherent visual narratives, yet maintaining consistent characters, objects, and location

Hardware

Falkor-IRAC: Graph-Constrained Generation for Verified Legal Reasoning in Indian Judicial AI

arXiv:2605.14665v1 Announce Type: new Abstract: Legal reasoning is not semantic similarity search. A court judgment encodes constrained symbolic reasoning: precedent propagation, procedural state tran

hardwarearxiv-cs-ai

Fast Adversarial Attacks with Gradient Prediction

arXiv:2605.14868v1 Announce Type: new Abstract: Generating adversarial examples at scale is a core primitive for robustness evaluation, adversarial training, and red-teaming, yet even 'fast' attacks s

researcharxiv-cs-lg

Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution

model-releasesarxiv-cs-cl

arXiv:2605.15138v1 Announce Type: cross Abstract: Standard unlearning evaluations measure behavioral suppression in full precision, immediately after training, despite every deployed language model be

GenExam: A Multidisciplinary Text-to-Image Exam

arXiv:2509.14232v5 Announce Type: replace Abstract: Exams are a fundamental test of expert-level intelligence and require integrated understanding, reasoning, and generation. Existing exam-style bench

Tutorials

Large Language Models for Web Accessibility: A Systematic Literature Review

arXiv:2605.13873v1 Announce Type: cross Abstract: Web accessibility aims to ensure that web content and services are usable by people with diverse abilities. In recent years, Large Language Models (LL

tutorialsarxiv-cs-ai

Tutorials

LLMs learn scientific taste from institutional traces across the social sciences

arXiv:2603.16659v3 Announce Type: replace Abstract: Reinforcement-learned reasoning has powered recent AI leaps on verifiable tasks, including mathematics, code, and structure prediction. The harder b

tutorialsarxiv-cs-ai

MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving

arXiv:2605.14201v1 Announce Type: cross Abstract: Vision-language-action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings due to bein

safetyarxiv-cs-cv

Hardware

Parallelizing Counterfactual Regret Minimization

arXiv:2605.14277v1 Announce Type: new Abstract: Parallelization has played an instrumental role in the field of artificial intelligence (AI), drastically reducing the time taken to train and evaluate

hardwarearxiv-cs-ai

PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation

arXiv:2605.14269v1 Announce Type: cross Abstract: Generating realistic human motion is a central yet unsolved challenge in video generation. While reinforcement learning (RL)-based post-training has d

Quantifying and Mitigating Premature Closure in Frontier LLMs

arXiv:2605.15000v1 Announce Type: cross Abstract: Premature closure, or committing to a conclusion before sufficient information is available, is a recognized contributor to diagnostic error but remai

Safety-Constrained Reinforcement Learning with Post-Training Reachability Verification for Robot Navigation

arXiv:2605.14174v1 Announce Type: new Abstract: Safe navigation for mobile robots demands policies that remain reliable under the high-consequence perception uncertainty of cluttered environments. Yet

safetyarxiv-cs-ro

SceneParser: Hierarchical Scene Parsing for Visual Semantics Understanding

arXiv:2605.14923v1 Announce Type: new Abstract: General scene perception has progressed from object recognition toward open-vocabulary grounding, part localization, and affordance prediction. Yet thes

SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding

arXiv:2510.13016v3 Announce Type: replace Abstract: A truly capable AI system must do more than detect objects or recognize activities in isolation. It must form unified, grounded representations of w

Towards Robotic Dexterous Hand Intelligence: A Survey

arXiv:2605.13925v1 Announce Type: new Abstract: Robotic dexterous hands are central to contact-rich manipulation, with rapid progress driven by advances in hardware, sensing, control, simulation, and

researcharxiv-cs-ro

Unsteady Metrics and Benchmarking Cultures of AI Model Builders

arXiv:2605.14164v1 Announce Type: new Abstract: The primary way to establish and compare competencies in foundation and generative AI models has shifted from peer-reviewed literature to press releases

A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data

arXiv:2603.14066v2 Announce Type: replace-cross Abstract: Many real-world multi-party negotiations unfold as sequences of binding, action-level commitments rather than a single final outcome, yet this

Code-Centric Detection of Vulnerability-Fixing Commits: A Unified Benchmark and Empirical Study

model-releasesarxiv-cs-lg

arXiv:2605.13138v1 Announce Type: cross Abstract: Automated detection of vulnerability-fixing commits (VFCs) is critical for timely security patch deployment, as advisory databases lag patch releases

Context Matters: Auditing Gender Bias in T2I Generation through Risk-Tiered Use-Case Profiles

arXiv:2605.13113v1 Announce Type: cross Abstract: Text-to-image (T2I) generative models are increasingly used to produce content for education, media, and public-facing communication, and are starting

Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models

arXiv:2605.12519v1 Announce Type: cross Abstract: Training language models to produce both correct answers and sound reasoning remains an open challenge. Reinforcement learning with verifiable rewards

CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis

arXiv:2605.12650v1 Announce Type: new Abstract: Foundation diffusion models can generate photorealistic natural images, but adapting them to medical imaging remains challenging. In medical adaptation,

safetyarxiv-cs-cv