Search: “evaluation”

arXiv:2606.00038v1 Announce Type: cross Abstract: Artificial intelligence (AI) literacy is increasingly recognized as a foundational competency for all university graduates. Yet students' engagement w

tutorialsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Tutorials

Build agents you can trust across any framework with open evals and a control standard

DGX agent

Learn how Microsoft helps developers build trustworthy AI agents with open evaluations, portable runtime controls, production observability, and security workflows that work across frameworks. The pos

tutorialsmicrosoft-foundry

2 Jun 2026

Business Utility of Large Language Models as Exploratory Data Analysis Agents

arXiv:2606.00051v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in analytical workflows, but their suitability as exploratory data analysis (EDA) agents in busines

Can LLMs Reason Structurally? Benchmarking via the Lens of Data Structures

arXiv:2505.24069v4 Announce Type: replace-cross Abstract: Large language models (LLMs) are deployed on increasingly complex tasks that require multi-step decision-making. Understanding their algorithm

CASTLE2026 Team WDL Technical Report

arXiv:2606.00712v1 Announce Type: new Abstract: The CASTLE Challenge @ EgoVis 2026 evaluates long-form egocentric video question answering over 600+ hours of multi-perspective recordings. Each four-ch

CityTrajBench: A Unified Benchmark for City-Scale Vehicle Trajectory Generation

arXiv:2606.02287v1 Announce Type: cross Abstract: Urban trajectory generation is a fundamental task for transportation simulation, urban planning, and mobility analytics. However, systematic compariso

COMAP: Co-Evolving World Models and Agent Policies for LLM Agents

arXiv:2606.02372v1 Announce Type: new Abstract: Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution. However, ex

Applications

Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument

arXiv:2606.01643v1 Announce Type: new Abstract: Sign Language Production (SLP) is the task of generating avatar sign language motion from natural language text. The quality of the generated motion is

applicationsarxiv-cs-cv

Research

Context-aware child-directed speech detection from long-form recordings

arXiv:2606.01134v1 Announce Type: cross Abstract: Automatically distinguishing child-directed speech from adult-directed speech in long-form recordings is key to scalable analyses of children's langua

researcharxiv-cs-lg

Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking

arXiv:2606.01120v1 Announce Type: new Abstract: In RAG-based fact-checking, LLMs are increasingly used as verifiers to check given claims against retrieved evidence. Their parametric knowledge can ind

Agents

Ev-Trust: An Evolutionarily Stable Trust Mechanism for Decentralized LLM-Based Multi-Agent Service Economies

arXiv:2512.16167v3 Announce Type: replace-cross Abstract: Decentralized LLM-based multi-agent service economies face three vulnerabilities that undermine traditional trust mechanisms: reduced cost of

agentsarxiv-cs-ai

Expected Value Alignment for Generative Reward Modeling in Formal Mathematics Verification

arXiv:2606.01160v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used with formal interactive theorem provers such as Lean 4. Scaling these systems with reinforcement lear

Failure of contextual invariance in large language models

arXiv:2603.23485v2 Announce Type: replace-cross Abstract: Standard evaluation practices assume that large language model (LLM) outputs are stable when prompts are embedded in contextually equivalent d

Research

French parsing enhanced with a word clustering method based on a syntactic lexicon

arXiv:2606.00634v1 Announce Type: new Abstract: This article evaluates the integration of data extracted from a French syntactic lexicon, the Lexicon-Grammar (Gross, 1994), into a probabilistic parser

researcharxiv-cs-cl

InsightVQA: High-Dimensional Emotion-Cognitive Visual Question Answering Benchmark

arXiv:2606.02171v1 Announce Type: new Abstract: Visual emotion understanding requires models not only to recognize emotional states, but also to why they arise and perform higher-level cognitive reaso

Interpretable Policy Distillation for Power Grid Topology Control

arXiv:2606.00561v1 Announce Type: cross Abstract: Deep reinforcement learning (RL) offers a promising route to real-time power grid operation, yet large neural policies are costly to evaluate, hard to

K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts

arXiv:2606.02404v1 Announce Type: new Abstract: Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, b

LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation

arXiv:2602.16953v3 Announce Type: replace Abstract: Execution-aware LLM agents offer a promising paradigm for learning from tool feedback, but such feedback can be expensive and slow to obtain, making

Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study

arXiv:2511.02086v2 Announce Type: replace Abstract: Purpose: In this paper, we develop and clinically evaluate a depth-only, markerless augmented reality (AR) registration pipeline on a head-mounted d

safetyarxiv-cs-cv

MASCOT: Towards Multi-Agent Socio-Collaborative Companion Systems

arXiv:2601.14230v2 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) are emerging as promising socio-collaborative companions for emotional and cognitive support. However, existing syst

MBench: A Comprehensive Benchmark on Memory Capability for Video World Models

arXiv:2606.00793v1 Announce Type: new Abstract: Recent advancements in video-based world models have demonstrated an unprecedented ability to synthesize high-fidelity visual sequences. However, a fund

Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning

arXiv:2606.01301v1 Announce Type: new Abstract: Hallucinations in medical large language models (LLMs) pose serious risks for clinical decision support, particularly when models must reason over compl

MedGym:A Unified Continuous-Time Benchmark for Dynamic Medical Treatment Reinforcement Learning

model-releasesarxiv-cs-lg

arXiv:2606.01028v1 Announce Type: new Abstract: Medical treatment recommendation poses several challenges to reinforcement learning (RL): patient physiology evolves in continuous time, measurements an

MemoNoveltyAgent: A Historical Research Memory-Aware Agent Workflow for Paper Novelty Assessment

arXiv:2603.20884v2 Announce Type: replace Abstract: To alleviate the heavy burden of paper screening, researchers increasingly rely on existing AI agents, such as AI reviewers or DeepResearch, for pap

Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation

arXiv:2509.03456v2 Announce Type: replace-cross Abstract: Off-policy evaluation (OPE) and off-policy learning (OPL) are foundational for decision-making in offline contextual bandits. Recent advances

safetyarxiv-cs-lg

Applications

Optimal Transport-based Permutation-Invariant Bayesian Optimization of Offshore Wind Farm Layouts

arXiv:2606.00009v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely and successfully adopted for solving optimization problems having an expensive-to-evaluate, black-box, and non-conv

applicationsarxiv-cs-ai

POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems

arXiv:2606.02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and halluci

Position: Good Embodied Reward Models Need Bad Behavior Data

arXiv:2606.01036v1 Announce Type: new Abstract: This position paper argues that to obtain reliable embodied reward models, the community must invest in ``bad'' robot data: failed, suboptimal, error-pr

safetyarxiv-cs-ro

Question-Aware Evidence Ledgers for Video Relational Reasoning

arXiv:2606.02506v1 Announce Type: new Abstract: The VRR-QA challenge evaluates visual relational reasoning in videos, where answers often depend on implicit spatial relations, event boundaries, target

Research

R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking

arXiv:2606.01113v1 Announce Type: new Abstract: The CoVR-R challenge evaluates composed video retrieval, where a system must retrieve a target video from a large gallery given a reference video and a

researcharxiv-cs-cv

RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates

arXiv:2506.11083v3 Announce Type: replace Abstract: We introduce RedDebate, a novel multi-agent debate framework that provides the foundation for Large Language Models (LLMs) to identify and mitigate

safetyarxiv-cs-cl

RenoBench: A Citation Parsing Benchmark

arXiv:2603.25640v2 Announce Type: replace-cross Abstract: Accurate parsing of citations is necessary for machine-readable scholarly infrastructure. But, despite sustained interest in this problem, exi

RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenes

arXiv:2606.00828v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have shown strong visual understanding and are increasingly deployed in embodied AI systems, where reliable perception und

Local Ai

Spectral Audit of In-Context Operator Networks

arXiv:2606.02427v1 Announce Type: cross Abstract: Existing evaluations of neural operators and in-context operator learning rely primarily on prediction error, but accurate output prediction does not

local-aiarxiv-cs-lg

Tutorials

Strong Stochastic Flow Maps

arXiv:2606.01086v1 Announce Type: cross Abstract: Flow and diffusion models generate high-quality samples in many modalities; however, many network evaluations are required during inference due to num

tutorialsarxiv-cs-ai

The Assistant as a Privileged Persona: A canonical reference in cross-persona self-recognition

model-releasesarxiv-cs-lg

arXiv:2606.00545v1 Announce Type: new Abstract: Post-trained language models can recognize their own outputs from a sentence or two out of context. In a companion paper itep{jack2026twomodes} we showe

TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning

arXiv:2606.01591v1 Announce Type: new Abstract: The TimeLogic Challenge evaluates formal temporal-logic reasoning over video - 16 operators (before, after, until, since, always, co-occur, ordering, ..

Agents

Tracking the Behavioral Trajectories of Adapting Agents

arXiv:2606.02536v1 Announce Type: new Abstract: Text files such as skill files, memory files, and behavioral configuration files play a central role in defining how modern agents act. Through edits by

agentsarxiv-cs-ai

TVIR: Building Deep Research Agents Towards Text--Visual Interleaved Report Generation

arXiv:2606.02320v1 Announce Type: new Abstract: Deep Research Agents have shown strong capability in multi-step information retrieval, reasoning, and long-form report generation, but existing benchmar

Local Ai

Two-Fidelity Best-Action Identification for Stochastic Minimax Tree

arXiv:2606.01708v1 Announce Type: cross Abstract: We study fixed-confidence best-action identification (BAI) in stochastic minimax trees. This problem is increasingly relevant in modern AI planning, w

local-aiarxiv-cs-ai

What to Format and How: A Benchmark and Workflow Approach for Document Formatting

arXiv:2606.01936v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have opened up new possibilities for automated document formatting. However, real-world formatting often

Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories

arXiv:2606.02060v1 Announce Type: new Abstract: Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final ans

3DAE: Binaural Quality Assessment for Audio Novel View Synthesis with Spatial Maps and Benchmark

arXiv:2605.30469v1 Announce Type: cross Abstract: 3D audio and novel-view acoustic synthesis models are usually evaluated with global metrics.However, global metrics often hide where and why binaural

Auto-Discovery-Bench: Diagnosing Structured State Tracking in Oracle-Guided Discovery

arXiv:2502.15224v2 Announce Type: replace-cross Abstract: Interactive discovery requires agents to maintain and update structured beliefs over many rounds of feedback. Before evaluating agents in nois

BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors

arXiv:2605.31153v1 Announce Type: new Abstract: Given the surge of harmful AI-generated imagery online, reliably distinguishing authentic images from generated ones has become an urgent research topic

safetyarxiv-cs-cv

Agents

Choosing the Lens: Strategic Perspective Activation in Context-Dependent Argumentation

arXiv:2605.31581v1 Announce Type: new Abstract: The same arguments often need to be evaluated under different external regimes. An agent with influence over the regime has a strategic lever that stand

agentsarxiv-cs-ai

Conditional Coverage Diagnostics for Conformal Prediction

arXiv:2512.11779v2 Announce Type: replace-cross Abstract: Evaluating conditional coverage remains one of the most persistent challenges in assessing the reliability of predictive systems. Although con

Hardware

Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch

arXiv:2511.17826v2 Announce Type: replace-cross Abstract: Deterministic inference is increasingly critical for large language model (LLM) applications such as LLM-as-a-judge evaluation, multi-agent sy

hardwarearxiv-cs-cl