Search: “evaluation”

Stress-testing large language model agents in a robotic chemistry laboratory

arXiv:2607.23045v1 Announce Type: new Abstract: AI is evaluated through knowledge, reasoning and plan generation, yet scientific agency requires reliable physical action and adaptation to evidence. He

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

TEmBed-T: A Multi-Dimensional Benchmark for Table-Level Embeddings

DGX agent

arXiv:2607.24130v1 Announce Type: cross Abstract: Tabular data is the dominant structured-data modality, and learning table representations has become a core research direction. Table-level embeddings

model-releasesarxiv-cs-lg

28 Jul 2026

Text-based Tactile Graphics Generation for the Visually Impaired

arXiv:2607.22674v1 Announce Type: cross Abstract: Tactile graphics are a primary medium for blind and low-vision (BLV) individuals to access non-textual information. However, they are difficult to sca

researcharxiv-cs-cv

The Cost of Knowing: A Resource-Aware Protocol for Benchmarking Hallucination Beyond Static Leaderboards

arXiv:2607.24063v1 Announce Type: new Abstract: On standard factuality tasks, frontier models now cluster near the top of the scale. The question is therefore shifting from how factual a system is tow

Understanding Tone-Dependent Inference Cost in Large Language Models

arXiv:2607.23915v1 Announce Type: cross Abstract: We examine how prompt tone affects both accuracy of the LLM answers and inference cost as reflected in output-token consumption. Experiments were perf

Which Models Perform Better in Inheritance Reasoning?

arXiv:2606.13751v4 Announce Type: replace Abstract: This paper presents the participation of team PSL in the QIAS 2026 Shared Task on Arabic Islamic inheritance reasoning. The task evaluates the abili

Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents

arXiv:2606.13385v2 Announce Type: replace-cross Abstract: LLM-based web agents are increasingly deployed in real-world settings such as e-commerce, where they interact extensively with untrusted web c

XMatchAD: A Cross-Modal Matching Perspective on Reconstruction-based Anomaly Detection

arXiv:2607.23658v1 Announce Type: new Abstract: The remarkable success of reconstruction-based methods in Unsupervised Anomaly Detection (UAD) lies in their ability to identify and localize anomalies

Bounding the Causal Impact of ML-assisted Decision-Making via Counterfactual Correctness

model-releasesarxiv-cs-lg

arXiv:2607.21806v1 Announce Type: new Abstract: Predictive machine learning (ML) models are increasingly used to aid human decision-makers across various high-risk domains such as healthcare and crimi

Embodiment-Induced Coordination Regimes in Tabular Multi-Agent Q-Learning

arXiv:2601.17454v2 Announce Type: replace-cross Abstract: Centralized value learning underlies a broad class of multi-agent reinforcement learning methods, but its claimed advantage is typically evalu

agentsarxiv-cs-lg

From Obligation to Specification: A Survey on Validating EU AI Act Requirements in RE

arXiv:2607.21608v1 Announce Type: cross Abstract: With the EU AI Act entering into force, organizations developing or operating AI systems face new obligations on transparency, risk management, and tr

safetyarxiv-cs-cl

From traditional ML to AI agents: How Booking.com scales AI observability with Arize

How Booking.com built a unified AI observability stack with Arize for agentic GenAI workflows and traditional ML — from telemetry collection and PII redaction to latency monitors and evaluations. The

agentsarize-ai

ISPCloak: Weaponizing ISP for Optimization-Free Physical Camouflage against Deepfake Detectors

arXiv:2607.21897v1 Announce Type: new Abstract: The rapid advancement of generative models has spurred the critical need to evaluate the worst-case robustness of deepfake detectors. In this paper, we

researcharxiv-cs-cv

Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision

arXiv:2603.07025v2 Announce Type: replace Abstract: Speech Large Language Models (LLMs) that understand and follow instructions in many languages are useful for real-world interaction, but are difficu

Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection

arXiv:2607.21776v1 Announce Type: cross Abstract: Talking-face (TF) deepfake generation synthesizes photore- alistic facial video from a static source image and an au- dio signal, producing forgeries

researcharxiv-cs-cv

Probing Latent Colombian Identity Inferences in Qwen2.5-7B with Natural Language Autoencoders

arXiv:2607.21774v1 Announce Type: new Abstract: Large language models may infer demographic attributes from subtle linguistic cues even when those attributes are not explicitly stated. This pilot stud

safetyarxiv-cs-cl

The Coordination Gap: Multi-Agent Alternation Metrics for Temporal Fairness in Repeated Games

arXiv:2603.05789v5 Announce Type: replace-cross Abstract: Repeated multi-agent interactions require evaluation metrics that capture not only payoff distributions but also their temporal organization.

safetyarxiv-cs-lg

ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation

arXiv:2607.22530v1 Announce Type: new Abstract: Contact-rich robot manipulation requires physical interaction cues that are often invisible to cameras, making tactile sensing essential for robust cont

safetyarxiv-cs-ro

Applications

A Polynomial Architecture-Attribution Co-Design Framework for Exact Aumann-Shapley Attribution in GNNs

arXiv:2607.21094v1 Announce Type: new Abstract: We study feature-level and node-level explanations for graph neural networks (GNNs) through the lens of Aumann-Shapley attribution. Path-integral method

applicationsarxiv-cs-lg

Agentic Designer: Progressive Multi-Agent Collaboration for Structure-Aware Interior Layout Generation

arXiv:2607.20866v1 Announce Type: new Abstract: Generating realistic interior furniture layouts that strictly adhere to architectural constraints (e.g., walls, doors, and windows) remains a fundamenta

Automated Synthesis and Adversarial Validation of Executable Causal Research Pipelines

model-releasesarxiv-cs-lg

arXiv:2607.21173v1 Announce Type: new Abstract: While automated research systems promise to accelerate empirical analysis, they are prone to silent failures: instances in which analysis code executes

Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment

arXiv:2607.20476v1 Announce Type: new Abstract: We present an empirical benchmark evaluating how five large language models assess multisensor physical hazard data. Testing 60 scenarios across three c

Best-of-Evidence: Best-of-N Selection under Partial Verification

arXiv:2607.20950v1 Announce Type: new Abstract: BoN improves model outputs by sampling several candidates and selecting one with a proxy score, but it assumes that complete candidates can be evaluated

researcharxiv-cs-lg

CANN Bench: Benchmarking Agent Generated Kernels against Real NPU and Algorithmic Limits

arXiv:2607.20518v1 Announce Type: new Abstract: AI agents are now capable of writing, compiling, and iteratively optimizing low-level operator kernels on different hardware platforms. Existing benchma

CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languages

arXiv:2607.21016v1 Announce Type: new Abstract: Culture is lived through conversation, yet existing Indonesian cultural commonsense benchmarks evaluate LLMs on short and isolated prompts, stripping aw

DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Servers

arXiv:2607.20531v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed over Model Context Protocol (MCP) servers, yet the benchmarks used to evaluate them score th

Local Ai

ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models

arXiv:2607.20499v1 Announce Type: new Abstract: Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability. We presen

local-aiarxiv-cs-ai

Explainable Deepfake Detection Challenge

arXiv:2607.21007v1 Announce Type: new Abstract: Deepfake detection is moving beyond binary classification decisions toward systems that can also explain the visual evidence supporting those decisions.

FORGE-plus: Force-Budgeted Recovery for Contact-Rich Assembly with a Frozen LLM Supervisor

arXiv:2607.21227v1 Announce Type: new Abstract: Force-conditioned reinforcement learning (RL) enables tight-clearance assembly under a commanded force ceiling, but practical deployment requires determ

safetyarxiv-cs-ro

From a Word-Level Dictionary to Sentence-Level Semantics: Multilingual Grievance Labelling with Contextual Models

arXiv:2607.20946v1 Announce Type: new Abstract: Grievance is one of the warning signs analysts look for when assessing threats of violence. It is increasingly measured at scale from online text, most

Human-in-the-Loop Large Language Model Framework for Identification of Cutaneous Immune-Related Adverse Events

arXiv:2607.20428v1 Announce Type: new Abstract: This study evaluated a retrieval-augmented, multi-agent large language model (LLM)-driven, human-in-the-loop framework for detecting cutaneous immune-re

agentsarxiv-cs-cl

ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues

arXiv:2604.01925v2 Announce Type: replace-cross Abstract: Large Language Models increasingly suppress biased outputs when demographic identity is stated explicitly, yet may still exhibit implicit bias

Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning

arXiv:2607.21591v1 Announce Type: new Abstract: Diffusion and flow-matching models dominate conditional image generation, yet inference-time scaling for these models is far less developed than for aut

safetyarxiv-cs-cv

LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning

researchapple-ml-research

Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposit

LeanFlow: A Case Study in Workflow-Driven Lean Autoformalization

arXiv:2607.20503v1 Announce Type: new Abstract: We present and evaluate LeanFlow, an LLM agent system specialized for translating mathematical papers into buildable Lean projects. Recent verifier-in-t

agentsarxiv-cs-ai

Lessons and Open Questions from a Unified Study of Camera-Trap Species Recognition Over Time

arXiv:2603.20509v2 Announce Type: replace Abstract: Camera traps are vital for large-scale biodiversity monitoring, yet accurate automated analysis remains challenging due to diverse deployment enviro

Tutorials

LLMs Get Lost in Evolving User Intent

arXiv:2607.20734v1 Announce Type: new Abstract: As LLMs become more capable, they are increasingly deployed as collaborative agents, taking on user-delegated tasks through iterative interaction. Yet g

tutorialsarxiv-cs-lg

MemTools: A Unified Research Framework for Interoperable Agent Memory

arXiv:2607.21404v1 Announce Type: new Abstract: While memory systems are essential for agent architectures, pervasive architectural fragmentation restricts systematic research. Existing implementation

MIRROR: Learning from the Other View for Multi-Modal Reasoning

arXiv:2607.21552v1 Announce Type: new Abstract: Unlike large language models (LLMs) that exhibit strong reasoning capabilities, vision-language models (VLMs) struggle with visual reasoning, even on ge

researcharxiv-cs-ai

MVEI & EmObserver: Empowering MLLM-Oriented Visual Emotional Intelligence via Emotion Statement Judgement

arXiv:2607.21061v1 Announce Type: new Abstract: Affective Image Content Analysis (AICA) aims to recognize and understand emotions elicited by visual content, representing an indispensable step toward

REGARD: Regional Affective Differences in Large Language Models

arXiv:2607.20722v1 Announce Type: new Abstract: Large language models trained and aligned within different linguistic and regional ecosystems may frame the same political, cultural, and geopolitical e

Response drift across frontier large language models

arXiv:2607.20454v1 Announce Type: cross Abstract: All frontier large language models (LLMs) exhibit response drift -- producing outputs that deviate from expert-validated references -- yet the magnitu

researcharxiv-cs-ai

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation

arXiv:2607.21137v1 Announce Type: new Abstract: Independent sidewalk mobility is essential for blind and visually impaired pedestrians (BVIPs), yet smartphone-based assistive navigation requires perce

safetyarxiv-cs-cv

Scaling Closed-Loop Feature Channel Configuration with LLMs

arXiv:2607.20516v1 Announce Type: cross Abstract: Promising initial results in closed-loop large-language-model-based channel-configuration search demonstrated that neural-network widths can be optimi

Applications

Smooth Neural Point Processes via B-Splines

arXiv:2607.21098v1 Announce Type: new Abstract: Temporal point processes (TPPs) provide a general and flexible framework for modeling sequences of events in continuous time. Neural networks have been

applicationsarxiv-cs-lg

Statistical Inference for Generative Model Comparison

model-releasesarxiv-cs-lg

arXiv:2501.18897v4 Announce Type: replace-cross Abstract: Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty qua

TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

arXiv:2607.11007v2 Announce Type: replace Abstract: Few-shot multimodal classification commonly attaches a lightweight head, such as k-nearest neighbors, logistic regression, or a linear SVM, to a fro

researcharxiv-cs-lg

WaveformQA: Benchmarking LLM Temporal Reasoning on Digital Waveforms

arXiv:2607.20638v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated strong capabilities in code generation and reasoning, yet their ability to perform temporal reasoning ove