Search: “evaluation”

FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving

model-releasesarxiv-cs-ro

arXiv:2608.13395v1 Announce Type: new Abstract: Reinforcement learning improves autonomous-driving vision-language-action (VLA) models by evaluating trajectories sampled from the current policy. Group

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

GeoCache: Training-Free Acceleration of Multi-View Texture Diffusion via Geometric Delta Transport

DGX agent

arXiv:2608.13255v1 Announce Type: cross Abstract: Geometry-conditioned multi-view diffusion enables high-quality 3D texture generation, but its repeated per-view denoiser evaluations introduce substan

researcharxiv-cs-ai

14 Aug 2026

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

arXiv:2608.13555v1 Announce Type: cross Abstract: Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos. Kin

Jagged Judges: Epistemic Stability Under Silence, Pressure, and Persistence

arXiv:2608.12645v1 Announce Type: new Abstract: LLM judges have become central infrastructure for model evaluations, online grading, and reward modeling. Judges are typically validated by accuracy on

Large Language Models Persuade Without Planning Theory of Mind

arXiv:2602.17045v2 Announce Type: replace Abstract: A growing body of work attempts to evaluate the theory of mind (ToM) abilities of humans and large language models (LLMs) using static, non-interact

safetyarxiv-cs-cl

PatientAct: Theory-Grounded Mental Health Client Simulation

arXiv:2608.12750v1 Announce Type: cross Abstract: LLM-based simulated clients are increasingly used to train novice counselors, evaluate LLM therapists, and generate synthetic data. However, current s

Personalized Scorer Modeling: A Learning-Based Framework for Deriving Robust Sleep Stage Labels from Multiple Experts

arXiv:2608.12446v1 Announce Type: cross Abstract: Sleep stage classification is important for the diagnosis and management of sleep disorders, yet most automatic staging studies evaluate models agains

PhysMaster: Building an Autonomous AI Physicist for Theoretical and Computational Physics Research

arXiv:2512.19799v2 Announce Type: replace Abstract: Advances in LLM reasoning and tool use have enabled agentic science, yet frontier theoretical and computational physics remains challenging because

Privacy-Preserving RAG by Concealing Sensitive Information from External LLMs

arXiv:2608.12675v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) is widely used to improve the performance of Large Language Models (LLMs) in answering user queries. Existing priva

Robust Dempster-Shafer Evidence Fusion with Chaos-Conflict Measurement and Historical-Experience Weighting

arXiv:2608.13108v1 Announce Type: new Abstract: Multi-source evidence fusion under Dempster-Shafer theory faces two persistent challenges: existing conflict measures assess inter-evidence inconsistenc

Applications

Semantic Radiance Fields as Simulators for Spatial Reasoning in Real-World Scenes

arXiv:2608.13095v1 Announce Type: cross Abstract: Training and evaluating spatial reasoning in embodied agents requires diverse environments that are both geometrically faithful and semantically query

applicationsarxiv-cs-cv

Accuracy and Order Sensitivity Diverge Under Label-Free Strategies

arXiv:2608.11947v1 Announce Type: cross Abstract: Multiple-choice benchmarks are widely used to evaluate large language models, but MCQ scores conflate knowledge with sensitivity to option order, whic

Automated binary classification of hazelnut X-ray images: A deep-learning benchmark for quality assessment

model-releasesarxiv-cs-cv

arXiv:2608.11759v1 Announce Type: new Abstract: Non-destructive X-ray imaging can reveal internal hazelnut defects that are difficult to detect by external inspection alone; however, automated interpr

Commonsense on Demand: Generating and Selectively Integrating Commonsense Knowledge for Natural Language Inference

arXiv:2507.15100v3 Announce Type: replace-cross Abstract: Natural Language Inference (NLI) determines whether a premise entails, contradicts, or is neutral with respect to a hypothesis. The task is of

Large-scale AI-Ready Data for Anti-Cancer Drug Response Modeling

model-releasesarxiv-cs-lg

arXiv:2608.11444v1 Announce Type: cross Abstract: Drug response prediction (DRP) models are an active area of research in pharmacogenomics, with growing potential to accelerate the identification of e

On the Definition of Intelligence

arXiv:2507.22423v3 Announce Type: replace Abstract: To engineer AGI, we should first capture the essence of intelligence in a species-agnostic form that can be evaluated, while being sufficiently gene

ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generation

arXiv:2411.15122v2 Announce Type: replace-cross Abstract: AI-driven models have demonstrated significant potential in automating radiology report generation for chest X-rays. However, there is no stan

RoadWeaver: Large-Scale Lane-Level HD Map Generation from Scratch for Autonomous Driving Simulation

arXiv:2608.11580v1 Announce Type: cross Abstract: Autonomous driving simulation requires diverse and scalable lane-level HD maps to support long-horizon evaluation across complex road networks. Existi

Local Ai

Socioduality: A Relational Process Framework for Human-AI Interaction

arXiv:2608.11322v1 Announce Type: cross Abstract: Human-AI research often evaluates individual capabilities, combined performance, or final outputs, but these approaches do not preserve how one party'

local-aiarxiv-cs-ai

A Graph Neural Network--Guided Genetic Algorithm for Physical Internet Supply Chain Optimization under Cost Uncertainty

arXiv:2608.10245v1 Announce Type: cross Abstract: Inventory and distribution planning in Physical Internet networks requires coordinating factory-hub assignments, factory supply, lateral transshipment

researcharxiv-cs-lg

Tutorials

Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?

arXiv:2608.10867v1 Announce Type: new Abstract: Gradient-free post-training has emerged as a compelling alternative to gradient-based optimization for large language models (LLMs), but existing approa

tutorialsarxiv-cs-lg

Cost-Efficient Estimation of General Abilities Across Benchmarks

model-releasesarxiv-cs-cl

arXiv:2604.01418v2 Announce Type: replace Abstract: Thousands of diverse benchmarks have been developed to measure the quality of large language models (LLMs). Yet prior work has demonstrated that LLM

Agents

Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories

arXiv:2608.10319v1 Announce Type: cross Abstract: Large language model (LLM)-powered agents have rapidly evolved from code-completion tools into solvers of complex software engineering tasks. As devel

agentsarxiv-cs-ai

FaithformBench: Benchmarking Faithfulness of Mathematical Chain-of-Thought Autoformalisation

arXiv:2608.10916v1 Announce Type: cross Abstract: Autoformalisation (AF) systems map natural language reasoning steps into formal statements in a proof assistant such as Lean. We consider how to asses

Robustness of transferability estimation metrics for medical imaging

arXiv:2608.09999v1 Announce Type: cross Abstract: In transfer learning, the choice of source model largely influences the performance on a target dataset. Still, selecting a fitting source remains a c

researcharxiv-cs-cv

Agents

The Deliberative Deficit: An Empirical Critique of LLMs in Democratic Discourse

arXiv:2608.10186v1 Announce Type: cross Abstract: LLMs are increasingly deployed in settings that require collective reasoning on complex, value-laden problems. Confidence in these deployments rests l

agentsarxiv-cs-ai

UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs

arXiv:2608.10042v1 Announce Type: cross Abstract: Tool-use LLMs are increasingly asked to act on users' behalf, but existing benchmarks usually focus on profile recall, style imitation, generic tool u

AndroidReality: How Far Are Mobile Agents from the Real World?

arXiv:2608.07775v1 Announce Type: new Abstract: Mobile agents have achieved promising results on clean online benchmarks such as AndroidWorld, yet their performance often degrades sharply in real-worl

Local Ai

Auditing Medical Vision-Language Models on Chest Radiographs: Estimating Reference Agreement Across Institutions

arXiv:2608.07550v1 Announce Type: new Abstract: Vision-language models return structured chest-radiograph findings through interfaces exposing no confidence score, so a receiving institution cannot re

local-aiarxiv-cs-cv

Agents

Business Truth, not SQL Accuracy: A Rule-Gated 7B Analytics Agent Outperforms a Direct-Prompted 32B Baseline

arXiv:2608.09254v1 Announce Type: new Abstract: LLM analytics agents are evaluated on SQL syntax accuracy, but production failures look different: questions with two valid business definitions, questi

agentsarxiv-cs-ai

Coarse-to-Fine Registration of Jawbone CT and Intraoral Scan Data Using GeDi and ICP with Pseudo-IOS Ground Truth

arXiv:2608.07564v1 Announce Type: cross Abstract: In digital dentistry and oral surgery, the registration of jawbone CT and intraoral scanner (IOS) data is essential for integrating internal bone stru

CodecArena: Codec Quality Assessment via Visual Reinforcement Learning

model-releasesarxiv-cs-cv

arXiv:2608.09139v1 Announce Type: new Abstract: Video coding is advancing into the low and ultra-low bitrate regime, driven by end-to-end codecs that replace the hand-crafted pipeline with jointly opt

Contrastive Mask Fidelity: Reference-Free Auditing of Ground-Truth Masks in Remote Sensing Semantic Segmentation

arXiv:2608.09101v1 Announce Type: new Abstract: Semantic segmentation models are trained and evaluated against human-drawn masks, yet remote-sensing annotations are often coarse, incomplete, or misali

researcharxiv-cs-cv

Decoding-Level Taboo: A Diagnostic Stress Test for LLM Robustness

model-releasesarxiv-cs-cl

arXiv:2608.09900v1 Announce Type: new Abstract: Large language model evaluations typically focus on performance under nominal conditions, creating an illusion of capability where models comfortably wa

Tutorials

Ensemble learning of pathology foundation models for precision oncology

arXiv:2508.16085v2 Announce Type: replace Abstract: Histopathology is essential for cancer diagnosis and treatment selection, and pathology foundation models learn visual representations from whole-sl

tutorialsarxiv-cs-cv

Tutorials

Experience-Sensitive Game Learning: A Behavioral Study of Humans and Language Agents

arXiv:2608.07490v1 Announce Type: cross Abstract: Large language model agents are increasingly evaluated through games, but most benchmarks emphasize final outcomes rather than how players learn from

tutorialsarxiv-cs-ai

FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models

arXiv:2608.08736v1 Announce Type: new Abstract: Fitness Action Quality Assessment (AQA) is important for intelligent sports training, yet the capabilities of Multimodal Large Language Models (MLLMs) i

Gaming Without an Attacker: Benchmark Fingerprinting in LLM-Driven Search Under Selection Pressure

arXiv:2608.08722v1 Announce Type: cross Abstract: Benchmarks for systems that are optimized against the evaluation signal measure something different from what they claim. We document this concretely

Hidden Language Consistency Phenomena in Reasoning LLMs

arXiv:2608.08447v1 Announce Type: cross Abstract: Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended langu

How Do Large Language Models Judge Social Attraction? Evidence from Theory-Grounded Persona Ratings Across Multiple LLMs and Humans

arXiv:2608.09717v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to perform subjective evaluations traditionally made by humans, yet their validity as social judges

How Far Do Foundation Models Transfer to Infant Signals? A Cross-Dataset Transfer Audit with a Unified Need Ontology

arXiv:2608.08989v1 Announce Type: cross Abstract: Public infant cry corpora are small, label-incompatible, and almost always evaluated one corpus at a time. We ask what this practice hides and what fi

IDATA: Scalable Invertible Diffusion for Unrestricted Adversarial Transfer Attack

arXiv:2608.08734v1 Announce Type: new Abstract: Unrestricted adversarial transfer attacks are important for evaluating the black-box robustness of deep visual models. Diffusion-based attacks have show

researcharxiv-cs-cv

Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation

arXiv:2608.09420v1 Announce Type: new Abstract: User simulators are widely used as scalable environments for training and evaluating interactive assistants. Generating the next user turn is inherently

safetyarxiv-cs-cl

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks

arXiv:2507.19634v4 Announce Type: replace-cross Abstract: Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a s

MiraMind: Benchmarking Reliable Mental Health Reasoning beyond Answer Accuracy

model-releasesarxiv-cs-cl

arXiv:2512.09636v3 Announce Type: replace Abstract: Mental-health reasoning with large language models (LLMs) is an evidence-constrained judgment problem: models must transform limited, subjective, an

MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models

arXiv:2603.28590v3 Announce Type: replace Abstract: Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs. When such a mis

Physics-Informed Policy Iteration for High-Dimensional Hamilton--Jacobi--Bellman Equations: Interior Error Bounds without Boundary Data

arXiv:2508.01718v2 Announce Type: replace Abstract: We develop a physics-informed policy-iteration method for stationary second-order Hamilton--Jacobi--Bellman equations arising in continuous-time sto

safetyarxiv-cs-lg

Software Engineering for and with GUI Agent

arXiv:2608.09278v1 Announce Type: cross Abstract: GUI agents have advanced rapidly, producing a growing body of frameworks, benchmarks, and applications. However, this growth has outpaced the maturity