Search: “evaluation”

arXiv:2608.11322v1 Announce Type: cross Abstract: Human-AI research often evaluates individual capabilities, combined performance, or final outputs, but these approaches do not preserve how one party'

local-aiarxiv-cs-ai

13 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

A Graph Neural Network--Guided Genetic Algorithm for Physical Internet Supply Chain Optimization under Cost Uncertainty

DGX agent

arXiv:2608.10245v1 Announce Type: cross Abstract: Inventory and distribution planning in Physical Internet networks requires coordinating factory-hub assignments, factory supply, lateral transshipment

researcharxiv-cs-lg

12 Aug 2026

Tutorials

Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?

arXiv:2608.10867v1 Announce Type: new Abstract: Gradient-free post-training has emerged as a compelling alternative to gradient-based optimization for large language models (LLMs), but existing approa

tutorialsarxiv-cs-lg

Cost-Efficient Estimation of General Abilities Across Benchmarks

arXiv:2604.01418v2 Announce Type: replace Abstract: Thousands of diverse benchmarks have been developed to measure the quality of large language models (LLMs). Yet prior work has demonstrated that LLM

Agents

Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories

arXiv:2608.10319v1 Announce Type: cross Abstract: Large language model (LLM)-powered agents have rapidly evolved from code-completion tools into solvers of complex software engineering tasks. As devel

agentsarxiv-cs-ai

FaithformBench: Benchmarking Faithfulness of Mathematical Chain-of-Thought Autoformalisation

arXiv:2608.10916v1 Announce Type: cross Abstract: Autoformalisation (AF) systems map natural language reasoning steps into formal statements in a proof assistant such as Lean. We consider how to asses

Robustness of transferability estimation metrics for medical imaging

arXiv:2608.09999v1 Announce Type: cross Abstract: In transfer learning, the choice of source model largely influences the performance on a target dataset. Still, selecting a fitting source remains a c

researcharxiv-cs-cv

Agents

The Deliberative Deficit: An Empirical Critique of LLMs in Democratic Discourse

arXiv:2608.10186v1 Announce Type: cross Abstract: LLMs are increasingly deployed in settings that require collective reasoning on complex, value-laden problems. Confidence in these deployments rests l

agentsarxiv-cs-ai

UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs

arXiv:2608.10042v1 Announce Type: cross Abstract: Tool-use LLMs are increasingly asked to act on users' behalf, but existing benchmarks usually focus on profile recall, style imitation, generic tool u

AndroidReality: How Far Are Mobile Agents from the Real World?

arXiv:2608.07775v1 Announce Type: new Abstract: Mobile agents have achieved promising results on clean online benchmarks such as AndroidWorld, yet their performance often degrades sharply in real-worl

Local Ai

Auditing Medical Vision-Language Models on Chest Radiographs: Estimating Reference Agreement Across Institutions

arXiv:2608.07550v1 Announce Type: new Abstract: Vision-language models return structured chest-radiograph findings through interfaces exposing no confidence score, so a receiving institution cannot re

local-aiarxiv-cs-cv

Agents

Business Truth, not SQL Accuracy: A Rule-Gated 7B Analytics Agent Outperforms a Direct-Prompted 32B Baseline

arXiv:2608.09254v1 Announce Type: new Abstract: LLM analytics agents are evaluated on SQL syntax accuracy, but production failures look different: questions with two valid business definitions, questi

agentsarxiv-cs-ai

Coarse-to-Fine Registration of Jawbone CT and Intraoral Scan Data Using GeDi and ICP with Pseudo-IOS Ground Truth

arXiv:2608.07564v1 Announce Type: cross Abstract: In digital dentistry and oral surgery, the registration of jawbone CT and intraoral scanner (IOS) data is essential for integrating internal bone stru

safetyarxiv-cs-ai

CodecArena: Codec Quality Assessment via Visual Reinforcement Learning

arXiv:2608.09139v1 Announce Type: new Abstract: Video coding is advancing into the low and ultra-low bitrate regime, driven by end-to-end codecs that replace the hand-crafted pipeline with jointly opt

Contrastive Mask Fidelity: Reference-Free Auditing of Ground-Truth Masks in Remote Sensing Semantic Segmentation

arXiv:2608.09101v1 Announce Type: new Abstract: Semantic segmentation models are trained and evaluated against human-drawn masks, yet remote-sensing annotations are often coarse, incomplete, or misali

researcharxiv-cs-cv

Decoding-Level Taboo: A Diagnostic Stress Test for LLM Robustness

arXiv:2608.09900v1 Announce Type: new Abstract: Large language model evaluations typically focus on performance under nominal conditions, creating an illusion of capability where models comfortably wa

Tutorials

Ensemble learning of pathology foundation models for precision oncology

arXiv:2508.16085v2 Announce Type: replace Abstract: Histopathology is essential for cancer diagnosis and treatment selection, and pathology foundation models learn visual representations from whole-sl

tutorialsarxiv-cs-cv

Tutorials

Experience-Sensitive Game Learning: A Behavioral Study of Humans and Language Agents

arXiv:2608.07490v1 Announce Type: cross Abstract: Large language model agents are increasingly evaluated through games, but most benchmarks emphasize final outcomes rather than how players learn from

tutorialsarxiv-cs-ai

FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models

arXiv:2608.08736v1 Announce Type: new Abstract: Fitness Action Quality Assessment (AQA) is important for intelligent sports training, yet the capabilities of Multimodal Large Language Models (MLLMs) i

Gaming Without an Attacker: Benchmark Fingerprinting in LLM-Driven Search Under Selection Pressure

arXiv:2608.08722v1 Announce Type: cross Abstract: Benchmarks for systems that are optimized against the evaluation signal measure something different from what they claim. We document this concretely

Hidden Language Consistency Phenomena in Reasoning LLMs

arXiv:2608.08447v1 Announce Type: cross Abstract: Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended langu

How Do Large Language Models Judge Social Attraction? Evidence from Theory-Grounded Persona Ratings Across Multiple LLMs and Humans

arXiv:2608.09717v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to perform subjective evaluations traditionally made by humans, yet their validity as social judges

researcharxiv-cs-ai

How Far Do Foundation Models Transfer to Infant Signals? A Cross-Dataset Transfer Audit with a Unified Need Ontology

arXiv:2608.08989v1 Announce Type: cross Abstract: Public infant cry corpora are small, label-incompatible, and almost always evaluated one corpus at a time. We ask what this practice hides and what fi

researcharxiv-cs-ai

I gave DeepSeek V4 Flash basic vision by training a 40M connector on 100K examples

model-releasesr-localllama

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon

IDATA: Scalable Invertible Diffusion for Unrestricted Adversarial Transfer Attack

arXiv:2608.08734v1 Announce Type: new Abstract: Unrestricted adversarial transfer attacks are important for evaluating the black-box robustness of deep visual models. Diffusion-based attacks have show

researcharxiv-cs-cv

Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation

arXiv:2608.09420v1 Announce Type: new Abstract: User simulators are widely used as scalable environments for training and evaluating interactive assistants. Generating the next user turn is inherently

safetyarxiv-cs-cl

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks

arXiv:2507.19634v4 Announce Type: replace-cross Abstract: Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a s

MiraMind: Benchmarking Reliable Mental Health Reasoning beyond Answer Accuracy

arXiv:2512.09636v3 Announce Type: replace Abstract: Mental-health reasoning with large language models (LLMs) is an evidence-constrained judgment problem: models must transform limited, subjective, an

MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models

arXiv:2603.28590v3 Announce Type: replace Abstract: Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs. When such a mis

Physics-Informed Policy Iteration for High-Dimensional Hamilton--Jacobi--Bellman Equations: Interior Error Bounds without Boundary Data

arXiv:2508.01718v2 Announce Type: replace Abstract: We develop a physics-informed policy-iteration method for stationary second-order Hamilton--Jacobi--Bellman equations arising in continuous-time sto

safetyarxiv-cs-lg

Software Engineering for and with GUI Agent

arXiv:2608.09278v1 Announce Type: cross Abstract: GUI agents have advanced rapidly, producing a growing body of frameworks, benchmarks, and applications. However, this growth has outpaced the maturity

safetyarxiv-cs-ai

SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs

arXiv:2608.07862v1 Announce Type: new Abstract: Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguist

TCS-BENCH: Benchmarking State-of-the-Art Generative AI Theoretical Computer Science Research Ability

arXiv:2608.09538v1 Announce Type: cross Abstract: We introduce TCS-Bench, a benchmark for evaluating Large Language Models (LLMs) on research-level Theoretical Computer Science (TCS) proof generation.

Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution

arXiv:2608.06811v1 Announce Type: cross Abstract: Resolving a real software issue with a large language model (LLM) agent is a long repair episode, often tens to hundreds of steps spanning exploration

Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitation

arXiv:2608.06955v1 Announce Type: new Abstract: Large language models (LLMs) are trained on corpora that contain expressions of human judgment about films, books, music, and more. Yet whether LLMs sys

GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models

arXiv:2608.06769v1 Announce Type: new Abstract: Recent Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse vision-language tasks, creating an urgent need for more

How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality

arXiv:2604.06756v2 Announce Type: replace Abstract: Large language models (LLMs) has been widely adopted as a scalable surrogate for human evaluation, yet such judges remain imperfect and susceptible

researcharxiv-cs-cl

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

arXiv:2509.16462v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economi

safetyarxiv-cs-cl

Multi-Perspective Triad Interaction Graph Neural Network for Cognitive Distortion Detection

arXiv:2608.06785v1 Announce Type: new Abstract: Cognitive distortion detection is a key task in computational mental health, yet existing approaches often overlook the psychological structure of disto

safetyarxiv-cs-cl

UniREditBench: A Unified Reasoning-based Image Editing Benchmark

arXiv:2511.01295v3 Announce Type: replace Abstract: Recent advances in multi-modal generative models have driven substantial improvements in image editing. However, current generative models still str

WebRider: Persona-Conditioned Intent Controllers for Live-Web Assistance

arXiv:2608.06704v1 Announce Type: new Abstract: Delegating a web task involves more than asking a question; it requires transferring a policy: what to verify, how to handle uncertainty, which preferen

YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family

arXiv:2608.07051v1 Announce Type: new Abstract: Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous op

Now we have a timeline of the OpenAI accidental attack against Hugging Face

model-releasessimon-willison

My comment on Now we have a timeline of the OpenAI accidental attack against Hugging Face — Hacker News.I think one of the most interesting details here might be tucked away in that first bulletin poi

8 Aug 2026

Clinician input steers AI toward accurate and harmful recommendations

model-releasesarxiv-cs-lg

arXiv:2603.14158v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are entering clinical workflows, yet evaluations rarely assess how clinician reasoning shapes model behavior duri

EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?

arXiv:2608.06022v1 Announce Type: new Abstract: Epitopes determine where antibodies bind antigens and shape downstream therapeutic properties such as functional blockade and escape resistance, making

From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs

arXiv:2608.05560v1 Announce Type: cross Abstract: Timely anticipation of physical hazards is essential for real-world safety, yet existing MLLM evaluations focus on harmful content or general risks, l

Hypothesis Testing with Conditional Queries: Learnability and the Value of Interaction

arXiv:2608.06262v1 Announce Type: new Abstract: Model evaluations may fix all tests before observing any responses or select later tests using earlier responses. We study this choice in a conditional-

safetyarxiv-cs-lg

Integrating Human Linguistic Insights into AI: Theory-Driven Representation for Multilingual Text-to-Speech

arXiv:2204.07228v2 Announce Type: replace Abstract: This paper explores the integration of human linguistic insights into multilingual text-to-speech (TTS) systems by evaluating the Featurally Undersp

researcharxiv-cs-cl