Search: “evaluation”

Evidential Deep Learning for Multi-Modal Anti-UAV Detection

arXiv:2609.01742v1 Announce Type: new Abstract: Anti-UAV systems increasingly fuse multiple sensors, yet their detection heads provide no per-modality reliability signal. This study evaluates whether

researcharxiv-cs-cv

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

DGX agent

arXiv:2609.02679v1 Announce Type: cross Abstract: When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited huma

researcharxiv-cs-ai

3 Sept 2026

LLM-as-a-Judge Is Not an Oracle: Why Self-Improving Agents Need Deterministic Guardrails

arXiv:2609.02246v1 Announce Type: new Abstract: Self-improving agent pipelines have a problem at their center. An optimizer rewrites prompts to score higher, and the score comes from a judge that is i

safetyarxiv-cs-ai

LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematization

arXiv:2607.10103v2 Announce Type: replace-cross Abstract: As large language models (LLMs) become widely deployed, their outputs can be copied, transformed, and redistributed at scale without reliable

Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting

arXiv:2609.02649v1 Announce Type: new Abstract: Aggregating noisy, conflicting textual hypotheses into a reliable consensus is a fundamental challenge when deploying NLP systems in real-world industri

MESSY STREETS: A Benchmark for Geocoding Real-World Addresses

arXiv:2609.01612v1 Announce Type: cross Abstract: We introduce MESSY STREETS, a benchmark for evaluating geocoders on verbatim web addresses, with existence verification and controlled measurement of

MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts

arXiv:2609.02379v1 Announce Type: cross Abstract: While existing work on LLM authorship attribution (AA) has made progress, available benchmarks remain limited, often focusing on English, controlled s

Post-Training Language Models for Gold-Medal Performance in Coding Competitions

arXiv:2609.02849v1 Announce Type: cross Abstract: Competitive programming has become a key test of large language model reasoning, with international competitions such as IOI and ICPC representing its

Agents

RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems

arXiv:2609.01622v1 Announce Type: cross Abstract: The rise of agentic AI has catalyzed a shift toward self-iterating systems, opening new frontiers for the autonomous optimization of production recomm

agentsarxiv-cs-ai

Local Ai

SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition

arXiv:2609.02036v1 Announce Type: new Abstract: Few-step diffusion models substantially compress temporal computation, making the spatial cost of each model evaluation an increasingly dominant source

local-aiarxiv-cs-cv

Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM

arXiv:2512.15662v4 Announce Type: replace Abstract: Human beings solve complex problems through critical thinking, where reasoning and evaluation are intertwined to converge toward correct solutions.

researcharxiv-cs-ai

TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding

arXiv:2609.01810v1 Announce Type: new Abstract: Farsi, spoken by more than 120 million people, lacks a comprehensive benchmark for dialogue generation and understanding. We introduce TALKFA, a unified

VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation

arXiv:2609.02462v1 Announce Type: new Abstract: End-to-end autonomous driving in urban environments requires robust decision-making under partial observability and complex multi-agent interactions. Se

WinoQueer-NL: Assessing Bias in Dutch Language Models toward LGBTQ+ Identities

arXiv:2609.02651v1 Announce Type: new Abstract: While English language models have been widely examined for anti-queer bias, Dutch models remain understudied. To address this gap, we developed a cultu

WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling

arXiv:2609.01608v1 Announce Type: cross Abstract: Black-box optimization problems remain challenging because of large, weakly structured, and high-dimensional search spaces. Existing methods often suf

Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbooks

arXiv:2609.00763v1 Announce Type: new Abstract: Hierarchical Knowledge graph (KG)-based retrieval augmented generation (RAG) has emerged as a powerful approach for supporting large language models wit

Local Ai

Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling

arXiv:2609.00949v1 Announce Type: cross Abstract: Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now

local-aiarxiv-cs-ai

Can LLMs Design Video Coding Tools? A Case Study on Planar Mode

arXiv:2609.01535v1 Announce Type: cross Abstract: This paper explores whether large language models (LLMs) can design video coding tools, a highly challenging task due to the intricate algorithmic cou

Controllable Image Captioning with Prompt-Conditioned Scene Rewards

arXiv:2609.00709v1 Announce Type: cross Abstract: Large Vision-Language Models produce fluent image descriptions but offer limited semantic control: users cannot reliably specify whether captions shou

Data-Driven Persona-Conditioned Agents for A/B Test Simulation

arXiv:2609.01038v1 Announce Type: new Abstract: A/B testing is the gold standard for evaluating product changes, but each experiment requires real user traffic, engineering effort, and weeks of measur

DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation

arXiv:2609.00646v1 Announce Type: new Abstract: Commercial short-drama production follows a multi-stage chain: script, storyboard, keyframe imagery, shot-level video, and the finished short drama. Mos

EDRAC: Benchmarking Arabic Dialect Reading Comprehension

arXiv:2609.01113v1 Announce Type: cross Abstract: Dialectal Arabic (DA) remains under-resourced compared to Modern Standard Arabic (MSA), particularly for machine reading comprehension (MRC) and quest

Tutorials

FedReview: Review and Dispose Poisoned Updates without Validation Datasets or Historic Knowledge

arXiv:2402.16934v2 Announce Type: replace-cross Abstract: Federated learning has emerged as a decentralized approach for training high-performance models without accessing user data. Despite its effec

tutorialsarxiv-cs-ai

Fi-ImageNet-1k: An OOD Benchmark From the Inside of the ImageNet-1k Validation Set

arXiv:2609.01027v1 Announce Type: new Abstract: Out-of-distribution (OOD) detection predicts whether a test image belongs to none of the predefined classes. To evaluate this task, benchmarks need imag

GenScale: A Benchmark for Relative Object Scale in Image Generation and Editing

arXiv:2609.00525v1 Announce Type: new Abstract: Modern image generation and editing systems can produce photorealistic, prompt-aligned images, but still often render familiar objects at implausible re

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

arXiv:2609.01437v1 Announce Type: cross Abstract: As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly

HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation

arXiv:2609.01046v1 Announce Type: cross Abstract: Production LLMs must handle inputs that attempt to override system instructions, bypass safety policies or elicit harmful responses. A common mitigati

Integrating Traffic Noise Emission Modelling into Variable Speed Limit Control

arXiv:2609.01339v1 Announce Type: new Abstract: Road traffic noise remains a major environmental challenge, yet most speed management strategies are static and do not respond to short-term variations

safetyarxiv-cs-ro

Applications

Is Knowledge Distillation Actually Greener? A Case Study in Machine Translation

arXiv:2602.09691v2 Announce Type: replace Abstract: Knowledge distillation (KD) is a technique to compress a larger teacher system into a smaller student. In machine translation, KD is commonly evalua

applicationsarxiv-cs-cl

LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning

arXiv:2601.13880v2 Announce Type: replace Abstract: Personalized lifestyle health analysis requires long-horizon, multi-dimensional reasoning over heterogeneous lifestyle signals, and recent advances

MeshSplatBench: A Unified Benchmark for Triangle-Based Neural Rendering

arXiv:2609.01306v1 Announce Type: cross Abstract: Triangle-based neural rendering bridges neural scene representations and conventional graphics pipelines by optimizing explicit geometric primitives c

Scene Graph-based Driving Scenario Extraction for Automotive Egocentric Datasets

model-releasesarxiv-cs-ro

arXiv:2609.00333v1 Announce Type: new Abstract: Extracting scenarios from unlabelled real-world sensor data streams is a critical but challenging task in the development process of automated driving s

StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions

arXiv:2609.01081v1 Announce Type: cross Abstract: Large language models often answer the same multiple-choice question inconsistently when it is posed under support-oriented and elimination-oriented f

researcharxiv-cs-ai

Agents

The Rise of Verbal Reinforcement Learning

arXiv:2609.01597v1 Announce Type: cross Abstract: Natural language is emerging as a primary feedback channel for improving language agents, capable of conveying intent, preferences, and causal structu

agentsarxiv-cs-ai

Training-Free Inpainting Across Domains with a Frozen Text-to-Image Diffusion Model

arXiv:2609.00862v1 Announce Type: new Abstract: We show that a frozen generic text-to-image diffusion model can perform conditional inpainting across three evaluated natural-image domains with one fix

researcharxiv-cs-cv

VoiceLongMemEval: Do Assistants Remember How You Sounded?

arXiv:2609.00570v1 Announce Type: new Abstract: With the growing scale of multi-agent architectures and large language models, deployed AI assistants are increasingly tasked with reasoning over long,

Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR

arXiv:2609.01354v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text answer

Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance

arXiv:2606.18613v4 Announce Type: replace-cross Abstract: The most plausible near-term role of medical LLMs is to assist rather than replace physicians, yet current evaluations often test isolated cap

Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

arXiv:2606.15017v2 Announce Type: replace Abstract: Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume t

Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oracle

arXiv:2608.30581v1 Announce Type: new Abstract: Large language models (LLMs) are used as post hoc explainers of sequential decision-making policies, producing natural-language explanations of why an a

safetyarxiv-cs-ai

Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs

arXiv:2608.30256v1 Announce Type: cross Abstract: Logical reasoning with large language models (LLMs) is a critical capability, as it reflects a system's ability to correctly deduce hypotheses from a

researcharxiv-cs-ai

BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

arXiv:2608.31105v1 Announce Type: new Abstract: Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of

safetyarxiv-cs-ai

Breaking Darknet CAPTCHAs with general purpose LLM

arXiv:2608.28794v1 Announce Type: cross Abstract: Our work evaluates the effectiveness of automated methods for solving CAPTCHA challenges commonly encountered in darknet environments. These CAPTCHAs

safetyarxiv-cs-cv

Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks

arXiv:2510.01359v3 Announce Type: replace-cross Abstract: Code-capable large language model (LLM) agents are embedded in software engineering workflows where they can read, write, and execute code, ra

CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy

arXiv:2608.30768v1 Announce Type: new Abstract: Automatic neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved enco

Local Ai

Cost-Effective Repository Exploration for Agentic Issue Localization

arXiv:2608.29675v1 Announce Type: cross Abstract: Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository fi

local-aiarxiv-cs-ai

Applications

Foundation Models Meet Agriculture: Challenges Beyond Pretraining

arXiv:2608.30392v1 Announce Type: new Abstract: Global food security and sustainable climate action increasingly rely on robust, scalable agricultural monitoring. Earth observation foundation models h

applicationsarxiv-cs-lg

Generative vs. Encoder Models for Multilingual NER: A Comprehensive Empirical Study on Naamapadam

arXiv:2608.29959v1 Announce Type: new Abstract: Language is humanity's most consequential technology, yet for over a billion speakers across India's twenty-two constitutionally recognised languages, i