Search: “evaluation”

Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym

arXiv:2609.37267v1 Announce Type: new Abstract: Proactive LLM agents can turn idle compute into useful support before users ask. Yet even correct work can misread user context, impose review costs, or

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

From Judgment Quality to Downstream Utility: Rethinking LLM-as-a-Judge for Open-Ended Tasks

DGX agent

arXiv:2609.37145v1 Announce Type: new Abstract: LLM-as-a-Judge is increasingly used to evaluate policy responses on open-ended tasks that lack ground-truth answers. Existing work often directly conver

safetyarxiv-cs-ai

30 Sept 2026

LEGO-Anything: Coding Agents for 3D Scene Reconstruction

arXiv:2609.36380v1 Announce Type: new Abstract: A 3D scene reconstructed from a single image is most useful when represented not as a rendering or a fixed 3D output, but as an explicit scene program w

MARCO: Multi-Round Agentic Reinforcement for Conditional Molecular Optimization

arXiv:2609.36683v1 Announce Type: cross Abstract: Molecular optimization is inherently iterative: a candidate is proposed, evaluated against several objectives, and revised while preserving a relation

MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

arXiv:2508.14925v2 Announce Type: replace-cross Abstract: By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a c

Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark

arXiv:2609.36039v1 Announce Type: cross Abstract: Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing st

OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells

arXiv:2609.37773v1 Announce Type: new Abstract: Artificial Intelligence Virtual Cells (AIVCs) are envisioned as scientific agents that simulate cellular responses, explain underlying mechanisms, and s

Retrieve, Reproduce, Reveal: Dissecting Retrieval-Augmented Software Vulnerability Detection

arXiv:2609.37669v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) is increasingly used to enhance Large Language Model (LLM)-based software vulnerability detection by grounding pr

Sharp Convergence and Sampling Trade-offs for Riemannian Diffusion under Nonnegative Ricci Curvature

arXiv:2609.36568v1 Announce Type: new Abstract: Diffusion models have emerged as state-of-the-art generative models, with recent extensions from Euclidean spaces to Riemannian manifolds. However, exis

VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics

arXiv:2609.37287v1 Announce Type: cross Abstract: Image translation is a fundamental capability of multimodal models for multilingual applications, requiring visual understanding and meaning preservat

A Multi-Dataset Benchmark of YOLO-Based Weed Detection in Precision Agriculture

arXiv:2609.33991v1 Announce Type: new Abstract: Weed detection is an important component of precision agriculture, enabling site-specific weed management and reducing unnecessary herbicide use. Althou

AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modeling

arXiv:2605.05586v2 Announce Type: replace Abstract: High-fidelity CFD is essential for aerodynamic design, but repeated simulations are computationally expensive, motivating surrogate models for rapid

AgentWare: Automating the Lifecycle of Agentic Applications across the Edge-to-Cloud Continuum

arXiv:2609.34586v1 Announce Type: cross Abstract: Deploying LLM-enabled agentic applications across the Edge-to-Cloud continuum remains challenging due to hardware heterogeneity, deployment complexity

agentsarxiv-cs-lg

Applications

Federated Computation of ROC and PR Curves

arXiv:2510.04979v2 Announce Type: replace Abstract: Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are fundamental analytical tools for evaluating classification models, prov

applicationsarxiv-cs-lg

ForVis: An In-Field Dataset and Benchmark for VIO Using Under-Canopy UAV Flights in Forests

model-releasesarxiv-cs-ro

arXiv:2609.35482v1 Announce Type: new Abstract: Visual-inertial Simultaneous Localization and Mapping (VI-SLAM) for UAVs remains difficult to evaluate in real forest environments, where motion, illumi

GeoShrink: Accelerating Diffusion Transformers with Two Lines of Code

arXiv:2609.33723v1 Announce Type: new Abstract: Diffusion transformers incur substantial inference cost through repeated model evaluations along a sampling trajectory. We introduce GeoShrink, a traini

researcharxiv-cs-cv

MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models

arXiv:2609.33893v1 Announce Type: new Abstract: Large audio-language models (LALMs) produce fluent responses about audio but often hallucinate by making plausible yet ungrounded claims. Existing audio

Safety

Not All Errors Matter: Decision-Relevant Prediction Error Predicts Planning Quality

arXiv:2609.32322v1 Announce Type: new Abstract: World models are typically trained and evaluated by prediction error, assuming that more accurate predictions lead to better decisions. We show that thi

safetyarxiv-cs-lg

Physics-Informed Neural Networks for Depth-Averaged Avalanche Dynamics

arXiv:2609.34916v1 Announce Type: cross Abstract: Accurate prediction of avalanche motion is essential for hazard assessment in mountainous terrain. This study develops and evaluates a physics-informe

PlaylistEval: Can Video-Language Judges Be Trusted at Day Scale and Beyond?

arXiv:2609.34314v1 Announce Type: new Abstract: Video-language models are increasingly used as judges of video understanding, both for evaluating model outputs and for training reward models. Whether

Printability-Constrained Adversarial Decals for Near-Nadir Aerial Perception: Measured Ink Gamuts, Nested Realism Constraints, and a Physical-World Bound

arXiv:2609.33513v1 Announce Type: new Abstract: Adversarial patches for aerial perception are typically evaluated as digital composites, with printing left as an implementation detail. This study impo

researcharxiv-cs-cv

REALIS: A Curated Dataset for Studying the Challenges of AI Image Detection

arXiv:2609.32734v1 Announce Type: new Abstract: AI-generated image detectors are often evaluated on benchmarks where real and synthetic images differ in content, quality, or generation artifacts, allo

Safety

The Statistical Benefits of Multiple Responses for Learning from Demonstrations

arXiv:2609.33291v1 Announce Type: cross Abstract: Many generative systems return multiple candidate responses and are evaluated according to the best one. Recent work shows that, when demonstrations a

safetyarxiv-cs-lg

A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory

arXiv:2609.30813v1 Announce Type: new Abstract: Evaluating claim admission in shared agent memory is challenging because repeated claims may be mistaken for independent evidence. An agent may copy or

28 Sept 2026

Auditing and Repairing LLM-as-Judge Failures in a Production Text-to-SQL Pipeline

arXiv:2609.30290v1 Announce Type: new Abstract: Production text-to-SQL pipelines often end with an LLM-as-judge whose agreement with human annotators has never actually been measured. When we checked

28 Sept 2026

Seasonal and Quantum-inspired Models for Neutron Monitor Time Series Forecasting

arXiv:2609.30281v1 Announce Type: new Abstract: We present a focused and reproducible study of multi-horizon forecasting on the Lomnicky Stit neutron monitor (LMKS) time series. Our evaluation suite c

28 Sept 2026

Coding Agents for Generalized Task and Motion Planning Problems

arXiv:2609.30233v1 Announce Type: cross Abstract: Task and motion planning (TAMP) problems remain difficult even with full observability and object-centric states because discrete decisions are tightl

COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages

arXiv:2609.28826v1 Announce Type: new Abstract: Machine translation (MT) for Indian languages remains constrained by the limited availability of high-quality, Indic-centric parallel corpora and evalua

LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches

arXiv:2604.01754v2 Announce Type: replace-cross Abstract: Mathematical reasoning is a hallmark of human intelligence, and whether large language models (LLMs) can meaningfully perform it remains a cen

MLPerf Automotive

arXiv:2510.27065v2 Announce Type: replace Abstract: We present MLPerf Automotive, the first standardized public performance benchmark for evaluating Machine Learning systems that are deployed for AI a

ModularSQL: A Runtime Guardrail for the Multiplicity Blind Spot in Text-to-SQL

arXiv:2609.29573v1 Announce Type: new Abstract: Text-to-SQL systems are increasingly deployed on production databases, where queries that pass benchmark evaluation can still produce results that disto

Reward Hacking Challenges Oversight of Autonomous Research Agents

arXiv:2609.28614v1 Announce Type: new Abstract: Autonomous research agents can design experiments, evaluate results, and write reports, giving them control over both a scientific result and the eviden

agentsarxiv-cs-cl

TopU-LBVS: A Realistic Multi Target Benchmark for Ligand Based Virtual Screening

arXiv:2609.29740v1 Announce Type: cross Abstract: Ligand-based virtual screening (LBVS) is a practical first-pass tool in early-stage drug discovery, but existing benchmarks can overestimate performan

An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice

arXiv:2609.28335v1 Announce Type: new Abstract: Claims about AI safety reach audiences well beyond the AI community, yet many rely on opaque evidence or static assessments, when supporting evidence is

Applications

Cross-Lingual Legal QA for Vietnamese Labour Law: Retrieval, Translation, and Verifier-Guided Correction

arXiv:2609.27376v1 Announce Type: new Abstract: Cross-lingual legal question answering must retrieve statutes across languages while preventing unsupported legal claims. We introduce a bilingual evalu

applicationsarxiv-cs-cl

Applications

Not What You Meant: Can LLMs Follow a Specified Negation Semantics?

arXiv:2609.27517v1 Announce Type: new Abstract: Negation does not carry a uniform interpretation across domains. In legal, regulatory, and medical reasoning, the intended interpretation depends on the

applicationsarxiv-cs-ai

Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms

arXiv:2609.26942v1 Announce Type: cross Abstract: Current literature evaluates large language models (LLMs) on multilingual kinship understanding using multiple choice benchmarks, treating it as a rec

AgentBetta: Verification-Driven Adaptive Configuration of an AI Nano-Agent through Selective Expansion and Verified Contraction

arXiv:2609.23512v1 Announce Type: new Abstract: Large language model agents are typically deployed with predefined configurations, although the required model capability, context, tools, permissions,

Safety

Beyond Reconstruction Error: Analytical and Data-Driven Action Tokenization for Autoregressive Vision-Language-Action Models

arXiv:2609.25820v1 Announce Type: cross Abstract: Discrete action tokenization is central to autoregressive vision-language-action (VLA) models, yet action representations are often evaluated primaril

safetyarxiv-cs-lg

BigO(Bench): Can LLMs Generate Code with Controlled Time and Space Complexity?

arXiv:2503.15242v3 Announce Type: replace Abstract: We introduce BigO(Bench), a novel coding benchmark designed to evaluate the capabilities of generative language models in understanding and generati

FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing

arXiv:2609.25298v1 Announce Type: new Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely

Tutorials

Identifying Intelligent Processes via Online Sequential Testing

arXiv:2609.26193v1 Announce Type: cross Abstract: Active sequential hypothesis testing studies how to identify an unknown hypothesis with a given set of sensing actions. We study this in the setting o

tutorialsarxiv-cs-lg

Initialization and Stopping Tolerance in CPU Dermoscopic Segmentation

arXiv:2609.25685v1 Announce Type: new Abstract: Contour initialization and numerical stopping can jointly affect the evaluation of active-contour segmentation. We examine their interaction using the o

researcharxiv-cs-cv

SafetyFlow: An Agent-Flow System for Automated LLM Safety Benchmarking

arXiv:2508.15526v2 Announce Type: replace Abstract: The rapid proliferation of large language models (LLMs) has intensified the requirement for reliable safety evaluation to uncover model vulnerabilit

An Iterative LangGraph Agent for Text-to-SQL: Natural Language Access to the Chicago Crime Database

arXiv:2609.22917v1 Announce Type: new Abstract: Non-technical stakeholders frequently cannot write the SQL needed to extract insights from operational databases. We built and evaluated a Text-to-SQL a

Are Coreset Selection Methods Worth Their Cost?

arXiv:2609.22894v1 Announce Type: cross Abstract: Coreset selection picks a representative subset of the labeled training set to make training cheaper. However, it is usually evaluated by downstream a

Automatic multimodal UX improvement recommendations from LLM agent user simulations

arXiv:2609.22971v1 Announce Type: new Abstract: Evaluating user experience (UX) on live websites through user testing is expensive, subjective, and difficult to scale. LLM agents offer a promising rou

agentsarxiv-cs-cl

DolphinBench: Mapping the Pareto Frontier of Agent Memory

arXiv:2609.24971v1 Announce Type: new Abstract: Agents today often take real-world actions that depend on long-term memory and context recall over time. However, most current memory benchmarks are bui