Search: “evaluation”

Learning to Sell: Reinforcement Learning for Strategic Large Language Model Agents in Multi-Product Markets

arXiv:2609.33289v2 Announce Type: replace Abstract: Autonomous large language model (LLM) agents operating in multi-product markets must make sequential decisions under information asymmetry and resou

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

LESS: Lightweight Evolutionary Supernet Search in Minutes

DGX agent

arXiv:2610.01468v1 Announce Type: cross Abstract: Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of ca

hardwarearxiv-cs-lg

2 Oct 2026

M^2Weather: A Benchmark for Joint Multi-Station and Multi-Variable Weather Forecasting

arXiv:2610.00370v1 Announce Type: new Abstract: Station weather forecasting is fundamentally shaped by both complex spatial dependencies across stations and strong physical coupling among weather vari

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop

model-releasesarxiv-cs-cv

arXiv:2610.00559v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have shown strong multimodal reasoning capabilities, yet whether they truly capture the physical consistency underlying re

SALD: Self-Referenced Advantage Learning for Diffusion Models

arXiv:2610.01496v1 Announce Type: new Abstract: Recent work on language-model adaptation has shown that single models can obtain informative training signals by evaluating their behavior in demonstrat

researcharxiv-cs-cv

TabJoinBench: A Benchmark for Joinable Table Discovery

arXiv:2610.00817v1 Announce Type: cross Abstract: Join discovery aims to identify tables from large data repositories that can augment a query table with complementary information, enabling downstream

Beyond Model Ranking: Regime Diagnosis for Distributional-Statistical Misspecification in Industrial Time-Series Forecasting

arXiv:2609.40117v1 Announce Type: new Abstract: Time-series forecasting models achieve strong benchmark performance but exhibit severe systematic bias in industrial deployments. This train--deploy gap

cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents

arXiv:2609.40284v1 Announce Type: cross Abstract: Computer use agents (CUAs), which use graphical user interfaces (GUIs) to complete tasks on a computer, have recently surpassed human performance on m

Disagreement-Regularized Imitation Learning for Image-Based Continuous Control with Gaussian and Beta Policies

arXiv:2609.38407v1 Announce Type: new Abstract: Purpose: Behavior cloning can accumulate errors when a learned controller visits states outside the demonstrated distribution. This study evaluates whet

safetyarxiv-cs-lg

Evidence First, Arithmetic Second: A System Report and Failure Analysis for DocSem

arXiv:2609.39013v1 Announce Type: new Abstract: EVICALC, our system for the DocSem shared task, achieved 8.61% joint accuracy on 1,730 tasks in the official final test evaluation. It reads a PDF, sele

researcharxiv-cs-cl

Tutorials

From Search to Signal: Online Post-Training in Automatic Heuristic Design

arXiv:2609.39383v1 Announce Type: cross Abstract: Large language model (LLM)-based automatic heuristic design (AHD) iteratively proposes and refines heuristics, pairing design rationales with executab

tutorialsarxiv-cs-ai

MADBench: Benchmarking the Security of Multi-Agent Debate

arXiv:2609.39146v1 Announce Type: new Abstract: Multi-agent debate (MAD) can improve large language model (LLM) reasoning by allowing multiple agents to exchange and critique their answers to the same

MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting

arXiv:2609.38523v1 Announce Type: cross Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle com

OpenCollab: A Multi-Agent Coding Framework with Programmable Collaboration and Controllable Runtime

arXiv:2609.38345v1 Announce Type: cross Abstract: Multi-agent coding systems are designed to tackle complex software engineering tasks through collaboration. However, existing evaluations typically as

QuantCode Model: Specializing Language Models for Executable Algorithmic Trading Code

model-releasesarxiv-cs-cl

arXiv:2609.39420v1 Announce Type: new Abstract: Large language models are strong general-purpose code generators, but executable algorithmic trading remains a demanding specialization target: a model

RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection

arXiv:2609.39229v1 Announce Type: cross Abstract: Automatic evaluation of faithfulness increasingly relies on a large language model acting as a judge, yet the most reliable judges are proprietary fro

RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models

arXiv:2609.39551v1 Announce Type: new Abstract: Auto-research agents, LLM systems that propose, implement, train, and evaluate model changes across iterations, promise to automate applied ML's experim

Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Models

arXiv:2609.38491v1 Announce Type: new Abstract: Clinical research in psychiatry increasingly relies on large scale collection of spoken language data to identify acoustic and linguistic biomarkers. Ye

Search Shapes Conclusions: Auditing Evidence Selection Bias in Deep Research Agents

arXiv:2609.39026v1 Announce Type: new Abstract: Deep Research agents synthesize evidence into cited reports, yet a well-cited report can still reach a misleading conclusion. Citation correctness check

SETA: Scaling Environments for Terminal Agents

arXiv:2607.10891v2 Announce Type: replace Abstract: Large language models (LLMs) are rapidly shifting toward agents that solve tasks through diverse interfaces, including web and graphical user interf

SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling

arXiv:2609.38397v1 Announce Type: new Abstract: Virtual clients offer a cost-effective approach to support applications such as A/B testing, recommender system development, and interface evaluation. H

TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories

arXiv:2609.38353v1 Announce Type: cross Abstract: Long-term memory lets LLM agents recall past interactions and remain consistent across sessions, but memory systems are hard to compare because they o

ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing

arXiv:2609.40356v1 Announce Type: cross Abstract: Recent video generation is increasingly realistic and controllable, yet video editing remains less developed, particularly for precise local edits tha

AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations

arXiv:2609.36915v1 Announce Type: cross Abstract: Aerial manipulators extend robotic manipulation into 3D workspaces that are difficult for ground-based robots to access, creating new opportunities fo

Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable

arXiv:2609.37616v1 Announce Type: cross Abstract: Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on t

safetyarxiv-cs-cl

CAD-Native Transformer Operators for AI-Aided Engineering

arXiv:2609.36806v1 Announce Type: new Abstract: Modern engineering systems, from automobiles to aircraft, are designed by using precise, continuous parametric computer-aided design (CAD) models. Evalu

researcharxiv-cs-ai

Cloud CISO Perspectives: How cybersecurity startups can win CISOs

model-releasesgoogle-cloud-ai

Welcome to the second Cloud CISO Perspectives for September 2026. Today, Alicja Cade and Nick Godfrey, senior directors, Office of the CISO, share their guidance for cybersecurity startups on how to w

CruxBench: A Benchmark of Information Discovery

arXiv:2609.35879v1 Announce Type: cross Abstract: Benchmarks for large language models (LLMs) typically evaluate the accuracy of answers against fixed reference labels. But a central step in many comp

Tutorials

Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it

arXiv:2609.36612v1 Announce Type: new Abstract: Unlearning in large language models (LLMs) is typically evaluated at the output level, where a model appears to suppress sensitive or undesirable conten

tutorialsarxiv-cs-lg

Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym

arXiv:2609.37267v1 Announce Type: new Abstract: Proactive LLM agents can turn idle compute into useful support before users ask. Yet even correct work can misread user context, impose review costs, or

From Judgment Quality to Downstream Utility: Rethinking LLM-as-a-Judge for Open-Ended Tasks

arXiv:2609.37145v1 Announce Type: new Abstract: LLM-as-a-Judge is increasingly used to evaluate policy responses on open-ended tasks that lack ground-truth answers. Existing work often directly conver

safetyarxiv-cs-ai

LEGO-Anything: Coding Agents for 3D Scene Reconstruction

model-releasesarxiv-cs-cv

arXiv:2609.36380v1 Announce Type: new Abstract: A 3D scene reconstructed from a single image is most useful when represented not as a rendering or a fixed 3D output, but as an explicit scene program w

MARCO: Multi-Round Agentic Reinforcement for Conditional Molecular Optimization

model-releasesarxiv-cs-cl

arXiv:2609.36683v1 Announce Type: cross Abstract: Molecular optimization is inherently iterative: a candidate is proposed, evaluated against several objectives, and revised while preserving a relation

MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

arXiv:2508.14925v2 Announce Type: replace-cross Abstract: By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a c

Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark

arXiv:2609.36039v1 Announce Type: cross Abstract: Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing st

OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells

arXiv:2609.37773v1 Announce Type: new Abstract: Artificial Intelligence Virtual Cells (AIVCs) are envisioned as scientific agents that simulate cellular responses, explain underlying mechanisms, and s

Retrieve, Reproduce, Reveal: Dissecting Retrieval-Augmented Software Vulnerability Detection

arXiv:2609.37669v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) is increasingly used to enhance Large Language Model (LLM)-based software vulnerability detection by grounding pr

Sharp Convergence and Sampling Trade-offs for Riemannian Diffusion under Nonnegative Ricci Curvature

arXiv:2609.36568v1 Announce Type: new Abstract: Diffusion models have emerged as state-of-the-art generative models, with recent extensions from Euclidean spaces to Riemannian manifolds. However, exis

researcharxiv-cs-lg

VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics

arXiv:2609.37287v1 Announce Type: cross Abstract: Image translation is a fundamental capability of multimodal models for multilingual applications, requiring visual understanding and meaning preservat

Vulnerability Discovery and Exploitation Trends in the AI Era

model-releasesgoogle-cloud-ai

Written by: Robin Grunewald, Supriya Mazumdar, Kelli Vanderlee Introduction Google Threat Intelligence Group (GTIG) examines vulnerability disclosure and exploitation statistics to evaluate the impact

🚨 Yesterday, the 'White House Accord on Super Intelligence' was signed. What was supposed to be about AI safety and accountability was inst…

🚨 Yesterday, the 'White House Accord on Super Intelligence' was signed. What was supposed to be about AI safety and accountability was instead a PROBLEMATIC endorsement of self-regulation. My comments

safetygary-marcus--x

A Multi-Dataset Benchmark of YOLO-Based Weed Detection in Precision Agriculture

model-releasesarxiv-cs-cv

arXiv:2609.33991v1 Announce Type: new Abstract: Weed detection is an important component of precision agriculture, enabling site-specific weed management and reducing unnecessary herbicide use. Althou

AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modeling

arXiv:2605.05586v2 Announce Type: replace Abstract: High-fidelity CFD is essential for aerodynamic design, but repeated simulations are computationally expensive, motivating surrogate models for rapid

researcharxiv-cs-lg

AgentWare: Automating the Lifecycle of Agentic Applications across the Edge-to-Cloud Continuum

arXiv:2609.34586v1 Announce Type: cross Abstract: Deploying LLM-enabled agentic applications across the Edge-to-Cloud continuum remains challenging due to hardware heterogeneity, deployment complexity

agentsarxiv-cs-lg

Applications

Federated Computation of ROC and PR Curves

arXiv:2510.04979v2 Announce Type: replace Abstract: Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are fundamental analytical tools for evaluating classification models, prov

applicationsarxiv-cs-lg

ForVis: An In-Field Dataset and Benchmark for VIO Using Under-Canopy UAV Flights in Forests

model-releasesarxiv-cs-ro

arXiv:2609.35482v1 Announce Type: new Abstract: Visual-inertial Simultaneous Localization and Mapping (VI-SLAM) for UAVs remains difficult to evaluate in real forest environments, where motion, illumi

GeoShrink: Accelerating Diffusion Transformers with Two Lines of Code

arXiv:2609.33723v1 Announce Type: new Abstract: Diffusion transformers incur substantial inference cost through repeated model evaluations along a sampling trajectory. We introduce GeoShrink, a traini

researcharxiv-cs-cv

MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models

arXiv:2609.33893v1 Announce Type: new Abstract: Large audio-language models (LALMs) produce fluent responses about audio but often hallucinate by making plausible yet ungrounded claims. Existing audio