Search: “evaluation”

ALoDLM: Adaptively Looped Diffusion Language Models

arXiv:2610.04198v1 Announce Type: cross Abstract: Diffusion language models (DLMs) enable fast generation by predicting multiple tokens in parallel, but their practical adoption remains limited by a p

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Arm-wise Compositional Generalization in Dual-Arm Vision-Language-Action Models

DGX agent

arXiv:2610.06184v1 Announce Type: new Abstract: Generalization in multi-arm collaboration can be studied as composing familiar atomic skills in new ways across arms. However, existing evaluations offe

model-releasesarxiv-cs-ro

6 Oct 2026

Assembling Insights for Agentic Machine Learning Engineering Systems

arXiv:2610.04927v1 Announce Type: cross Abstract: Agentic machine learning engineering (MLE) is an emerging AI4SE application for complex ML tasks and a step toward recursive self-improvement of AI sy

BazaarBench: Delegation Safety in Decentralized C2C Marketplaces Run by LLM Agents

arXiv:2610.06748v1 Announce Type: cross Abstract: In decentralized consumer-to-consumer (C2C) marketplaces, people list goods, negotiate with strangers, and rate one another, so trust rests on reputat

Agents

Bounded Reasoning: Cognitive Hierarchy in Human-versus-AI Cyber Defense

arXiv:2610.04878v1 Announce Type: cross Abstract: Human-agent evaluations often compress interaction into a single performance score, even when human and automated policies adapt differently over time

agentsarxiv-cs-lg

Local Ai

Discovered, Not Designed: Population Evolution for Collaborative and Compute-Intensive Model Discovery

arXiv:2610.05950v1 Announce Type: new Abstract: LLM-driven evolution enables iterative model development, but two practical goals remain underexplored: finding model designs that transfer across relat

local-aiarxiv-cs-lg

Every View Counts: View-Consistent Panoptic Quality for Multi-view Panoptic Segmentation

arXiv:2610.05911v1 Announce Type: new Abstract: Multi-view panoptic segmentation assigns a semantic class and a scene-level instance ID to every pixel of an unordered set of images, and recent feed-fo

Evolving in Thought Space: Training a Small Model at Test Time Unlocks Better Discoveries

arXiv:2610.06269v1 Announce Type: cross Abstract: Open-ended scientific discovery often requires repeatedly proposing and evaluating candidate solutions. LLM-based systems can support this process by

safetyarxiv-cs-lg

Fine-Grained Caching for Diffusion Transformers with Few Calibration Conditions

arXiv:2512.05134v2 Announce Type: replace Abstract: Diffusion transformers require repeated denoiser evaluations, making image and video generation computationally expensive. We propose a training-fre

safetyarxiv-cs-cv

Global Communication or Graph-Specific Memory?

arXiv:2610.05874v1 Announce Type: new Abstract: Scalable Graph Transformers are commonly trained and evaluated on static large graphs in a transductive setup. Many scalable Graph Transformer component

Gradient-Free Sampling from Generative Models via Stochastic Bounded Extremum Seeking

arXiv:2610.04568v1 Announce Type: cross Abstract: We introduce a sampling approach for energy- and score-based generative models that requires no gradient evaluations of the model. Replacing the drift

Multi-Crop Leaf Disease Recognition: A Unified Benchmark and Cross-Region Study

arXiv:2610.04456v1 Announce Type: new Abstract: Deep learning models for crop leaf disease recognition routinely report near-perfect accuracy yet are typically trained and evaluated on a single datase

Priced Guidance: Can Language Models Generate Future Research Ideas?

arXiv:2610.04976v1 Announce Type: new Abstract: We evaluate language models' capability to generate novel research ideas through the lens of compression. We aim to lower-bound the potentially tiny pro

RMMBench: A Comprehensive Benchmark for Robotic Mobile Manipulation

model-releasesarxiv-cs-ro

arXiv:2610.05414v1 Announce Type: new Abstract: Although the advancement of vision-language models (VLMs) has endowed robots with enhanced environmental understanding and task reasoning, a comprehensi

Scenario-Based Compositional Statistical Model Checking for Safety Specifications

arXiv:2610.05571v1 Announce Type: cross Abstract: In safety-critical domains such as autonomous driving, systems must be evaluated across a large number of environment conditions, often represented as

safetyarxiv-cs-ro

SFlexRCA: Lightweight, Scalable, and Flexible Root Cause Analysis for IIoT Edge Systems

arXiv:2610.04893v1 Announce Type: new Abstract: Industrial Internet of Things (IIoT) systems generate high-dimensional sensor telemetry from interconnected components, where faults can propagate acros

SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs

arXiv:2610.06413v1 Announce Type: new Abstract: Thinking-enabled vision-language models (VLMs) report ever-higher accuracy on spatial benchmarks, yet final-answer scores cannot reveal whether a correc

ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring

arXiv:2610.06744v1 Announce Type: cross Abstract: ufakzeka-karar is an open Turkish decision model with 182,494,466 parameters. Given a Turkish text and questions of a fixed answer type (a choice, a l

Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos

arXiv:2610.04432v1 Announce Type: new Abstract: Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manu

What Will Post-Training Fix? Per-Problem Gains Are Shared Across Independent RL Runs, and Existing Checkpoints Predict Them Better Than A Priori Signals

arXiv:2610.04978v1 Announce Type: new Abstract: Data selection, curricula and the evaluation of post-training recipes all assume that we can tell, before training, which problems a model will improve

When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models

arXiv:2610.05550v1 Announce Type: new Abstract: The component that dominates a latent world model's prediction error need not be the one whose repair most improves action selection. We show this by co

A Benchmark for Spatially Grounded Gesture Generation

arXiv:2610.03105v1 Announce Type: new Abstract: Communication in shared space interweaves verbal and non-verbal signals, and pointing gestures anchor language to the environment: 'put the cup on that

A Controlled Audit of Personal AI Memory for Rating Prediction

arXiv:2610.02764v1 Announce Type: new Abstract: In structured rating prediction, does a personal AI use historical item-rating associations, or mainly the user's rating tendencies? We audit this disti

Can AI Understand the Language of Origami?

arXiv:2603.13856v3 Announce Type: replace-cross Abstract: Building AI systems that can plan, act, and create in the physical world requires more than pattern recognition. Such systems must reason abou

Capability Scaling-Down Laws for LLM Compression

arXiv:2610.02462v1 Announce Type: cross Abstract: LLM compression reduces inference costs and memory requirements, but selecting a method and configuration remains largely empirical because comparable

researcharxiv-cs-cl

Clinical Concept Centers in LLMs

arXiv:2610.02829v1 Announce Type: new Abstract: Large language models are increasingly used in clinical settings. However, research into the reliability and performance of these models has focused alm

researcharxiv-cs-cl

Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs

arXiv:2610.02657v1 Announce Type: new Abstract: Converting a pretrained autoregressive (AR) model to a diffusion language model (dLLM) enables parallel generation without pretraining a new model. Publ

ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution

arXiv:2610.02873v1 Announce Type: cross Abstract: The evolution of linguistic style in conversations is an underexplored issue in NLP. Most style-control datasets focus on sentences or assume a static

safetyarxiv-cs-ai

Cross-cohort TB classification using clinical data gathered in Uganda and South Africa

arXiv:2610.03256v1 Announce Type: new Abstract: We present a first evaluation of machine learning applied to patient clinical and demographic data gathered in two different countries for the purpose o

CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking

arXiv:2610.02460v1 Announce Type: new Abstract: Recent benchmarks rely on user simulators to evaluate AI agents in multi-turn interaction. While existing simulation techniques demonstrate surface fide

safetyarxiv-cs-cl

CVE2AP: Automated Generation of PDDL-Encoded Attack Paths via Large Language Models

arXiv:2610.03383v1 Announce Type: new Abstract: Attack Path (AP) modeling is fundamental to cybersecurity analysis, where the Planning Domain Definition Language (PDDL) has been widely adopted to enco

DeltaWorld: Physically Consistent Interactive World Simulators via Action-Conditioned Latent Increment Learning

model-releasesarxiv-cs-ro

arXiv:2610.02691v1 Announce Type: new Abstract: Interactive world simulators can provide scalable environments for robot planning, policy training, and evaluation by predicting action consequences whi

EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents

arXiv:2610.03153v1 Announce Type: cross Abstract: Workspace agents combine large language models with execution harnesses to perform stateful, multi-step tasks that access or modify external resources

MiDShip: Multimodal Dataset of Ship Cargo Hold Structures for Engineering Design

arXiv:2610.02214v1 Announce Type: cross Abstract: Ship structures govern vessel strength, safety, and manufacturability, but their design must satisfy hundreds of classification society requirements,

OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

model-releasesarxiv-cs-cl

arXiv:2610.02986v1 Announce Type: new Abstract: Membership inference on large language models (LLMs) aims to determine whether a given text sample was included in an LLM's training data, without acces

Output Language Confusion under Multilingual Prompt Contamination

model-releasesarxiv-cs-cl

arXiv:2610.02926v1 Announce Type: new Abstract: Standard factual benchmarks assume clean monolingual prompts and exact-match scoring, two assumptions that break simultaneously in real-world multilingu

Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces

arXiv:2610.02580v1 Announce Type: new Abstract: Physical AI Smart Spaces is, to the best of our knowledge, the first benchmark to simultaneously provide large-scale, multi-class, and multi-camera 3D p

PowerBench: Measuring Language Model Bias in Power-shifting Requests

arXiv:2610.02303v1 Announce Type: new Abstract: Language models increasingly assist people with power-related requests, so systematic differences in whom they help could shift the distribution of powe

safetyarxiv-cs-lg

Rank-Aware Speculative Sampling for Diffusion Draft Trees

arXiv:2610.02251v1 Announce Type: new Abstract: Speculative sampling accelerates diffusion generation by verifying inexpensive draft states in parallel while preserving the target law. Recent tree-bas

StanceEval 2026: The Second Stance Detection Shared Task

arXiv:2610.03215v1 Announce Type: new Abstract: StanceEval 2026 is the second edition of the StanceEval shared task series on stance detection in Arabic social media text. Stance detection aims to ide

researcharxiv-cs-cl

TasteBench: Multimodal Benchmark for Sensory Prediction, from Molecules to Sustainable Foods

arXiv:2610.02599v1 Announce Type: new Abstract: Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory, that accelerate drug an

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

arXiv:2610.03585v1 Announce Type: cross Abstract: Security benchmarks for LLM-based agents often report the attack success rate (ASR) as a measure of model robustness and use these scores to compare d

World Embedding Benchmark

model-releasesarxiv-cs-cl

arXiv:2610.03632v1 Announce Type: cross Abstract: Physical fidelity has received increasing attention in world models and video generation, yet how video representations encode physical information re

Local Ai

Writerslogic at PAN 2026: Process over Content for Robust Detection under Domain Shift

arXiv:2610.03565v1 Announce Type: new Abstract: We describe the Writerslogic systems for three PAN at CLEF 2026 shared tasks (Reasoning Trajectory Detection, Voight-Kampff Generative AI Detection, and

local-aiarxiv-cs-cl

Benchmarking Generative Models for Weather Data Assimilation on Real Station Observations

arXiv:2610.00728v1 Announce Type: cross Abstract: Weather reanalysis products rely on computationally intensive numerical weather predictions followed by data assimilation that corrects the forecast t

CAGE-NAS: Certified Functional Descent for Efficient Model Growth

arXiv:2610.01173v1 Announce Type: new Abstract: The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be ex

CHORDONOMICON: A Dataset of 666,000 Songs and their Chord Progressions

arXiv:2410.22046v4 Announce Type: replace-cross Abstract: Chord progressions encapsulate important information about music, pertaining to its structure and conveyed emotions. They serve as the backbon

Claude’s hillclimb loop for AI agents: start with production traces

Anthropic's Claude hillclimb loop designs evals and improves agents one change at a time. Here's how to run that loop against production traces in Arize AX. The post Claude’s hillclimb loop for AI age

model-releasesarize-ai