Search: “evaluation”

KiToke: Distribution-Aware Token Compression for Video Large Language Models

arXiv:2604.03414v2 Announce Type: replace Abstract: Video Large Language Models (Video LLMs) achieve strong performance on video understanding tasks but suffer from high inference costs due to the lar

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

LED-BENCH: A Benchmark for Large-format Engineering Drawing Understanding

DGX agent

arXiv:2610.19571v1 Announce Type: new Abstract: Real-world engineering drawings are often large-format and information-dense. Answering practical engineering questions therefore requires both fine-gra

model-releasesarxiv-cs-cv

15 Oct 2026

Lost in Implicit Routing: How LLM Agents Fail to Load Relevant Skills at Scale

arXiv:2610.18522v1 Announce Type: new Abstract: Reusable skills let LLM agents load task-specific instructions on demand, but they make execution depend on implicit routing over skill metadata. We tes

researcharxiv-cs-ai

Manaca-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge

arXiv:2610.20570v1 Announce Type: new Abstract: Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or valida

Agents

Measuring and Reducing Unusable Replies of LLM Agents

arXiv:2610.19659v1 Announce Type: new Abstract: People delegate multi-step work to LLM agents and read only the reply that the agent hands back. Current evaluation scores task completion and the faith

agentsarxiv-cs-cl

MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue

arXiv:2610.20172v1 Announce Type: cross Abstract: Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel w

Applications

Nudge the Judge: Uncovering Program Decision Hijacking in Real-World Jev Applications

arXiv:2610.18612v1 Announce Type: cross Abstract: System One models such as Jev change how software makes decisions by mapping open-ended semantic contexts into structured judgments that directly driv

applicationsarxiv-cs-ai

PRIMEdit: Probability Redistribution for Instance-Aware Multi-Object Video Editing With a New Benchmark Dataset

arXiv:2412.12877v3 Announce Type: replace Abstract: Recent Artificial Intelligence (AI)-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the

Provable Transformer Context Compression for In-Context Reinforcement Learning

arXiv:2610.20004v1 Announce Type: new Abstract: Transformers can perform in-context reinforcement learning without parameter updates. However, existing theoretical constructions often require context

Query Your Way Out: Benchmarking Multi-Turn Knowledge Reasoning through Generalized Puzzle Solving

arXiv:2610.16133v1 Announce Type: cross Abstract: Puzzles offer a natural testbed for Large Language Models (LLMs) reasoning because they pose verifiable goals under partial information and require mu

Refine Actions If Needed: Adaptive Action Chunk Refinement for Robot Manipulation

arXiv:2610.19765v1 Announce Type: new Abstract: Computationally lightweight manipulation policies predict executable actions with low inference latency, but prediction errors can limit task success. I

safetyarxiv-cs-ro

ROAR: Unifying Runs across Heterogeneous AI-Driven Research Systems

arXiv:2610.03966v2 Announce Type: replace Abstract: Each run of an AI-driven research system (ADRS) is an expensive search over a vast solution space, and dependable evaluation requires many runs, mak

Single-Sample Decoding of a Two-Layer Sound-Word Canvas Model for Thai Klon Pad

arXiv:2610.14980v1 Announce Type: new Abstract: We present a generator of Thai klon pad verse, one stanza from a short topic, and evaluate it against human verse on held-out topics. The generator is a

safetyarxiv-cs-ai

Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

arXiv:2610.15141v1 Announce Type: new Abstract: Single-step retrosynthesis is a one-to-many inverse problem, yet most models are trained and evaluated against a single recorded reactant set. Reference

safetyarxiv-cs-ai

Stochastic CHAOS: Why Deterministic Inference Kills, and Distributional Variability Is the Heartbeat of Artifical Cognition

arXiv:2601.07239v2 Announce Type: replace Abstract: Deterministic inference is a comforting ideal in classical software: the same program on the same input should always produce the same output. As la

safetyarxiv-cs-ai

The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understanding

arXiv:2610.19102v1 Announce Type: new Abstract: Aerial and remote sensing video is increasingly evaluated like web video, by posing multiple-choice questions to multimodal large language models (MLLMs

Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

arXiv:2610.19108v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitat

Truthful and Expected-Payment-Optimal Model Routing in Graph-Based Agentic Workflows

arXiv:2610.19316v1 Announce Type: cross Abstract: Assigning routine tasks to cheaper large language model (LLM) providers can reduce generation cost, but switching providers may require processing acc

VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?

arXiv:2605.06068v2 Announce Type: replace Abstract: For years, we have built LLM serving systems like any other critical infrastructure, with a single general-purpose stack hand-tuned over many engine

When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents

arXiv:2610.19910v1 Announce Type: new Abstract: Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rul

Who's Behind the Text? Robust Model Authorship Attribution

arXiv:2610.19583v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly used to generate text, their misuse has become a growing societal problem, with implications for acad

XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity

arXiv:2605.05662v2 Announce Type: replace-cross Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. More

A Persian Vision-Language-Action Dataset and Benchmark for Ambiguity- and Infeasibility-Aware Instruction-to-Plan Grounding

arXiv:2610.15666v1 Announce Type: new Abstract: Converting natural-language instructions into executable action plans conditioned on a scene image is a core capability for embodied human-robot interac

Agents

Adversarial Robustness of Camera-Radar Sensor Fusion for Autonomous Driving

arXiv:2610.16012v1 Announce Type: cross Abstract: Autonomous driving (AD) relies on multi-sensor perception to interpret the surrounding environment. However, the robustness of camera--radar fusion un

agentsarxiv-cs-cv

Applications

AIRRI: Towards a Human-Readable, Machine-Resistant Reading Interface

arXiv:2610.18724v1 Announce Type: new Abstract: The rapid advancement of image-to-text extraction systems and large language models (LLMs) poses new challenges for educational assessment by making ass

applicationsarxiv-cs-cv

All-electron machine-learned orbital-free density functional for periodic CHNO systems

arXiv:2610.17975v1 Announce Type: cross Abstract: We introduce STRUCTURES-P, a machine-learned all-electron orbital-free density functional for periodic CHNO systems. Electrostatics are evaluated anal

researcharxiv-cs-lg

Agents

AlphaPilot: Multi-Turn Tool-Using LLM Agents for Adaptive Alpha Mining with Reinforcement Learning

arXiv:2610.15488v1 Announce Type: cross Abstract: Formulaic alpha mining seeks interpretable expressions that rank assets by future returns. The task is challenging because it demands simultaneous pro

agentsarxiv-cs-lg

Bayesian Optimization under Uncertainty for Training a Scale Parameter in Stochastic Models

arXiv:2510.06439v2 Announce Type: replace Abstract: Hyperparameter tuning is a challenging problem especially when the system itself involves uncertainty. Due to noisy function evaluations, optimizati

ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning

arXiv:2610.17183v1 Announce Type: cross Abstract: Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an

EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactions

arXiv:2610.18499v1 Announce Type: new Abstract: Egocentric video understanding spans diverse capabilities, modalities, and evaluation formats, yet these are often studied in isolation despite their ti

EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos

arXiv:2610.17128v1 Announce Type: new Abstract: Egocentric video provides a continuous first-person record of everyday activities, making it a natural foundation for wearable AI assistants. Unlike off

ForeTac: Predictive Contact Guidance for Generative Robot Policies

arXiv:2610.18118v1 Announce Type: new Abstract: Visually similar actions can produce contact loss, jamming, slippage, or irreversible damage, while tactile feedback may arrive after a critical transit

safetyarxiv-cs-ro

From Classification to Cross-Modal Understanding: Leveraging Vision-Language Models for Fine-Grained Renal Pathology

arXiv:2511.11984v2 Announce Type: replace Abstract: Fine-grained glomerular subtyping is essential for kidney biopsy interpretation, yet it presents a harder setting than the broad organ- or disease-l

Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?

arXiv:2610.18834v1 Announce Type: new Abstract: Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state c

Information-based Unified Design Framework for NMR Magnetic Tracking System

arXiv:2610.17873v1 Announce Type: new Abstract: Nuclear magnetic resonance (NMR) magnetic tracking offers a promising alternative to magnetic resonance imaging (MRI)-based localization for medical rob

researcharxiv-cs-ro

Interaction-as-a-Judge

arXiv:2610.17895v1 Announce Type: new Abstract: Coding agents are tasked with producing sophisticated interactive applications such as websites, desktop applications, and games. While humans assess th

safetyarxiv-cs-lg

LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration

arXiv:2610.14612v1 Announce Type: new Abstract: Chiplet technology is a key approach for extending Moore's Law. However, its design process is constrained by two primary challenges: the scarcity of ea

safetyarxiv-cs-lg

MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL

arXiv:2610.14883v1 Announce Type: cross Abstract: We present MARL-BattleGrounds (MARL-BGs), an easy-to-use JAX-native benchmark for heterogeneous and competitive multi-agent reinforcement learning (MA

Poisoned into Action: Context Attacks on Typed Decision Workflows

arXiv:2610.15537v1 Announce Type: cross Abstract: Typed decision interfaces restrict models to predefined actions, yet schema-valid choices can still trigger policy-violating state changes. Evaluating

safetyarxiv-cs-lg

PsyBer-Agent: A Psychology-Driven User Agent for Recommender Systems

arXiv:2610.13989v1 Announce Type: cross Abstract: Evaluating recommender systems before deployment requires simulators that remain realistic over long, closed-loop interaction cycles. Current LLM-base

Refined Grid Sampling for Efficient Implicit Neural Representations of Large-Scale Scientific Data

arXiv:2610.14205v1 Announce Type: new Abstract: Implicit neural representations (INRs) offer a powerful framework for modeling high-dimensional physical fields, but their training remains inefficient

researcharxiv-cs-lg

Safety-Oriented Trajectory Ranking from BEV Perception Outputs

arXiv:2610.14066v1 Announce Type: cross Abstract: Bird's-Eye-View (BEV) representations are increasingly used as compact inputs for autonomous driving perception and planning. However, lightweight and

safetyarxiv-cs-cv

scHFBiMamba: a bidirectional Mamba autoencoder for cardiac disease-state classification from single-nucleus RNA-seq data

arXiv:2610.15117v1 Announce Type: new Abstract: To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-n

TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery

arXiv:2610.14626v1 Announce Type: new Abstract: Mixed-variable optimization in scientific discovery often requires exploring large combinatorial spaces under costly experimental evaluations, making sa

researcharxiv-cs-lg

TRIG: Toward Measuring Texture Realism in Image Generation

arXiv:2610.15164v1 Announce Type: new Abstract: This paper studies the backbone used by metrics for evaluating the quality of generated images. Although generative models have advanced rapidly, some o

safetyarxiv-cs-cv

Unsupervised Data Drift Monitoring for Deployed Deepfake Speech Detectors

arXiv:2610.16826v1 Announce Type: cross Abstract: Deepfake speech detectors are typically evaluated as binary classifiers on fixed benchmarks, but deployed systems face evolving attacks, codecs, speak

researcharxiv-cs-cv

Tutorials

Use Of Convolutional Neural Networks In Detection Of Branch Dieback Disease In Pequi Trees

arXiv:2610.15049v1 Announce Type: new Abstract: This study aims to evaluate and compare convolutional neural network (CNN) architectures, specifically AlexNet, Inception, LeNet, VGG16, and MobileNet,

tutorialsarxiv-cs-cv

Tutorials

What DNA Foundation Models Learn Beyond Sequence Composition

arXiv:2610.15711v1 Announce Type: new Abstract: Do genomic foundation models (FMs) provide predictive information beyond simple sequence composition? We evaluate frozen representations from five DNA F

tutorialsarxiv-cs-lg