Search: “evaluation”

arXiv:2609.07925v2 Announce Type: new Abstract: We present FrogNano, a 4B coding agent designed to tackle software engineering (SWE) tasks efficiently and effectively, even under resource-constrained

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

From Simulated Citizens to Simulated Deliberation: Challenges in Representation and Interaction

DGX agent

arXiv:2609.07573v1 Announce Type: new Abstract: Multi-agent LLM deliberation has been explored as a scalable way to simulate public deliberation. For such simulations to be informative, persona agents

safetyarxiv-cs-ai

10 Sept 2026

Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction

arXiv:2602.24080v3 Announce Type: replace Abstract: The pursuit of human-like conversational agents has long been guided by the Turing test. For modern speech-to-speech (S2S) systems, a critical yet u

IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications

model-releasesarxiv-cs-cl

arXiv:2609.10539v1 Announce Type: new Abstract: A research idea may be novel, coherent, and scientifically plausible, yet its proposed method may remain insufficiently specified for faithful implement

Knowing Your Uncertainty -- On the application of LLM in social sciences

arXiv:2512.05461v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are rapidly being integrated into computational social science research, yet their blackboxed training and design

Limits of Reliability and Scaling in Language Models

arXiv:2607.14112v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are trained and evaluated as though perfect reliability is achievable for any task given sufficient scale. We sho

LiteRAG: Cost-Efficient Graph-Based Retrieval-Augmented Generation

model-releasesarxiv-cs-cl

arXiv:2609.10239v1 Announce Type: cross Abstract: Graph-based retrieval can improve multi-hop question answering, but existing approaches often incur high query-time costs and produce diffuse, oversiz

Mini-Batch Risk-Averse Deep Q-Learning: A Robot Navigation Case Study

arXiv:2609.07998v1 Announce Type: new Abstract: We study the control of Markov decision processes in which the quality of a policy is evaluated by a dynamic, time-consistent Markov risk measure rather

safetyarxiv-cs-ai

PGP-Clinical-TimeKAN: Prior-Guided Joint Probabilistic Forecasting of Clinical Trajectories

arXiv:2609.05488v1 Announce Type: new Abstract: Clinical deterioration unfolds through coupled, partially observed trajectories, not a single diagnostic label. We introduce PGP-Clinical-TimeKAN, a tra

Programmable World Model

arXiv:2609.10540v1 Announce Type: new Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent wo

SCCM : Stream Cruise Control Method for Automated Drift Detection and Adaptation

arXiv:2609.09432v1 Announce Type: new Abstract: Real-world datasets often exhibit evolving distributions, known as concept drift. Ignoring drift degrades predictive performance, while reliance on fixe

local-aiarxiv-cs-lg

Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modeling

arXiv:2609.06835v1 Announce Type: cross Abstract: Agentic AI systems execute complex tasks through long-horizon workflows of planning, tool use, and multi-agent coordination. Task failures in these sy

local-aiarxiv-cs-ai

StreamAlign: Streaming Text-Aligned Speech Tokenization

arXiv:2609.09719v1 Announce Type: new Abstract: Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretra

safetyarxiv-cs-cl

Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts

model-releasesarxiv-cs-ai

arXiv:2609.08236v1 Announce Type: new Abstract: Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers

TreeThink: A Modular Tree Search Library for Mathematical Reasoning with LLMs

model-releasesarxiv-cs-cl

arXiv:2607.11258v2 Announce Type: replace Abstract: Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily targ

Understanding the Impact of Model Pruning on Long-Tail Forgetting and Explanation Reliability in Medical Imaging

arXiv:2609.07803v1 Announce Type: new Abstract: Model pruning is widely used to compress deep neural networks, reducing memory and computational requirements with minimal impact on aggregate performan

Who Laughs with Whom? Disentangling Influential Factors in Humor Preferences across User Clusters and LLMs

arXiv:2601.03103v2 Announce Type: replace-cross Abstract: Humor preferences vary widely across individuals and cultures, complicating the evaluation of humor using large language models (LLMs). In thi

WolfSociety: Understanding Collective Risk from Harmful-Agent Scaling in Financial Agent Societies

arXiv:2609.05591v1 Announce Type: cross Abstract: Safety evaluations typically focus on individual agents, but interacting agents can spread harmful information and influence the environment in which

safetyarxiv-cs-ai

xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems

model-releasesarxiv-cs-ai

arXiv:2609.07784v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for everyday assistance, yet existing benchmarks only partially reflect the requests users naturally

Your Agent Says Yes: Interpreting Adversarial Market Behavior Beyond Individual Transactions

arXiv:2609.07675v1 Announce Type: cross Abstract: Transaction-local controls answer whether one financial request may proceed, but market behavior can be distributed across messages, agents, assets, a

local-aiarxiv-cs-ai

ActionSplice: In-Flight Action Editing for Interactive World Models

arXiv:2609.08230v1 Announce Type: new Abstract: Chunk-autoregressive video world models typically condition each generated chunk on one action. An action received during sampling must therefore wait f

researcharxiv-cs-cv

AhaBench: Do Agents Learn from Prior Experience? A Benchmark for Long-Horizon Continual Learning

arXiv:2609.05435v1 Announce Type: new Abstract: Modern language agents are expected to operate over long horizons: they ask follow-up questions, reuse worked examples, handle tool feedback, and adapt

Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning

arXiv:2512.23515v2 Announce Type: replace-cross Abstract: Signal decay and regime shifts pose recurring challenges for data-driven investment strategies in non-stationary markets, where conventional t

safetyarxiv-cs-lg

Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation

arXiv:2609.06433v1 Announce Type: new Abstract: The generation of safety-critical traffic scenarios is essential for training and evaluating autonomous vehicles. Prior approaches typically perturb the

safetyarxiv-cs-ro

Tutorials

Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI

arXiv:2609.06807v1 Announce Type: new Abstract: In this work, we comprehensively evaluate three popular feature-extraction paradigms in AI-based neuroimaging modeling: (1) computation of anatomical su

tutorialsarxiv-cs-cv

CoRL: Co-Evolutionary Reinforcement Learning for Adaptive Indirect Prompt-Injection Attacks and Defenses

arXiv:2609.07529v1 Announce Type: new Abstract: Tool-augmented language agents are vulnerable to indirect prompt injection (IPI). Unlike direct prompt injection, IPI hides adversarial instructions in

Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions

arXiv:2609.06140v1 Announce Type: cross Abstract: Agents can turn shared infrastructure into a channel for coordinated intrusion. The Hugging Face incident and a separate public-wiki investigation sho

safetyarxiv-cs-lg

Damage-Aware Bandit Pruning for Vision and Language Transformers

arXiv:2609.05448v1 Announce Type: cross Abstract: Structured post-training pruning of transformers requires selecting complete functional units whose suppression causes limited degradation. We formula

DART: Distributional Adversarial Recurrent Training for Algorithm Learning

arXiv:2609.05988v1 Announce Type: new Abstract: Recurrent reasoning models (RRMs) can solve structured problems, achieving easy-to-hard generalization through iterative computation in hidden space. Th

local-aiarxiv-cs-lg

Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions

arXiv:2406.08697v4 Announce Type: replace-cross Abstract: Offline reinforcement learning enables evaluation and optimization of sequential decisions from historical data, when it is not possible to de

safetyarxiv-cs-lg

Tutorials

Decomposing LLM-Judge Uncertainty to Target Expert Labels

arXiv:2609.06444v1 Announce Type: cross Abstract: An LLM judge evaluates outputs at scale. Experts should label only where it is least sure. Its natural escalation signal conflates two uncertainties:

tutorialsarxiv-cs-lg

Deep Learning Approach to Bearing and Induction Motor Fault Diagnosis via Data Fusion

arXiv:2506.11032v2 Announce Type: replace Abstract: Convolutional Neural Networks (CNNs) are used to evaluate accelerometer and microphone data for bearing and induction motor diagnosis. A Long Short-

researcharxiv-cs-lg

DF26: We Cannot Tell Fake From Real Anymore

arXiv:2609.07369v1 Announce Type: new Abstract: We introduce DF26, a novel benchmark for detecting AI-generated videos containing fully synthetic clips produced by recent text-to-video and image-to-vi

Disparity Has a Sign: Stereo Matching Beyond the Zero-Disparity Plane

arXiv:2609.06809v1 Announce Type: new Abstract: Modern stereo matching models fail when disparity crosses zero, with end-point error (EPE) rising by 4.6-37imes. Yet stereoscopic content, from cinema 3

Dynamic-Programming-Guided Hierarchical BPE and Empirical Analysis of Vocabulary Pruning

arXiv:2609.06898v1 Announce Type: cross Abstract: Byte Pair Encoding (BPE) constructs vocabularies through greedy pair merging, but the resulting merge order does not necessarily allocate a fixed mode

researcharxiv-cs-lg

GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning

model-releasesarxiv-cs-ro

arXiv:2609.05927v1 Announce Type: new Abstract: Robot manipulation foundation models require scalable evaluation and data generation across diverse scenarios, with simulation providing an environment

GRAN-TED: Generating Robust, Aligned, and Nuanced Text Embedding for Diffusion Models

arXiv:2512.15560v3 Announce Type: replace Abstract: The text encoder is a critical component of text-to-image and text-to-video diffusion models, fundamentally determining the semantic fidelity of the

Harnessing CLIP and DINO: An Uncertainty-Aware Cascaded Fusion Network for Generalizable Deepfake Image Detection

arXiv:2609.07670v1 Announce Type: new Abstract: The growing realism and accessibility of manipulated and generated faces threaten the trustworthiness of digital media. To detect such forgeries, deepfa

Human-Centric Image Captioning with Subject-Centered Spatial Understanding

arXiv:2609.08300v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) achieve remarkable performance on generic image captioning, they frequently suffer from structural halluc

Image-Scale Robustness and Visual Recognition Performance: A Cross-Architecture Analysis

arXiv:2609.06051v1 Announce Type: new Abstract: The sensitivity of visual recognition models to changes in image scale is well established, yet the factors governing this sensitivity across heterogene

Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics

arXiv:2609.08755v1 Announce Type: new Abstract: Many emerging video language modeling tasks require systems to move beyond clip-level abstraction and model visual content as it unfolds over extended t

safetyarxiv-cs-cv

LANTERN: A Closed-Loop Benchmark for VLM-Based Cooperative Driving with Temporally Grounded Warnings

model-releasesarxiv-cs-ro

arXiv:2609.06368v1 Announce Type: new Abstract: We present LANTERN, a closed-loop benchmark for temporally grounded cooperative warnings. LANTERN separates warning onset, hazard onset, warning termina

LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text

arXiv:2609.06993v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly generate Markdown that is consumed by renderers, agents, code extractors, and structured downstream pipeline

Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy

model-releasesarxiv-cs-ro

arXiv:2609.07470v1 Announce Type: new Abstract: Robot foundation models are trained and evaluated predominantly in English, and robot demonstration corpora do not exist for most languages. We study th

MOAE: Multi-Objective Agent Evolution with Pareto-Preserving Search

arXiv:2609.05992v1 Announce Type: cross Abstract: As LLM-based agents continue to advance, their evaluation has become increasingly multifaceted: a capable agent must not only achieve high task comple

NEO-BENCH: A New Multi-Source Benchmark for Generalizable Astronomical Streak Detection

arXiv:2609.06774v1 Announce Type: cross Abstract: Near-Earth Objects (NEOs) can appear as faint streaks in long-exposure astronomical images. Detecting these streaks across diverse observatories requi

NEST: Narrative Event Structures in Time for Long Video Understanding

arXiv:2606.19706v2 Announce Type: replace Abstract: Recent progress in vision-language models has enabled processing of increasingly long video sequences, but handling extended token streams does not

OdysSim: Building Foundation Models for Human Behavior Simulation

arXiv:2606.14199v2 Announce Type: replace-cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven p