Search: “benchmark”

North or South? Diagnosing LLMs' Understanding of North Korean Language, Society, and Knowledge

arXiv:2610.15282v1 Announce Type: cross Abstract: Large Language Models (LLMs) have made substantial progress in multilingual question answering, yet their ability to understand low-resource socioling

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Of Errors and Echoes: Responsibility-Aware Credit Assignment for Tool-Use Agents

DGX agent

arXiv:2610.16027v1 Announce Type: cross Abstract: Reinforcement learning improves tool-use agents, yet fine-grained credit assignment remains challenging even with step-level feedback. An observed err

safetyarxiv-cs-ai

15 Oct 2026

Optimization for One, Feasibility for Most: Decoupling Optimization from Feasibility in Chance-Constrained Programs

arXiv:2610.19027v1 Announce Type: cross Abstract: Chance-constrained stochastic programs (CCSPs) require solutions that satisfy uncertain constraints with high probability. Standard approaches either

PANINI-DR: Controlled Deep Research through Generative Semantic Workspaces

arXiv:2610.15208v1 Announce Type: cross Abstract: Deep research requires connecting multiple steps of search, reasoning, and computation, with intermediate findings guiding what to investigate next. A

PersEvalBench: Can LLM Evaluators Judge Contextually Appropriate Use of Persona Information?

arXiv:2610.19314v1 Announce Type: new Abstract: Large language model (LLM) evaluators are increasingly used to assess personalized assistants, yet whether they can reliably judge personalization has r

PLACID: Privacy-preserving Large language models for Acronym Clinical Inference and Disambiguation

arXiv:2603.23678v2 Announce Type: replace Abstract: Large Language Models (LLMs) offer transformative solutions across many domains, but healthcare integration is hindered by strict data privacy const

Plans Fade Fast: Why Context Management Is Load-Bearing for LLM Agents

arXiv:2606.22953v2 Announce Type: replace Abstract: Context managers summarize or evict old messages so agents can operate beyond a finite window. This creates a reliability problem: removing history

Policy-Verified Self-Training: Learning Agent Compliance from Verifiable Policy Rewards

arXiv:2610.19372v1 Announce Type: new Abstract: Runtime policy verifiers can prevent agents from executing disallowed actions, but their decisions are typically used only for enforcement and discarded

Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation

arXiv:2409.00696v4 Announce Type: replace-cross Abstract: Rating-based human evaluation has become an essential tool to accurately evaluate the impressive performance of large language models (LLMs).

RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution

arXiv:2610.14659v1 Announce Type: new Abstract: Streaming speech assistants can launch tools before users finish speaking, overlapping execution with incoming speech to reduce user waiting time. Howev

ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models

arXiv:2610.20503v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden featur

RingNav: Task-Shaped Circular Memory for Visual Heading

arXiv:2610.19044v1 Announce Type: cross Abstract: Tiny robots have limited computation and memory, yet visual navigation systems often maintain more state than a task requires. We ask whether persiste

Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution

arXiv:2610.19495v1 Announce Type: cross Abstract: Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed,

Research

SAM3RL: Memory Control via Reinforcement Learning for Visual Object Tracking

arXiv:2610.19931v1 Announce Type: new Abstract: Segment Anything Model 3 (SAM 3) is a foundation model for image and video segmentation that sets the state of the art in visual object tracking. Its tr

researcharxiv-cs-cv

Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers

arXiv:2610.17820v1 Announce Type: cross Abstract: TypeSafe AI's Jev is a new kind of safety judge. It generates no text and no label: it answers questions the developer defines in advance and returns

Same Claim, Different Interpretations: How LLM Agents Resolve Ambiguity in Scientific Claims

arXiv:2610.20457v1 Announce Type: new Abstract: Co-scientists and agentic scientific claim verifiers are increasingly used to assess claims across the sciences. Natural-language scientific claims, wri

Scheduling Parallel Optical Circuit Switches for AI Training

arXiv:2603.07373v2 Announce Type: replace-cross Abstract: The rapid growth of AI training has dramatically increased datacenter traffic demand and energy consumption, which has motivated renewed inter

Research

Selective Next-Location Prediction with MetaModelSelect: A Lightweight Post-Hoc Confidence Metamodel

arXiv:2610.19394v1 Announce Type: new Abstract: Next-location prediction can support proactive mobile-network decisions, but acting on unreliable predictions can waste resources. We apply MetaModelSel

researcharxiv-cs-lg

SkillForgeBench: Evaluating Agent Reliability Grounded in Agent Skill Specifications

arXiv:2610.14918v1 Announce Type: new Abstract: Evaluating an agent's ability to strictly satisfy diverse and procedurally complex requirements is essential to determining its practical reliability. C

SkillFusion: Towards Reliable Multi-Skill Composition for LLM Agents

arXiv:2610.15481v1 Announce Type: new Abstract: Independently authored skills can impose incompatible requirements, competing preferences, and redundant instructions when jointly loaded by an LLM agen

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

arXiv:2610.16814v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as pa

Research

State-Matched Sequence Conformal Prediction

arXiv:2610.19484v1 Announce Type: cross Abstract: Sequential conformal prediction needs to decide which past errors should calibrate uncertainty for the current target. Recency and online error feedba

researcharxiv-cs-lg

Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance

arXiv:2602.22583v2 Announce Type: replace Abstract: Example-based guidance is widely used to improve mathematical reasoning at inference time, yet its effectiveness is highly unstable across problems

SWEPatchTest: Evaluating Software Tests Beyond Reproduction

arXiv:2610.18004v1 Announce Type: cross Abstract: Coding agents use generated tests to decide whether a proposed patch satisfies an issue's requirements. A suite that fails on the buggy program and pa

TextualRL: Textual Reinforcement Learning for Context Management

arXiv:2610.18216v1 Announce Type: cross Abstract: LLM agents can adapt to task feedback by revising the textual context that guides their behavior, without changing their model parameters. However, tu

The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery Blog: https://pub.sakana.ai/agentic-scientist/ Gi…

model-releasesdavid-ha--x

The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery Blog: https://pub.sakana.ai/agentic-scientist/ GitHub: https://github.com/SakanaAI/AI-Scientist-v3 arXiv: htt

Tutorials

The Impact of Detector Architecture, Model Scale, and Scene Density on Multi-Pedestrian Tracking

arXiv:2610.19369v1 Announce Type: new Abstract: Pedestrian tracking in video is important for applications such as surveillance and scene analysis, but it becomes more challenging in crowded scenarios

tutorialsarxiv-cs-cv

Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces

arXiv:2610.15055v1 Announce Type: new Abstract: Self-evolving agents are usually scored by endpoint Gain: held-out success after an update minus success before. That difference can credit a frozen res

Training a Compact, Strong, and Realistic Text-to-Speech Model

arXiv:2610.14776v1 Announce Type: cross Abstract: Text-to-speech (TTS) systems are conventionally optimized for intelligibility and speaker similarity. However, neither objective captures the prosodic

Truthful and Expected-Payment-Optimal Model Routing in Graph-Based Agentic Workflows

arXiv:2610.19316v1 Announce Type: cross Abstract: Assigning routine tasks to cheaper large language model (LLM) providers can reduce generation cost, but switching providers may require processing acc

Uncovering Robustness Weaknesses in LLM Agents via Persona Evolution with Task-Integrity Constraints

arXiv:2610.16155v1 Announce Type: new Abstract: Interactive-agent evaluations commonly rely on standardized LLM-based user simulators that exhibit broadly cooperative behavior, potentially overlooking

Verification as an Architectural Layer for LLM Agents: A V-Model Design, and a Pilot Study of Its Deterministic Core

arXiv:2609.31937v2 Announce Type: replace-cross Abstract: Large language model (LLM) agents built on the ReAct pattern concentrate four responsibilities in one model: selecting a strategy, choosing ea

Verifier-Harness Co-Evolution for Training-Free Self-Improvement on Open-Ended Tasks

arXiv:2610.17609v1 Announce Type: new Abstract: Training-free self-improvement lets an agent use feedback to revise the response it is about to return and to update the harness state that shapes how i

Virtual-Alloy Physics-Aware Semantic System for extrapolative prediction of perovskite solar cells

arXiv:2610.18949v1 Announce Type: cross Abstract: Existing machine-learning models for perovskite solar cells struggle to predict devices made from new materials. Most treat device layers as arbitrary

When Pleasing the User Compromises Care: AI Sycophancy in Caregiver-Mediated Healthcare

arXiv:2610.14360v1 Announce Type: new Abstract: Large language model (LLM) alignment often assumes that the user interacting with a model is also the primary beneficiary of its behavior. Yet in many c

Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment

arXiv:2610.19052v1 Announce Type: new Abstract: Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable

Agentic Data Engineering for LLMs: System Design and a Controlled Empirical Study

arXiv:2610.14238v1 Announce Type: cross Abstract: LLM data engineering requires iterative experimentation to determine how data selection, rewards, and curricula affect model learning, making it a nat

agentsarxiv-cs-lg

Safety

AnyRank: Reliable Anytime Policy Ranking via Sequential Testing

arXiv:2610.14603v1 Announce Type: new Abstract: Due to the high cost of hardware experiments, robot policies are predominantly evaluated by comparing their empirical performance on a limited number of

safetyarxiv-cs-ro

APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation

arXiv:2610.18097v1 Announce Type: new Abstract: Post-training is crucial for improving text-to-image models, yet existing pipelines typically rely on manually curated training data and fixed training

Backward Conformal Prediction under Covariate Shift

arXiv:2610.16048v1 Announce Type: new Abstract: Backward conformal prediction (BCP) fixes the size of a prediction set and estimates its miscoverage, but assumes calibration and test data are exchange

BARTER: Compact Dynamic Sparse Training with Standard Dense Kernels

arXiv:2610.16147v1 Announce Type: new Abstract: Dynamic sparse training (DST) aims to reduce training costs while maintaining accuracy. Existing DST methods mask out parameters but keep matrix dimensi

BaT-SAM: Introducing Inductive Bias into SAM via a Mixture of Convolutional Experts for Brain Tumor Segmentation

arXiv:2610.15330v1 Announce Type: new Abstract: Accurate brain tumor segmentation from magnetic resonance imaging (MRI) is essential for diagnosis and treatment planning; however, manual delineation i

Bidirectional Adaptive Transformers for Multimodal Anomaly Detection

arXiv:2610.14598v1 Announce Type: new Abstract: The effective integration of features derived from RGB images and 3D point clouds presents a substantial challenge in the field of multimodal anomaly de

Brain2Face: A Study on the Relation of EEG Signals and Spatio-temporal Head Avatars

arXiv:2610.17356v1 Announce Type: new Abstract: Driving digital head avatars is classically done by hand-crafted artist design or by image-based facial performance capture. In this work, we ask whethe

DRMOT: A Dataset and Framework for RGBD Referring Multi-Object Tracking

arXiv:2602.04692v3 Announce Type: replace Abstract: Referring Multi-Object Tracking (RMOT) aims to track specific targets based on language descriptions, playing a vital role in interactive AI systems

agentsarxiv-cs-cv

DuSRFlow: Dual-Lens Super-Resolution via Parallax-Aware Flow Estimation and Complementary Reference Warping

arXiv:2610.14630v1 Announce Type: new Abstract: Dual-lens super-resolution (DuSR) aims to enhance a wide-angle low-resolution (LR) image by exploiting a telephoto reference (Ref) image. A central chal

Tutorials

ECG-RSI: Recursive Self-Improvement through Targeted Electrocardiogram Waveform Evolution

arXiv:2610.17718v1 Announce Type: cross Abstract: Electrocardiogram (ECG) models need supervision that targets evolving weaknesses and remains faithful to waveform evidence. We introduce ECG-RSI, a re

tutorialsarxiv-cs-lg

EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agents

arXiv:2610.16679v1 Announce Type: new Abstract: Tool-augmented vision-language agents compose perception and measurement to reason about spatial relationships, but effective execution also requires tr