Search: “benchmark”

Invent a Dataset: Measuring dataset generation abilities with zero seed

arXiv:2610.01674v1 Announce Type: new Abstract: Building datasets remains one of the most manual and brittle parts of AI development. In this technical report, we focus on the most extreme but also mo

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Measuring Iterative Temporal Reasoning with Time Puzzles

DGX agent

arXiv:2601.07148v4 Announce Type: replace-cross Abstract: Tool use, such as web search, has become a standard capability even in freely available large language models (LLMs). However, existing benchm

model-releasesarxiv-cs-ai

2 Oct 2026

Measuring the Microtask Eligibility Gap: When Is an Off-the-Shelf SLM Enough for an Agent Harness?

arXiv:2610.00025v1 Announce Type: new Abstract: Agent harnesses increasingly want to run small language models (SLMs) on the microtasks around a frontier large language model (LLM) planner: auto-appro

Measuring the Stability Assumption Behind Action Chunking

arXiv:2610.01626v1 Announce Type: new Abstract: Action chunking improves the performance of policies learned by behavioural cloning, and several mechanisms have been proposed to explain why, including

Hardware

MINCE: Shrinking LLM Evaluation Datasets via Few-Model Monte Carlo Calibration

arXiv:2606.22826v2 Announce Type: replace Abstract: Evaluating LLMs across many model variants---quantized, fine-tuned, or deployment-specific---requires running large benchmarks repeatedly, a process

hardwarearxiv-cs-ai

OpenMTB-Audit: Exposing Over-Refusal and Clinical Expert Perspectives in LLM-Based Molecular Tumor Board Safety Evaluation

arXiv:2610.01497v1 Announce Type: new Abstract: Molecular tumor boards integrate genomic findings, clinical context, and therapeutic evidence to support precision oncology. As AI enters this workflow,

PACT: End-to-End Learning of Human Pose, Contacts, and Forces from Video

arXiv:2610.00451v1 Announce Type: cross Abstract: Human motion, environmental contacts, and interaction forces are governed by common physical laws, yet existing approaches typically separate visual p

Pay for the Fault, Not the Flow: Label-Free In-Flow Multi-Agent Workflow Optimization

arXiv:2610.01017v1 Announce Type: new Abstract: Large language models (LLMs) increasingly construct multi-agent workflows that decompose a complex task and assign specialist agents from a pool. Howeve

Peacebell - a from-scratch small language model

model-releasesr-localllama

I've been using my free time during weeknights and weekends for the last 11 months working on and refining a small domain-specific language model. It specializes on information about World War II. The

Personalized Image Generation with Reasoning and Reflection

arXiv:2610.00737v1 Announce Type: cross Abstract: Personalized image generation has remained narrowly focused on conditional synthesis from curated visual exemplars, rather than capturing who a user i

PG-SFT: Balancing Capability Acquisition and Retention in Offline Agent Fine-Tuning

arXiv:2610.00949v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) on offline agent trajectories is the standard approach for training specialized tool-using agents, but forcing models to im

PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies

arXiv:2610.01383v1 Announce Type: new Abstract: Analogical reasoning involves identifying and preserving relational structures across domains. However, existing approaches to AI-driven multimodal anal

Q-Learning for Reachability in MEC-Free MDPs

arXiv:2610.01781v1 Announce Type: new Abstract: Reinforcement learning (RL) for reachability specifications is fundamental to sequential decision-making. Prior work establishes asymptotic convergence

Real-time Appearance-based Gaze Estimation for Open Domains

arXiv:2603.26945v2 Announce Type: replace Abstract: Appearance-based gaze estimation (AGE) has achieved remarkable performance in constrained settings, yet we reveal a significant generalization gap w

Reputation, Strategy, and Emotion Effects on Generative AI Cooperation: A Comparison Across Reasoning and Non-Reasoning Models

arXiv:2610.01222v1 Announce Type: new Abstract: As generative AI (Gen AI) systems take on increasingly autonomous roles in economically and socially consequential interactions, understanding their pro

Retrospective Open-Vocabulary Memory for Long-Term Object Search

arXiv:2610.00330v1 Announce Type: cross Abstract: Long-term object search requires learning where objects usually appear from repeated but uneven observations of a changing environment. We formulate r

REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection

arXiv:2502.14994v2 Announce Type: replace Abstract: The rapid advancement of AI-generated video poses challenges to digital authenticity and security. Current detection methods, often trained on speci

Revisiting Cross-Reconstruction for Generalizable Deepfake Detection

arXiv:2610.01544v1 Announce Type: new Abstract: Existing image forgery detectors often suffer from generalization to unseen manipulation methods due to the limited ability to capture transferable fore

Skin-Deep: A Geometric Diagnostic for Alignment Fragility in Large Language Model Representations

arXiv:2606.22676v2 Announce Type: replace Abstract: Refusal on a safety benchmark does not reveal how stable that behavior will remain after model updates. Benign downstream fine-tuning can weaken ref

SLIM: Simplex-Lattice Interpolation Merging

arXiv:2610.01037v1 Announce Type: new Abstract: Optimizing merging coefficients for large language models can require many costly benchmark evaluations. We propose extbf{Simplex-Lattice Interpolation

STATERA: Hidden Mass Estimation via Zero-Shot Sim-to-Real Kinematics using Frozen Temporal Tubelets

arXiv:2610.00003v1 Announce Type: cross Abstract: Vision models pretrained for frame-level appearance often struggle to infer hidden physical properties from motion. We study center-of-mass (CoM) loca

Watch, Infer, Coordinate: Inferring Robot Partner Constraints for Zero-Shot Coordination

arXiv:2610.02170v1 Announce Type: cross Abstract: Robots operating in the physical world will increasingly need to coordinate with other robots, particularly in manipulation tasks where an object may

When Do Biological Reasoning Models Use Their Biological Inputs?

arXiv:2610.00898v1 Announce Type: new Abstract: Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accura

Research

Which LLM to pick? Online Active Model Selection for Large Language Models

arXiv:2610.01592v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly applied to process streaming data, with practitioners relying on benchmarks to select the best model eve

researcharxiv-cs-lg

4MT-VLM: How Coarse Is a VLMs Cognitive Map?

arXiv:2609.39238v1 Announce Type: new Abstract: An agent that moves must recognise a place from a viewpoint it has never seen. We introduce 4MT-VLM, a dataset of procedurally generated landscapes, eac

Agents

After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data

arXiv:2609.38607v1 Announce Type: cross Abstract: Shadow removal looks nearly solved on established benchmarks, yet remains brittle in the real world. Models have advanced; the paired training data th

agentsarxiv-cs-ai

AIM: Agentic Idea Management for Automated Research

arXiv:2609.38445v1 Announce Type: new Abstract: Frontier LLMs are increasingly used to automate scientific research through iterative search. We distinguish idea-driven search from solution-driven sea

APTInvestBench: Evaluating Autonomous APT Investigation under Varying Telemetry

arXiv:2609.38954v1 Announce Type: cross Abstract: Large language model (LLM) agents could help security operations centers (SOCs) investigate advanced persistent threats (APTs) by turning weak leads i

Argument Structure Prediction in Online Conversations: A Comparative Study of Modeling Paradigms and Task Architectures

arXiv:2609.39225v1 Announce Type: new Abstract: Argument structure prediction (ASP) constructs complete argument structures from discourse by identifying argumentative units and their relations. While

Argus: A Real-EKS Study of When Predicting Spot Interruptions Beats Simple Checkpointing

arXiv:2609.39067v1 Announce Type: new Abstract: Elastic Compute Cloud (EC2) Spot is 60% to 90% cheaper than On-Demand but can be reclaimed on just a 2-minute notice; for expensive multi-node training

Agents

AutoDataBench: A Data-centric Testbed for Accelerating Auto Research

arXiv:2609.40097v1 Announce Type: new Abstract: Existing auto-research benchmarks often entangle multiple sources of improvement, including training frameworks, hyperparameters, compute budgets, and d

agentsarxiv-cs-cl

Autoresearch in Mixed-Integer Linear and Nonlinear Programming

arXiv:2609.39360v1 Announce Type: new Abstract: Despite recent progress in autoresearch, applying it to practical operations research problems, typically formulated as NP-hard mixed-integer linear or

BAM! Bayesian Anything Model: a foundation model for generative computational imaging

arXiv:2609.39660v1 Announce Type: new Abstract: Generative models are transforming Bayesian computational imaging, yet the field still lacks physics-aware foundation models. Current practice falls int

Breaking Babel: A Self-Evolving Multi-Agent System for Long-Form Subtitle Translation

arXiv:2609.38660v1 Announce Type: cross Abstract: Long-form subtitle translation requires reasoning over discourse and cultural context spanning episodes or entire series, while maintaining consistent

Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curves

arXiv:2609.40190v1 Announce Type: cross Abstract: Sampling several answers and keeping the one a verifier scores highest is one of the simplest ways to buy accuracy at test time. Its effect is reporte

Community-Driven API and AI Writer Design for Openly Scaling Community Notes

arXiv:2609.40067v1 Announce Type: cross Abstract: Community Notes is a crowd-sourced approach for adding context to posts on X. Contributors propose and rate notes, forming the inputs to an open-sourc

Comparison of techniques for fine-tuning open-weight models for entity extraction from radiology reports

arXiv:2609.40236v1 Announce Type: new Abstract: Converting free-text radiology reports into structured labels supports cohort building, quality assurance, and monitoring of clinical imaging models, bu

ECHO-G: Embodied Co-speech Humanoid mOtion Generation

arXiv:2609.39575v1 Announce Type: cross Abstract: Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To

FANVIDv2: Evaluating Video Super-Resolution by Face and Licence-Plate Recognition Under Compound Degradation

arXiv:2609.39649v1 Announce Type: new Abstract: Video super-resolution (VSR) is normally judged by PSNR and SSIM on clips that were downsampled bicubically, although in surveillance its purpose is to

FlapKAD: A Simulation Dataset of Coupled Wing Kinematics and Aerodynamic Dynamics for Flapping-Wing Aerial Vehicles

model-releasesarxiv-cs-ro

arXiv:2609.38719v1 Announce Type: new Abstract: Experimental investigation and modeling of flapping-wing aerial vehicles are limited by the scarcity of large-scale records that temporally align wing k

From DNA Design to DNA Slimming: Auditable Agentic Discovery of a Deletion-Only Designer

arXiv:2609.40143v1 Announce Type: new Abstract: Compact regulatory DNA can free up space in vector payloads, reduce synthesis and assay burden, and expose which sequence features drive predicted activ

From Verification Failures to Reusable Guidance for Coding Agents

arXiv:2609.39022v1 Announce Type: cross Abstract: Coding agents need to establish that a program satisfies a specification and that the specification captures the requested behavior. We study how expe

GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization

arXiv:2609.39128v1 Announce Type: new Abstract: Global video geo-localization aims to infer the geographic location of a video worldwide, evaluating performance across four geographic hierarchies: cit

GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis

arXiv:2609.38923v1 Announce Type: new Abstract: Working agents need to read diverse files, coordinate tools, and produce deliverables. Training such agents requires tasks built on many real files with

Hierarchical Reasoning Model

arXiv:2506.21734v4 Announce Type: replace Abstract: Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language

Industry

introducing 𝚌𝚕𝚎𝚏: our first models trained by @cloudflare's workers ai team. today, we're releasing two fast and accurate decision model…

introducing 𝚌𝚕𝚎𝚏: our first models trained by @cloudflare's workers ai team. today, we're releasing two fast and accurate decision models that top the benchmarks for quality and latency. use them host

industryclem-delangue--x

Large Language Models are Approximate Survival Estimators

arXiv:2609.38181v1 Announce Type: new Abstract: Survival analysis estimates time-to-event outcomes from patient covariates and is widely used for medical risk assessment. Patients seeking prognostic i

Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models

arXiv:2609.39820v1 Announce Type: cross Abstract: Vision-language-action (VLA) models generalize broadly across robotic manipulation tasks, but complex environments require balancing task success with