Search: “benchmark”

Adaptive Computation with Conformal Certification

arXiv:2610.19189v1 Announce Type: new Abstract: Scientific imaging data involving medical (e.g., 3D radiographic, microscopic images) and astronomical sensors pose different challenges from natural im

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Aligning Faithfulness for Theory-Scale Autoformalization

DGX agent

arXiv:2610.18498v1 Announce Type: cross Abstract: Autoformalization (AF) requires more than compilation: formal statements must preserve the meaning of their informal sources, a property known as fait

model-releasesarxiv-cs-ai

15 Oct 2026

ANCHOR: Assessing Naturalistic Constraint Hierarchies and Ordering in Repository-Grounded Agentic Coding

arXiv:2610.16859v2 Announce Type: replace Abstract: For repository-grounded coding agents, a functionally correct patch is not necessarily an acceptable one: developers also constrain how the task mus

Agents

Autonomous Science Bench: Evaluating AI Agents for Autonomous Scientific Discovery

arXiv:2610.18449v1 Announce Type: new Abstract: AI agents for autonomous scientific discovery must decide which experiments to run under a limited budget. Most benchmarks for science agents provide fi

agentsarxiv-cs-ai

Can LLMs Access their World Knowledge for Event Prediction?

arXiv:2610.19021v1 Announce Type: new Abstract: Recent work has questioned whether large language models (LLMs) can be useful for temporal event prediction. Despite vast pretrained world knowledge, LL

Confidently Wrong: Detecting Hallucinations in Financial Question Answering from LLM Internal States

arXiv:2607.11414v2 Announce Type: replace Abstract: Large language models (LLMs) in financial applications fail most consequentially when they are confidently wrong. Hedged, uncertain answers invite s

Conformalized Physics-Informed Weak Regression for Parameter Estimation in Epidemic Models

arXiv:2610.19058v1 Announce Type: cross Abstract: Compartment models of infectious diseases are often linear in coefficients that are physical parameters or products or ratios of them. We exploit this

Continuous Risk-Based Scheduling for Food Safety Inspection: Integrating Historical Decay with Event-Driven Consumer Review Signals

arXiv:2610.15191v1 Announce Type: new Abstract: Batch risk-based inspection scheduling struggles to detect a critical failure class: food facilities with recent clean inspections that subsequently dev

Research

CoWear: Towards Collaborative Wearable Inertial Localization with Multiple Smart Devices

arXiv:2610.20596v1 Announce Type: new Abstract: Consumer smart devices observe the same human motion from different body locations, yet existing inertial-localization benchmarks rarely provide synchro

researcharxiv-cs-ro

Research

ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies

arXiv:2610.19388v1 Announce Type: new Abstract: Massively multilingual speech modeling has made remarkable progress in recent years, yet existing training resources and benchmarks still underrepresent

researcharxiv-cs-cl

Evaluating Large Language Models for Multilingual Abstractive Summarization: A Comprehensive Study of English and Bengali

arXiv:2610.19003v1 Announce Type: new Abstract: Abstractive summarization remains challenging across languages, where linguistic quality and factual consistency may diverge, particularly in low-resour

Exhaustive Canonical Skeletons Generation: A Formal Dataset for Decomposable Symbolic Regression

arXiv:2610.20108v1 Announce Type: new Abstract: Neural symbolic regression (SR) models require diverse datasets of mathematical expressions for robust training. However, existing datasets rely predomi

Applications

Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization

arXiv:2604.06138v2 Announce Type: replace-cross Abstract: Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open

applicationsarxiv-cs-ai

How Far Can Coding Agents Go in Visual Generation?

arXiv:2610.19025v1 Announce Type: new Abstract: Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We

Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents

arXiv:2610.17757v1 Announce Type: new Abstract: Large language model agents can follow complex human instructions to solve real-world tasks. In practice, however, initial instructions are often incomp

KiToke: Distribution-Aware Token Compression for Video Large Language Models

arXiv:2604.03414v2 Announce Type: replace Abstract: Video Large Language Models (Video LLMs) achieve strong performance on video understanding tasks but suffer from high inference costs due to the lar

Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback

arXiv:2610.19380v1 Announce Type: new Abstract: User feedback enables LLMs to refine reusable personalized skills, but often mixes persistent preferences with request-specific corrections. Indiscrimin

MCPDisCo: Seesaw-Mitigated Tuning for MCP-Native Agents

arXiv:2610.14956v1 Announce Type: new Abstract: The Model Context Protocol provides a shared interaction substrate for the agentic web, allowing heterogeneous services to expose capabilities through a

Mistral just released Timestral, a time series foundation model, on Hugging Face State-of-the-art zero-shot forecasting across 3 major bench…

model-releasesclem-delangue--x

Mistral has released **Timestral**, a time‑series foundation model available on Hugging Face. The model achieves state‑of‑the‑art zero‐shot forecasting performance across three major benchmarks, suppo

More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning

arXiv:2610.19789v1 Announce Type: new Abstract: Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseli

NavAble: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation

arXiv:2610.19330v1 Announce Type: new Abstract: Reliable recognition of accessibility-critical objects (e.g., audible pedestrian signals, door-activation buttons, and handrails) is essential for assis

Neural POS Tagging for the Low-Resource Saraiki

arXiv:2610.20279v1 Announce Type: new Abstract: Part-of-speech tagging is a fundamental sequence labeling task and an important component for many downstream natural language processing applications.

@nvidia Run Nemotron 3.5 Super VL on Together AI Start building: https://www.together.ai/models/nemotron-3-5-super-vl

model-releasestogether-ai--x

NVIDIA’s Nemotron 3.5 Super VL is now available to run on the Together AI compute platform through an API. The model page—providing pricing, benchmarks, and documentation—is hosted at together.ai/mode

PhyMetric: Diagnosing Physical Plausibility in Text-to-Video Generation via Scene-Level QA

arXiv:2610.19823v1 Announce Type: new Abstract: Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt an

ProCARE: Real-World Study Automation via Profile-Grounded Evidence Contracts

arXiv:2610.16175v1 Announce Type: new Abstract: Real-world studies (RWS) analyze routinely collected healthcare data, such as electronic health records and insurance claims, to estimate treatment effe

PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

arXiv:2606.18060v2 Announce Type: replace Abstract: As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important. Oth

Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation

arXiv:2610.20335v1 Announce Type: new Abstract: Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction, a

Recognizing Dialogue Ambiguity via Candidate Set Visual Grounding

arXiv:2610.19723v1 Announce Type: new Abstract: Interpreting an ambiguous utterance in real-world interactions often requires context beyond a single utterance. When the utterance refers to objects in

Recursive Improvement of a Differentiable Scientific Software Ecosystem

arXiv:2610.04561v2 Announce Type: replace Abstract: Differentiable programming connects scientific computation with gradient-based inference, learning and design. Extending these capabilities across a

Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos

arXiv:2610.14495v1 Announce Type: new Abstract: Explainer videos -- a worked geometry problem, why the sky is blue, how a dynasty fell -- are usually one more target for pixel video generation. We arg

Reliable Text-to-SQL in Resource-Constrained Settings with Small LLMs via Iterative Retry Workflows

arXiv:2610.17305v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently improved Text-to-SQL generation, but their computational cost can hinder deployment in resource-constrained s

ROAR: Unifying Runs across Heterogeneous AI-Driven Research Systems

arXiv:2610.03966v2 Announce Type: replace Abstract: Each run of an AI-driven research system (ADRS) is an expensive search over a vast solution space, and dependable evaluation requires many runs, mak

SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition

arXiv:2610.14966v1 Announce Type: new Abstract: Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluat

SolBehave: Evaluating the Behavioural Correctness of LLM-Generated Smart Contracts

arXiv:2610.18745v1 Announce Type: cross Abstract: Large language models increasingly generate Solidity smart contracts that, once deployed, can never be patched, making a single latent fault permanent

StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions

arXiv:2610.17309v1 Announce Type: cross Abstract: We introduce StartLux-Decision, decision models from 0.8B to 27B parameters that combine efficient single-pass prediction with adaptive, confidence-gu

The Invisible Leash: Why RLVR May or May Not Escape Its Origin

arXiv:2507.14843v5 Announce Type: replace-cross Abstract: Recent advances highlight Reinforcement Learning with Verifiable Rewards (RLVR) as a promising approach to enhancing LLM capabilities. However

The Reasoning Trap -- Logical Reasoning as a Mechanistic Pathway to Situational Awareness

arXiv:2603.09200v2 Announce Type: replace-cross Abstract: Situational awareness, the capacity of an AI system to recognize its own nature, understand its training and deployment context, and reason st

Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

arXiv:2610.19108v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitat

Tutorials

What Does LALM Post-training Learn? Rethinking Audio Understanding and Reasoning Capabilities

arXiv:2610.19476v1 Announce Type: cross Abstract: Post-training often improves the performance of large audio-language models on audio understanding and reasoning benchmarks. However, these improvemen

tutorialsarxiv-cs-cl

3D-VITAL: Visibility-aware Identity Training from Artificial Liftings of 3D Reconstruction model for Cross-View Object Re-Identification

arXiv:2610.16692v1 Announce Type: new Abstract: We address the Cross-View Object Re-Identification (ReID) task, aiming to match identities across drastically different viewpoints. We focus on Single-V

AI Discovery in Combinatorics: From Problems to Solutions and Back to Bigger Problems

arXiv:2610.16555v1 Announce Type: cross Abstract: We introduce qtBench, a benchmark of 29 open problems in algebraic combinatorics. These problems ask for a simple combinatorial algorithm that explain

Artificiality's First Exam: An LLM-Referenced C-test for Supraintelligence

arXiv:2610.14005v1 Announce Type: cross Abstract: Under the accelerated pace of AI development, any benchmark whose difficulty is calibrated with humans or current AI systems becomes circular, or satu

AURORA-KITTI: Any-Weather Depth Completion and Denoising

arXiv:2603.14701v2 Announce Type: replace Abstract: Robust depth completion is fundamental to real-world 3D scene understanding, yet existing RGB--LiDAR fusion methods degrade significantly under adve

Beyond a Better Score: Long-Horizon Agentic ML Development and Evaluation Protocol for Physics Time Series

arXiv:2610.14024v1 Announce Type: new Abstract: Physics time-series data are central to fundamental discoveries involving dark matter, neutrinos, and gravitational waves. Machine learning has shown st

Can a Frontier VLM Serve as a Robot Policy for Memory-Dependent Manipulation?

model-releasesarxiv-cs-ro

arXiv:2610.17350v1 Announce Type: new Abstract: We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation ta

CAST: Real-Time Motion Capture for Any Skeleton Topology

arXiv:2610.14856v1 Announce Type: new Abstract: This paper studies the problem of recovering motion from monocular video, even when the target skeleton topology differs from the subject observed in th

Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation

arXiv:2610.17651v1 Announce Type: cross Abstract: Recent coding large language models (LLMs) can synthesize dynamic 4D worlds from natural language, but the resulting content remains a visual artifact

DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition

arXiv:2610.16956v1 Announce Type: new Abstract: Historical document analysis is crucial for cultural heritage preservation and low-resource language digitization. However, real historical documents of