Search: “benchmark”

TriVAL: A Tri-Validation Framework for Faithful Automatic Optimization Modeling

arXiv:2605.23966v1 Announce Type: cross Abstract: Optimization modeling serves as the pivotal bridge between natural-language problem descriptions and optimization solvers, and remains a cornerstone f

26 May 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

TypedCSIP: Typed Counterfactual Pretraining for Chinese Legislative Conflict Classification

DGX agent

arXiv:2605.25474v1 Announce Type: new Abstract: TypedCSIP is a typed counterfactual pretraining method for the conflict-classification task of the LCR-CN benchmark (Zhao et al., 2026): given a (superi

model-releasesarxiv-cs-cl

26 May 2026

When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure

arXiv:2605.23932v1 Announce Type: new Abstract: Despite strong medical benchmark accuracy, LLMs can exhibit severe multi-turn sycophancy in clinical dialogue, abandoning initial correct diagnosis unde

26 May 2026

Agents

Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory

arXiv:2601.22984v2 Announce Type: replace Abstract: Diagnosing failure patterns in Deep Research Agents (DRAs) remains a critical challenge. Existing benchmarks predominantly rely on end-to-end evalua

agentsarxiv-cs-ai

26 May 2026

Agentic Proving for Program Verification

arXiv:2605.23772v1 Announce Type: new Abstract: Agentic systems have recently emerged as state-of-the-art approaches for automated theorem proving in formal mathematics. To assess how far these capabi

Agents

MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination

arXiv:2605.22949v1 Announce Type: new Abstract: Foundation model agents increasingly operate in multi-agent deployments where a coordinator must decide which agent's response to trust. The standard ap

agentsarxiv-cs-lg

One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents

arXiv:2605.23652v1 Announce Type: new Abstract: On a 300-persona life-simulation benchmark, pcsp achieves compositional zero-shot persona identification up to 17x above chance, Spearman rho approx 0.7

SkillOpt: Executive Strategy for Self-Evolving Agent Skills

arXiv:2605.23904v1 Announce Type: new Abstract: Agent skills today are hand-crafted, generated one-shot, or evolved through loosely controlled self-revision, none of which behaves like a deep-learning

Speak-to-Structure: Evaluating LLMs in Open-domain Natural Language-Driven Molecule Generation

arXiv:2412.14642v4 Announce Type: replace Abstract: Recently, Large Language Models (LLMs) have demonstrated great potential in natural language-driven molecule discovery. However, existing datasets a

Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning

arXiv:2605.23171v1 Announce Type: cross Abstract: Recent advancements in instructional fine-tuning have injected noise into embeddings, with NEFTune (Jain et al., 2024) setting benchmarks using unifor

An interesting work on Physical AI: PhysX-Omni. First unified sim-ready generation framework for rigid, deformable, and articulated objects,…

model-releasesclem-delangue--x

An interesting work on Physical AI: PhysX-Omni. First unified sim-ready generation framework for rigid, deformable, and articulated objects, with a diverse dataset and new benchmark. 🌐 https://physx-o

24 May 2026

Chebyshev Policies and the Mountain Car Problem: Reinforcement Learning for Low-Dimensional Control Tasks

arXiv:2605.22305v1 Announce Type: new Abstract: We analytically solve the Mountain Car problem, a canonical benchmark in RL, and derive an optimal control solution, closing a gap after 36 years. This

Evaluation of Pipelines for Data Integration into Knowledge Graphs

arXiv:2605.22304v1 Announce Type: cross Abstract: Integrating new data into knowledge graphs (KG) typically involves different tasks that are executed within workflows or pipelines There are many poss

MapTab: Are MLLMs Ready for Multi-Criteria Route Planning in Heterogeneous Graphs?

arXiv:2602.18600v3 Announce Type: replace Abstract: Systematic evaluation of Multimodal Large Language Models (MLLMs) is crucial for advancing Artificial General Intelligence (AGI). However, existing

Safety

Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games

arXiv:2602.10894v2 Announce Type: replace Abstract: Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimizat

safetyarxiv-cs-lg

Shipper’s “After Automation” is the best framing I’ve read of why AI keeps generating more work, not less. The line that stuck with me: the …

model-releasesitamar-friedman--x

Shipper’s “After Automation” is the best framing I’ve read of why AI keeps generating more work, not less. The line that stuck with me: the frame is not the framer. Models climb whatever benchmark we

AgentCo-op: Retrieval-Based Synthesis of Interoperable Multi-Agent Workflows

arXiv:2605.20425v1 Announce Type: new Abstract: Designing multi-agent workflows is especially difficult in open-ended scientific settings where tasks lack curated training sets, reliable scalar evalua

Cambrian-P: Pose-Grounded Video Understanding

arXiv:2605.22819v1 Announce Type: new Abstract: Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that relates observations across video fram

researcharxiv-cs-cv

Declarative Data Services: Structured Agentic Discovery for Composing Data Systems

arXiv:2605.20690v1 Announce Type: new Abstract: Agentic discovery has shown that LLM-driven search can find novel algorithms, designs, and code under benchmark conditions. Translating the paradigm to

Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis

arXiv:2510.08759v2 Announce Type: replace Abstract: Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improving embodied agents. However, exi

Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines

arXiv:2605.20630v1 Announce Type: new Abstract: Industrial asset operations workflows are latency-sensitive because a single user query may require coordination over sensor data, work orders, failure

H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning

arXiv:2605.22629v1 Announce Type: new Abstract: Parametric human models capture global pose but cannot represent the non-rigid surface dynamics of clothing and soft tissue. Generic scene flow estimate

InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation

arXiv:2510.09724v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly capable of generating complete applications from natural language instructions, creating new opp

Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?

arXiv:2605.22109v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in human-facing roles where personality perception is critical, yet existing benchm

Tackle CSM in JPEG Steganalysis with Data Adaptation

arXiv:2605.21523v1 Announce Type: cross Abstract: Steganalysis models excel on benchmark datasets but struggle in the wild when analyzed images are produced by a processing pipeline unseen during trai

Targeting Clause Type Distributions: a Picklock for Random Satisfiability Problems

arXiv:2605.20328v1 Announce Type: cross Abstract: Optimization problems such as the NP-complete 3-SAT provide an important benchmark for the difficult task of finding ground-states in strongly correla

Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence

arXiv:2605.22414v1 Announce Type: new Abstract: Visual Question Answering (VQA) holds great promise for clinical support, particularly in ophthalmology, where retinal fundus photography is essential f

ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison

arXiv:2605.20278v1 Announce Type: cross Abstract: Long-form image captioning exposes a reward granularity problem in RL: captions are judged as whole sequences, while the important errors occur at the

Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models

arXiv:2411.01141v2 Announce Type: replace Abstract: There are two shortages in the current Large Language Models (LLMs) era. The first is short of multilingual models, where most LLMs are English-cent

researcharxiv-cs-cl

Enhanced Reinforcement Learning-based Process Synthesis via Quantum Computing

arXiv:2605.21213v1 Announce Type: cross Abstract: In this work, we present quantum reinforcement learning (RL) as a solution strategy for process synthesis problems. Building on our prior work, we dev

Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry

arXiv:2605.20241v1 Announce Type: cross Abstract: Prompt-level safety probes for large language models use hidden-state representations to separate safe from unsafe prompts, but strong average detecti

InternBootcamp Technical Report: Boosting LLM Reasoning with Verifiable Task Scaling

arXiv:2508.08636v2 Announce Type: replace Abstract: Large language models (LLMs) have revolutionized artificial intelligence by enabling complex reasoning capabilities. While recent advancements in re

Markovian Circuit Tracing for Transformer State Dynamic

arXiv:2605.20824v1 Announce Type: new Abstract: Many sequence computations are easier to study as movement through internal states than as isolated local circuits. We introduce Markovian Circuit Traci

Mechanics of Bias and Reasoning: Interpreting the Impact of Chain-of-Thought Prompting on Gender Bias in LLMs

arXiv:2605.20410v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in socially sensitive settings despite substantial documentation that they encode gender biases.

PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models

arXiv:2605.20873v1 Announce Type: cross Abstract: Planning is a fundamental capability for large language models (LLMs) because such complex tasks require models to coordinate goals, constraints, reso

Reliable Automated Triage in Spanish Clinical Notes: A Hybrid Framework for Risk-Aware HIV Suspicion Identification

arXiv:2605.21256v1 Announce Type: new Abstract: Standard clinical Natural Language Processing (NLP) benchmarks often yield inflated metrics by forcing deterministic classification on ambiguous instanc

researcharxiv-cs-cl

When Irregularity Helps: A Subclass Analysis of Inductive Bias in Neural Morphology

arXiv:2605.20558v1 Announce Type: new Abstract: Neural morphological generation systems often achieve high aggregate accuracy on benchmark datasets, yet such performance can conceal systematic errors

Safety

ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiders

arXiv:2605.19503v1 Announce Type: cross Abstract: Reinforcement learning for legged locomotion has matured into a stack of multi-component reward functions and physics-engine benchmarks whose morpholo

safetyarxiv-cs-ai

brain dump of how/why we use Evals to measure agents before & after shipping to prod 1. Good Evals simulate what our real users will do and …

model-releasesharrison-chase--x

brain dump of how/why we use Evals to measure agents before & after shipping to prod 1. Good Evals simulate what our real users will do and encounter. They’re not really random benchmark tasks, they r

CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models

arXiv:2605.20165v1 Announce Type: new Abstract: Vision-Language Models (VLMs) achieve strong performance on spatial question answering benchmarks, yet it remains unclear whether such gains reflect gen

researcharxiv-cs-cv

Local Ai

Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds

arXiv:2605.18827v1 Announce Type: cross Abstract: Multiple-choice QA benchmarks usually evaluate small language models (SLMs) as direct answerers, but deployed language-model systems increasingly rely

local-aiarxiv-cs-lg

DocQT: Improving Document Forgery Localization Robustness via Diverse JPEG Quantization Tables

arXiv:2605.19688v1 Announce Type: new Abstract: Document manipulation localization models achieve strong performance on public benchmarks yet fail to generalize to operational document workflows. We i

How researchers built a record-setting quantum circuit

IBM researchers have achieved a record-breaking quantum circuit by advancing quantum computing benchmarking and performance capabilities. The work likely demonstrates progress in quantum algorithm imp

researchibm-research

MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification

arXiv:2605.19752v1 Announce Type: new Abstract: Accurately identifying metabolites i.e. small molecules from mass spectrometry data remains a core challenge in metabolomics, with broad applications in

PhyWorld: Physics-Faithful World Model for Video Generation

arXiv:2605.19242v1 Announce Type: cross Abstract: World simulators can provide safe and scalable environments for training Physical AI systems before real-world deployment. Large video generation mode

Quantum Machine Learning for Cyber-Physical Anomaly Detection in Unmanned Aerial Vehicles: A Leakage-Free Evaluation with Proxy-Audited Feature Sets

arXiv:2605.19233v1 Announce Type: cross Abstract: Unmanned aerial vehicles (UAVs) are cyber-physical systems whose attack surface spans networked avionics and on-board sensor fusion: a compromised GPS

Industry

The Internet can't stop watching Figure AI's humanoid robots handling packages

Figure AI livestreamed three F.03 humanoid robots sorting packages continuously starting May 14, 2026 , generating significant viral attention and becoming a benchmark demonstration for autonomous hum

industryars-technica

To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents

arXiv:2605.18882v1 Announce Type: cross Abstract: LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models