Search: “benchmark”

PrionNER: A Named Entity Recognition Dataset for Prion Disease Biomedical Literature

arXiv:2605.28375v1 Announce Type: new Abstract: Prion diseases are rare, rapidly progressive, and fatal neurodegenerative disorders that remain difficult to diagnose, particularly in their early stage

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation

DGX agent

arXiv:2605.28091v1 Announce Type: new Abstract: Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple

model-releasesarxiv-cs-cv

28 May 2026

Soro: A Lightweight Foundation Model and Chatbot for Tajik

arXiv:2605.27379v1 Announce Type: new Abstract: We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and co

The Importance of Being Statistically Earnest: A Critical Re-evaluation of GSM-Symbolic

arXiv:2605.28700v1 Announce Type: new Abstract: The GSM-Symbolic benchmark (Mirzadeh et al., 2025) reported consistent performance drops across 25 Large Language Models (LLMs) when tested on template-

The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study

arXiv:2504.04540v2 Announce Type: replace-cross Abstract: 3D Large Language Models (LLMs) leveraging spatial information in point clouds for 3D spatial reasoning attract great attention. Despite some

Tool Forge: A Validation-Carrying Toolchain for Governed Agentic Execution

arXiv:2605.28000v1 Announce Type: cross Abstract: Large language model agents are increasingly expected to perform operational work: calling APIs, manipulating files, assembling workflows, and acting

UniMaia: Steering Chess Policies with Language for Human-like Play

arXiv:2605.27767v1 Announce Type: cross Abstract: Recent advances in large language models have enabled natural language to serve as a flexible interface for controlling complex systems, but often at

Visualizing Latent Phase Structures in Locomotion Policies: A Multi-Environment Study with Temporal Feature Extension

arXiv:2605.28186v1 Announce Type: cross Abstract: Deep reinforcement learning (DRL) has been shown to achieve high performance on locomotion control tasks in MuJoCo benchmarks such as HalfCheetah, Ant

safetyarxiv-cs-ai

Announcing ESMFold2, our new state-of-the-art structure prediction model capable of predicting structure from single sequences or MSAs. ESMF…

model-releasesyann-lecun--x

Announcing ESMFold2, our new state-of-the-art structure prediction model capable of predicting structure from single sequences or MSAs. ESMFold2 improves on benchmarks of protein-protein interaction a

Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Models

arXiv:2506.09532v5 Announce Type: replace-cross Abstract: We present Athena-PRM, a multimodal process reward model (PRM) designed to evaluate the reward score for each step in solving complex reasonin

safetyarxiv-cs-ai

Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models

arXiv:2605.27311v1 Announce Type: new Abstract: Chart question-answering (QA) benchmarks aim to pose questions that require visual reasoning to correctly answer, but models can often reach solutions t

researcharxiv-cs-cl

ContextGuard: Structured Self-Auditing for Context Learning in Language Models

arXiv:2605.26827v1 Announce Type: cross Abstract: Recent benchmarks reveal that despite strong reasoning capabilities, large language models (LLMs) still struggle to faithfully apply complex contextua

researcharxiv-cs-ai

Conv-to-Bench: Evaluating Language Models Via User-Assistant Dialogues In Code Tasks

arXiv:2605.26440v1 Announce Type: new Abstract: The rapid advancement of Large Language Models (LLMs) has outpaced the scalability of traditional evaluation benchmarks, which remain heavily dependent

safetyarxiv-cs-cl

ECSEL: Explainable Classification via Signomial Equation Learning

arXiv:2601.21789v2 Announce Type: replace-cross Abstract: We introduce ECSEL, an explainable classification method that learns formal expressions in the form of signomial equations, motivated by the o

Evidence Absence Is Not Evidence Insufficiency: Diagnosing NEI Construction Artifacts in Fact Verification

arXiv:2605.26663v1 Announce Type: new Abstract: Evidence absence is not evidence insufficiency, but fact verification benchmarks can make them observationally similar. The Not Enough Information (NEI)

researcharxiv-cs-cl

IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams

model-releasesarxiv-cs-cv

arXiv:2605.27074v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) achieve strong performance on reactive question answering, but real-world streaming assistants require p

JobBench: Aligning Agent Work With Human Will

arXiv:2605.26329v1 Announce Type: new Abstract: Current benchmarks for occupational AI agents are scoped primarily by economic values, telling a replacement story. We introduce JobBench, which evaluat

LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval

model-releasesarxiv-cs-lg

arXiv:2510.13217v2 Announce Type: replace-cross Abstract: Search systems are increasingly used for reasoning-intensive queries, where what makes a document relevant requires understanding or reasoning

Omanic: Towards Step-wise Evaluation of Multi-hop Reasoning in Large Language Models

arXiv:2603.16654v2 Announce Type: replace-cross Abstract: Evaluating the reasoning abilities of large language models (LLMs) solely from final answers can obscure failures in intermediate steps, espec

Local Ai

Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study

arXiv:2605.26870v1 Announce Type: cross Abstract: Background: Large language models are typically evaluated as models, benchmarks, or short conversational episodes. Less is known about what happens wh

local-aiarxiv-cs-ai

PLAID: A Unified Data Model for Machine Learning on Heterogeneous Physics Simulations

arXiv:2505.02974v3 Announce Type: replace Abstract: Machine learning-based surrogate models have emerged as a powerful tool to accelerate simulation-driven scientific workflows, but their adoption is

researcharxiv-cs-lg

Reasoning, Code, or Both? How Large Language Models Handle Variations in Math Questions

arXiv:2605.26414v1 Announce Type: new Abstract: Large Language Models (LLMs) achieve impressive accuracy on mathematical reasoning benchmarks, yet their performance drops when problems are modified wi

RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations

arXiv:2605.26177v1 Announce Type: cross Abstract: Code agents are currently having skillful performance on repository-level software engineering benchmarks, but it remains unclear whether success on e

researcharxiv-cs-ai

Self-Ensembling Vision-Language Models for Chart Data Extraction

arXiv:2605.27298v1 Announce Type: new Abstract: Charts effectively convey quantitative information, but the underlying data are often locked in image form, hindering reuse and analysis. Manually digit

SpaceVista: All-Scale Visual Spatial Reasoning from mm to km

model-releasesarxiv-cs-cv

arXiv:2510.09606v2 Announce Type: replace Abstract: With the current surge in spatial reasoning explorations, researchers have made significant progress in understanding indoor scenes, but still strug

TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews

arXiv:2605.26911v1 Announce Type: new Abstract: LLM-generated peer reviews are increasingly common at major venues, yet their deficiencies are hard to detect because they are uniformly fluent and well

VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions

arXiv:2605.27141v1 Announce Type: new Abstract: Large language models (LLMs) have evolved into interactive agents that collaborate with users in real-world tasks. Effective collaboration in such setti

AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialogue

arXiv:2605.23974v1 Announce Type: new Abstract: Current language models create two safety challenges: risk must be detected early enough to avoid exposing harmful continuation, and the harmfulness its

Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks

arXiv:2505.24876v2 Announce Type: replace-cross Abstract: Deep reasoning is fundamental for solving complex tasks, especially in vision-centric scenarios that demand sequential, multimodal understandi

An Interpretable CF-RL-TOPSIS Fusion Model for Skills-Aware Talent Recommendation

arXiv:2605.24155v1 Announce Type: cross Abstract: Effective skills-aware talent recommendation must balance behavioral transition patterns, trajectory-sensitive adaptation, and inspectable occupation-

Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows

arXiv:2605.24219v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as autonomous agents that reason, use tools, and act over multiple steps. Yet most hallucination

Beyond Literal Translation: Evaluating Cultural Effectiveness in Social Media UGC

arXiv:2605.25626v1 Announce Type: new Abstract: Social media platforms enable large-scale cross-lingual communication, but translating user-generated content (UGC) remains challenging due to its infor

Binding Visual Features Point by Point

arXiv:2605.25427v1 Announce Type: cross Abstract: Despite success on standard benchmarks, vision language models display persistent failures on tasks involving processing of multi-object scenes, inclu

researcharxiv-cs-ai

Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express

arXiv:2605.25891v1 Announce Type: cross Abstract: We find a mismatch between what large language models encode about a causal question and what they answer. On anti-commonsense CLadder items, a fixed

Applications

ChainzRule: Sample-Efficient, Robust Deep Learning Across Tabular, NLP, and Vision Tasks

arXiv:2605.24340v1 Announce Type: new Abstract: Production deep learning systems across enterprise domains operate under constraints that academic benchmarks routinely obscure: labeled data is expensi

applicationsarxiv-cs-lg

ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference

arXiv:2510.02361v2 Announce Type: replace-cross Abstract: Transformer-based large models excel in natural language processing and computer vision, but face severe computational inefficiencies due to t

Closed-Form Node Classification with Exact Graph Unlearning

arXiv:2605.25662v1 Announce Type: new Abstract: Graph neural networks for node classification are typically trained by gradient descent over hundreds or thousands of epochs. Recent work has shown that

researcharxiv-cs-lg

Efficient Long-Horizon Vision-Language-Action Models via Static-Dynamic Disentanglement

model-releasesarxiv-cs-ro

arXiv:2602.03983v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon vision-language model (

Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3

arXiv:2605.25931v1 Announce Type: new Abstract: We systematically investigate all 25 public ARC-AGI-3 games and find that every one is reachable through non-intelligent strategies: 10 in a single blin

Fundamental Limits for Sensor-Based Control via the Gibbs Variational Principle

arXiv:2603.18454v2 Announce Type: replace-cross Abstract: Fundamental limits on the performance of feedback controllers are essential for benchmarking algorithms, guiding sensor selection, and certify

researcharxiv-cs-ro

LLMs Show No Signs Of Individuated Metacognition

arXiv:2605.24299v1 Announce Type: new Abstract: Confidence-weighted routing, selective abstention, and ensemble weighting all assume that a model's stated confidence is informative about its capabilit

researcharxiv-cs-lg

M^3-Verse: A 'Spot the Difference' Challenge for Large Multimodal Models

arXiv:2512.18735v2 Announce Type: replace-cross Abstract: Modern Large Multimodal Models (LMMs) have demonstrated extraordinary ability in static image and single-state spatial-temporal understanding.

PoseRefer: Pathway-Local Parameters for Semantically Grounded Reference Resolution

arXiv:2605.24622v1 Announce Type: new Abstract: A robot resolving ``put the cup on that one'' must fuse gesture, language, and scene geometry, yet 3D grounding benchmarks only partially capture this r

researcharxiv-cs-ro

Small Models, Strong Priors: Architectural Inductive Bias for Parameter-Efficient Neural PDE Solvers

arXiv:2605.25949v1 Announce Type: cross Abstract: Neural PDE solvers have followed the scaling trajectory of vision and language, with recent foundation models reaching billions of parameters. We argu

SMDD-Bench: Can LLMs Solve Real-World Small Molecule Drug Design Tasks?

arXiv:2605.21740v2 Announce Type: replace Abstract: LLM agents have incredible potential for scientific discovery applications. However, the performance of LLM agents on real-world, small molecule dru

StakeBench: Evaluating Language Understanding Grounded in Market Commitment

arXiv:2605.26074v1 Announce Type: cross Abstract: Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers ha

safetyarxiv-cs-ai

Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Training

arXiv:2603.17198v2 Announce Type: replace-cross Abstract: A foundational principle in cognitive science holds that intelligent agents do not learn by storing experiences as isolated instances, but by

Applications

Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions

arXiv:2605.24452v1 Announce Type: cross Abstract: Legal NLP benchmarks evaluate models on randomly split data, implicitly assuming that legal language is stationary. We test this assumption by fine-tu

applicationsarxiv-cs-ai