Search: “benchmark”

Zero-shot reasoning for simulating scholarly peer-review

arXiv:2510.02027v2 Announce Type: replace Abstract: Scholarly publishing requires scalable scrutiny supported by auditable evidence. This paper presents a two-component benchmark of xPeer, the peer-re

6 Aug 2026

Research

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Ai2 expands collaboration with Hugging Face to accelerate open science

DGX agent

Ai2 is expanding its partnership with Hugging Face to give its growing portfolio of fully open models, datasets, benchmarks, and applications the storage, bandwidth, and integrations needed to reach m

researchai2

5 Aug 2026

ANCHOR-RE: An Agentic Neuro-Symbolic Framework for Grounded Biomedical Relation Extraction

arXiv:2608.03154v1 Announce Type: new Abstract: Biomedical relation extraction (BioRE) extracts structured knowledge from biomedical literature for applications such as knowledge base construction and

BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems

arXiv:2608.02612v1 Announce Type: new Abstract: Formulating an optimization problem strongly affects the quality of the final solution, yet good formulations usually require substantial expertise. Rec

ChiEngMixBench: Evaluating Large Language Models on Expert-Style Chinese-English Terminology Mixing

arXiv:2601.16217v2 Announce Type: replace-cross Abstract: Large language models increasingly mediate multilingual professional communication, where useful generation requires adapting to community con

DeepSeek V4 Flash 0731 at 10–17 t/s (nothink) on MacBook M5 Pro 64GB*, partly via SSD streaming

model-releasesr-localllama

Inspired by a post from u/giveen I motivated claude (no patinence on my side to work through everything myself) to help me get DS running on my MacBook M5 Pro 64GB and it exceeded my expectations.. be

Document OCR is not Getting Commoditized (by Frontier Models) The most common question I get is whether frontier models are going to eat all…

model-releasesjerry-liu--x

Document OCR is not Getting Commoditized (by Frontier Models) The most common question I get is whether frontier models are going to eat all document processing solutions - just screenshot the page an

GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks

arXiv:2608.03764v1 Announce Type: new Abstract: Agent self-evolution updates an agent's persistent state from prior experience and reuses it to solve related tasks more effectively. Evaluating self-ev

Agents

LiveEvalBench: Toward Open-World Evaluation for Web Generation

arXiv:2608.03689v1 Announce Type: new Abstract: Large language models are increasingly capable of synthesizing executable frontend projects, yet existing benchmarks still treat web generation as a sta

agentsarxiv-cs-ai

MMLongBench-Doc-V2: A Corrected-Annotation, Semantics-Aware Revision of MMLongBench-Doc

arXiv:2608.03397v1 Announce Type: new Abstract: MMLongBench-Doc is a long-document QA benchmark of 1,082 questions over 135 PDFs. Two properties of it push measured scores away from the quantity they

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding

arXiv:2608.03708v1 Announce Type: new Abstract: Text-to-image diffusion models enable personalization of specific visual concepts from a small number of reference images. However, generating a single

Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds

arXiv:2608.02636v1 Announce Type: cross Abstract: Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying mode

TACT: Taxonomy-Aligned Post-Training for Pedagogically Adaptive English Tutoring

arXiv:2608.03952v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to provide conversational practice for English-as-a-second-language (ESL) learners. Effective ESL tut

The production of meaning in the processing of natural language

arXiv:2603.20381v2 Announce Type: replace-cross Abstract: Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing s

Safety

Averaging Bias: Human Faithfulness Annotations are not Locally Faithful

arXiv:2608.00205v1 Announce Type: new Abstract: Evaluation of faithfulness of text summarization treats a model generated summary as faithful only if every of its sentences is supported by the source

safetyarxiv-cs-cl

Capability Provenance in Language Models: A Case Study in Social Reasoning

arXiv:2606.19625v2 Announce Type: replace Abstract: We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-r

ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces

arXiv:2608.01968v1 Announce Type: new Abstract: Transformer models are most often understood through what they do: their benchmark performance, generation quality, or behavior on downstream tasks. Yet

Control Under Compression: Reliability Frontiers for Tool-Using Agents

arXiv:2608.01056v1 Announce Type: cross Abstract: Tool-using language-model agents are governed not only by task prompts but also by persistent system-side instructions that specify tools, arguments,

Floor, Ceiling, and the Fusion Gap: How Much of Crowd Reading Attention Can Machines Predict?

arXiv:2608.01704v1 Announce Type: cross Abstract: A benchmark score means nothing without knowing what a trivial method achieves and what the best possible method could achieve. We construct both boun

Gaokerena: A Small Persian Medical Language Model Family

arXiv:2608.00932v1 Announce Type: new Abstract: The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused

HP-JEPA: Hierarchical Partitioning for Multi-Resolution Graph Joint-Embedding Predictive Learning

arXiv:2608.00491v1 Announce Type: new Abstract: Graph self-supervised learning aims to learn transferable representations from large-scale unlabeled graph data. Joint-embedding predictive architecture

Safety

Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation

arXiv:2608.00279v1 Announce Type: cross Abstract: Recent radiology-adapted vision-language models have achieved strong performance on standard report generation benchmarks, yet their robustness and ge

safetyarxiv-cs-cv

Look Up and Look Back: Hidden Attention and Latent Orientation in a Frozen Foundation Model for Panoramic SLAM

arXiv:2608.00925v1 Announce Type: new Abstract: Monocular panoramic SLAM benefits from substantial visual overlap under large camera rotations, yet remains prone to errors caused by camera tilt, scale

Machine-Precision Prediction of Low-Dimensional Chaotic Systems from Noise-Free Data

arXiv:2507.09652v2 Announce Type: replace-cross Abstract: Low-dimensional chaotic systems such as the Lorenz-63 model are commonly used to benchmark system-agnostic methods for learning dynamics from

MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing

arXiv:2608.02059v1 Announce Type: new Abstract: Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-

OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

arXiv:2608.00677v1 Announce Type: new Abstract: AI agents operate in persistent environments where early state changes can influence decisions far into the future. Unlike conventional language-model i

Qwen-CUA: Native Computer Use for (almost) Everything

arXiv:2608.02352v1 Announce Type: cross Abstract: Native computer use offers a general interface for agents to operate almost any software available to people, but requires long-horizon state tracking

Real-Time Visual Obstruction Detection in Surgical Augmented Reality

arXiv:2608.00232v1 Announce Type: new Abstract: Surgical augmented reality (AR) can provide contextual guidance by overlaying virtual annotations, tool cues, and procedural information onto the surgic

Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation

arXiv:2608.01666v1 Announce Type: new Abstract: However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open q

Research

The Label Defines the Timescale: Trait-State Limits of Temporal-Aggregate Learning

arXiv:2608.01587v1 Announce Type: cross Abstract: Machine-learning benchmarks often pair a label that aggregates a long temporal horizon with input observed through one or a few short windows. Their a

researcharxiv-cs-lg

When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification

arXiv:2608.01409v1 Announce Type: new Abstract: Biomedical fact-checking systems must do more than predict whether a claim is supported, contradicted, or unaddressed: they should also produce evidence

A robust association between LLM use and scientific productivity: Assessing stopping-time selection

arXiv:2607.28968v1 Announce Type: cross Abstract: Renault, Bergeaud, and Bosquet (hereafter RBB) argue that dating LLM adoption as the first month in which an author's abstract is flagged induces a st

ActionParty: Multi-Subject Action Binding in Generative Video Games

arXiv:2604.02330v2 Announce Type: replace-cross Abstract: Recent advances in video diffusion have enabled the development of 'world models' capable of simulating interactive environments. However, the

Agentic Harness for Real-World Compilers

arXiv:2603.20075v2 Announce Type: replace-cross Abstract: Compilers are critical to modern computing, yet fixing compiler bugs is difficult. While recent large language model (LLM) advancements enable

Appreciate it! Now let's understand the world through the eyes of Qwen3.8. 🥳

Qwen3.8-Max from Alibaba’s Qwen team achieved second place in the Vision Arena benchmark, scoring 1,305 points. It trails only Claude Fable 5 (High), which leads by a slim 13‑point margin. The post un

model-releasesqwen--x

Autonomous Repair for Multi-Agent Systems via Monte-Carlo Tree Search

arXiv:2607.29055v1 Announce Type: cross Abstract: Multi-agent systems (MAS) are increasingly deployed to solve complex tasks. In case of incorrect or unsatisfactory outputs, users have to manually loc

CAER: Conflict-Aware Evidence Routing with Dual Prefix Experts for Multimodal Large Language Models

arXiv:2607.28991v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in multimodal understanding and generation. However, when textual inp

Combining Large Language Models and Symbolic Reasoning for Multi-Robot Temporal Planning through Explainable Knowledge Bases

arXiv:2502.19135v2 Announce Type: replace Abstract: We present PLANTOR, a framework for generating and executing multi-robot task plans from natural-language task descriptions through LLM-assisted kno

COSI-Lab: Conference Living Lab for Modeling Multi-Perspective Multimodal Social Intention

arXiv:2607.28649v1 Announce Type: cross Abstract: COSI-Lab presents a multimodal, multi-sensor dataset of an interdisciplinary scientific workshop containing 32 academics at an international conferenc

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data

arXiv:2607.29120v1 Announce Type: new Abstract: Relational Prior-Data Fitted Networks (PFNs) such as RDB-PFN approximate Bayesian inference over multi-table relational databases by pretraining on mill

GO-PRE: Goal-Oriented Next-Best-View Selection via Predictive Rendering Entropy for Active 3D Reconstruction

arXiv:2607.29037v1 Announce Type: new Abstract: Active 3D reconstruction relies on active view selection to maximize reconstruction fidelity under limited capture budgets. However, most existing metho

Research

Library Reachability in LSR-Synth: How Anti-Memorization Design Changes the Measurement of Symbolic Discovery

arXiv:2607.28684v1 Announce Type: new Abstract: Existing benchmarks for scientific equation discovery are largely composed of well-known equations available in the public domain, making it difficult t

researcharxiv-cs-ai

Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures

arXiv:2607.28802v1 Announce Type: new Abstract: Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the

Multi-Agent Planning with Spatio-Temporal and Topological Constraints using STL-GO

arXiv:2607.28679v1 Announce Type: new Abstract: Multi-agent planning problems arise in a variety of engineering applications, such as multi-robot wildfire fighting and unmanned aerial inspection in fa

OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning

arXiv:2607.29401v1 Announce Type: new Abstract: Video Scene Procedure Planning (VSPP) supplies the target start-goal observations in advance, leaving open how a planner should act when the evidence mu

Qwen3.8-Max matches Kimi K3 and DeepSeek V4 Flash

model-releasesr-localllama

Qwen3.8-Max (2.4T) is another massive contribution to the open weight community. On benchmarks, it performs closely to Kimi K3 and DeepSeek V4 flash across all categories and is better at coding and s

.@ssankar says Palantir was able to make Nvidia's Nemotron Ultra model 'better than frontier': 'I literally almost felt gaslit when, within …

model-releasesclem-delangue--x

.@ssankar says Palantir was able to make Nvidia's Nemotron Ultra model 'better than frontier': 'I literally almost felt gaslit when, within 24 hours of getting Nemotron up with no post-training, this

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing

arXiv:2607.28887v1 Announce Type: cross Abstract: Large language models increasingly write and repair production code, yet evidence is mounting that their test-passing patches leave codebases harder t