Search: “benchmark”

Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence

arXiv:2608.06756v1 Announce Type: new Abstract: Vision-language models are increasingly serving as the reasoning core of embodied agents. Robot execution is inherently iterative: each action reshapes

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

CellWorld: From Gene-Level Reconstruction to Latent Cell Prediction in Spatial Transcriptomics Foundation Models

DGX agent

arXiv:2608.06659v1 Announce Type: new Abstract: This paper shows that latent-space predictive pretraining can provide a scalable route to foundation models for spatial transcriptomics. Existing spatia

researcharxiv-cs-ai

10 Aug 2026

Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitation

arXiv:2608.06955v1 Announce Type: new Abstract: Large language models (LLMs) are trained on corpora that contain expressions of human judgment about films, books, music, and more. Yet whether LLMs sys

Agents

Deal Me Maybe: The Role of Emotions in Multi-Agent Negotiation

arXiv:2608.06922v1 Announce Type: new Abstract: Negotiation is a demanding social task for LLM agents, requiring strategic reasoning, persuasion, and interpersonal adaptation. Yet existing benchmarks

agentsarxiv-cs-ai

Safety

Does Splitting a Triage Decision Across Agents Hide Bias or Help Catch It? A Multi-Agent Simulation Study of LLM-Based Resource Allocation Under Audit Capacity Constraints

arXiv:2608.06949v1 Announce Type: new Abstract: Prior benchmarking work has shown that a single large language model (LLM), forced to make life-or-death resource-allocation decisions, exhibits measura

safetyarxiv-cs-ai

EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revision

arXiv:2608.07196v1 Announce Type: new Abstract: Many methods for automated multi-agent system design optimize prompts and topologies during an initial design stage and then deploy the resulting system

Georeferencing Non-Gazetteered Place Names using Biological Specimen Records

arXiv:2608.06884v1 Announce Type: cross Abstract: Biological specimen records collected by natural history institutions constitute a rich source of temporal geographic knowledge, capturing biodiversit

Local Ai

Human-AI Perceptual Alignment by Playing Hues and Cues

arXiv:2608.07141v1 Announce Type: new Abstract: Evaluating the perceptual alignment between Contrastive Vision-Language Models (CVLMs) and humans is typically constrained by traditional benchmarks tha

local-aiarxiv-cs-cv

LSEAD: A Privacy-Preserving LLM-Based Speech Analysis Framework for Early Alzheimer's Disease Screening

arXiv:2608.07378v1 Announce Type: cross Abstract: Early diagnosis of Alzheimer's disease (AD) is critical for enabling timely interventions that may slow disease progression and improve patient outcom

MemWM: Memory-Augmented Text-Based World Model

arXiv:2608.07107v1 Announce Type: new Abstract: World models are increasingly used to support planning in agents by predicting how environment states evolve in response to agent actions. Yet fluent ne

PHASE-Tree: Modeling Character-State Evolution in Long-Horizon Role-Playing Dialogue

arXiv:2608.06975v1 Announce Type: cross Abstract: Long-horizon role-playing demands that characters remain recognizable as they evolve with the narrative. Yet existing work falls short on two fronts:

PHOENIX: Fine-Tuned SLM-Powered Autonomous Satellite Lifetime Extension via Predictive Self-Healing and Multi-Agent AI Recovery

arXiv:2608.07126v1 Announce Type: cross Abstract: Most CubeSats, small and low-cost satellites roughly the size of a shoebox, do not survive as long as they were designed to: a study of 178 missions f

ResidencyRL: Reinforcement Learning in Simulated Clinical Environments

arXiv:2608.07418v1 Announce Type: new Abstract: In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters,

SAGEO Arena: A Realistic Environment for Evaluating Search-Augmented Generative Engine Optimization

arXiv:2602.12187v2 Announce Type: replace-cross Abstract: Search-Augmented Generative Engines (SAGE) have emerged as a new paradigm for information access, bridging web-scale retrieval with generative

Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery

arXiv:2608.06931v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory

Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs

arXiv:2608.07014v1 Announce Type: new Abstract: Aggregate scaling curves suggest that Video LLMs improve smoothly or saturate as visual budgets grow. We show that this view can conceal large, opposing

Surg-UniWorld: A Unified Surgical World Model with Multimodal Control Experts

arXiv:2608.06770v1 Announce Type: new Abstract: Controllable surgical world models can provide a generative foundation for surgical artificial intelligence and simulation by synthesizing realistic ins

EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents

arXiv:2608.05519v1 Announce Type: new Abstract: Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic. In deployment, however, the choice among a local look

Grounded Well-Condition Anomaly Detection on the Volve Field: Constructed Labels, a Baseline, and a Dual-Head Model

arXiv:2608.05685v1 Announce Type: new Abstract: Most public benchmarks for machine-condition monitoring come from test rigs, where faults are induced on purpose and every event is known. Real producti

Research

Is Self-Pretraining really useful to improve diagnosis in medical Time Series?

arXiv:2608.06122v1 Announce Type: cross Abstract: Inspired by recent evidence that transformer architectures benefit from Self-PreTraining (SPT) on long-context benchmarks, we investigate whether simi

researcharxiv-cs-ai

Research

MirrorNet: Can Medical Image Anonymization Really Protect Patient Identity?

arXiv:2608.05938v1 Announce Type: new Abstract: Medical images are routinely de-identified---names, dates, and other metadata removed---and then shared for research, teaching, and public benchmarks un

researcharxiv-cs-cv

OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Quality

arXiv:2608.05263v1 Announce Type: new Abstract: Multi-agent orchestration frameworks are moving from demos to production, yet benchmarks typically report task accuracy without diagnosing why a pipelin

SEAM: Global consistency beyond local accuracy in scientific machine learning

model-releasesarxiv-cs-lg

arXiv:2608.05702v1 Announce Type: new Abstract: Scientific machine learning commonly validates models at the level of a subdomain, a benchmark split, or an explanation for one prediction. Yet such loc

Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings

arXiv:2608.04735v1 Announce Type: new Abstract: Chain-of-thought (CoT) monitoring is increasingly treated as an important safety layer for frontier reasoning models. Most monitorability evaluations st

Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchen

arXiv:2608.04865v1 Announce Type: new Abstract: Event cameras, also known as neuromorphic cameras, have gained significant attention in recent years due to their high temporal resolution, high dynamic

Research

Design Choices That Matter: A Functional ANOVA Analysis for Remote Sensing Multi-Label Classification

arXiv:2608.04702v1 Announce Type: cross Abstract: Benchmarking deep learning (DL) models for multi-label classification (MLC) of remote sensing images (RSI) typically yields rankings that do not gener

researcharxiv-cs-ai

FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation

arXiv:2608.04374v1 Announce Type: cross Abstract: Large language models can produce fluent financial analysis, but fluency alone does not establish whether a report is suitable for institutional deliv

GEB-Bench: Abstract Structures Told in Many Voices

arXiv:2608.04111v1 Announce Type: cross Abstract: Can a model look at a river delta and a lightning bolt and see that they share a structure? We introduce GEB-Bench, a benchmark whose unit is an abstr

Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos

arXiv:2608.04939v1 Announce Type: new Abstract: Social media videos often communicate meanings that go beyond their visible actions, captions, or speech. A mundane clip may become humorous, ironic, or

Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition

arXiv:2608.05115v1 Announce Type: cross Abstract: Can computer vision help make classrooms safer? In this pilot study, we investigate privacy-aware and computationally efficient classroom incident rec

SmartMage: Dynamic Modality Orchestration for 3D Scene Understanding

arXiv:2608.05137v1 Announce Type: new Abstract: Understanding 3D scenes is fundamental to embodied intelligence, requiring joint reasoning over heterogeneous information from multiple modalities, incl

Zero-shot reasoning for simulating scholarly peer-review

arXiv:2510.02027v2 Announce Type: replace Abstract: Scholarly publishing requires scalable scrutiny supported by auditable evidence. This paper presents a two-component benchmark of xPeer, the peer-re

ANCHOR-RE: An Agentic Neuro-Symbolic Framework for Grounded Biomedical Relation Extraction

arXiv:2608.03154v1 Announce Type: new Abstract: Biomedical relation extraction (BioRE) extracts structured knowledge from biomedical literature for applications such as knowledge base construction and

BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems

arXiv:2608.02612v1 Announce Type: new Abstract: Formulating an optimization problem strongly affects the quality of the final solution, yet good formulations usually require substantial expertise. Rec

ChiEngMixBench: Evaluating Large Language Models on Expert-Style Chinese-English Terminology Mixing

arXiv:2601.16217v2 Announce Type: replace-cross Abstract: Large language models increasingly mediate multilingual professional communication, where useful generation requires adapting to community con

GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks

arXiv:2608.03764v1 Announce Type: new Abstract: Agent self-evolution updates an agent's persistent state from prior experience and reuses it to solve related tasks more effectively. Evaluating self-ev

Agents

LiveEvalBench: Toward Open-World Evaluation for Web Generation

arXiv:2608.03689v1 Announce Type: new Abstract: Large language models are increasingly capable of synthesizing executable frontend projects, yet existing benchmarks still treat web generation as a sta

agentsarxiv-cs-ai

MMLongBench-Doc-V2: A Corrected-Annotation, Semantics-Aware Revision of MMLongBench-Doc

arXiv:2608.03397v1 Announce Type: new Abstract: MMLongBench-Doc is a long-document QA benchmark of 1,082 questions over 135 PDFs. Two properties of it push measured scores away from the quantity they

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding

arXiv:2608.03708v1 Announce Type: new Abstract: Text-to-image diffusion models enable personalization of specific visual concepts from a small number of reference images. However, generating a single

Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds

arXiv:2608.02636v1 Announce Type: cross Abstract: Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying mode

TACT: Taxonomy-Aligned Post-Training for Pedagogically Adaptive English Tutoring

arXiv:2608.03952v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to provide conversational practice for English-as-a-second-language (ESL) learners. Effective ESL tut

The production of meaning in the processing of natural language

arXiv:2603.20381v2 Announce Type: replace-cross Abstract: Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing s

Safety

Averaging Bias: Human Faithfulness Annotations are not Locally Faithful

arXiv:2608.00205v1 Announce Type: new Abstract: Evaluation of faithfulness of text summarization treats a model generated summary as faithful only if every of its sentences is supported by the source

safetyarxiv-cs-cl

Capability Provenance in Language Models: A Case Study in Social Reasoning

arXiv:2606.19625v2 Announce Type: replace Abstract: We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-r

ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces

model-releasesarxiv-cs-lg

arXiv:2608.01968v1 Announce Type: new Abstract: Transformer models are most often understood through what they do: their benchmark performance, generation quality, or behavior on downstream tasks. Yet

Control Under Compression: Reliability Frontiers for Tool-Using Agents

arXiv:2608.01056v1 Announce Type: cross Abstract: Tool-using language-model agents are governed not only by task prompts but also by persistent system-side instructions that specify tools, arguments,

Floor, Ceiling, and the Fusion Gap: How Much of Crowd Reading Attention Can Machines Predict?

arXiv:2608.01704v1 Announce Type: cross Abstract: A benchmark score means nothing without knowing what a trivial method achieves and what the best possible method could achieve. We construct both boun

Gaokerena: A Small Persian Medical Language Model Family

arXiv:2608.00932v1 Announce Type: new Abstract: The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused