Search: “benchmark”

Deblurring in the Wild: A Real-World Image Deblurring Dataset from Smartphone High-Speed Videos

arXiv:2506.19445v4 Announce Type: cross Abstract: We introduce the largest real-world image deblurring dataset constructed from smartphone slow-motion videos. Using 240 frames captured over one second

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python

DGX agent

arXiv:2607.21069v1 Announce Type: new Abstract: The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penal

model-releasesarxiv-cs-lg

24 Jul 2026

Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts

arXiv:2607.20462v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly integrated into clinical workflows, stressing the need for reliable traceability of model-generated output

researcharxiv-cs-ai

MVEI & EmObserver: Empowering MLLM-Oriented Visual Emotional Intelligence via Emotion Statement Judgement

arXiv:2607.21061v1 Announce Type: new Abstract: Affective Image Content Analysis (AICA) aims to recognize and understand emotions elicited by visual content, representing an indispensable step toward

When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs

arXiv:2607.21445v1 Announce Type: new Abstract: Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this pap

A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities

arXiv:2607.19716v1 Announce Type: new Abstract: Pain is a complex and pervasive phenomenon affecting a large percentage of the population, and accurate assessment is essential for effective clinical m

Closing the Lab-to-Store Gap: A Data-Efficient Post-Training and Experience-Driven Learning VLA Framework for Retail Humanoids

arXiv:2607.20345v1 Announce Type: cross Abstract: Closing the gap between benchmark performance and reliable real-world operation remains a central challenge for Vision-Language-Action (VLA) humanoid

CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generation

arXiv:2506.10890v2 Announce Type: replace Abstract: Graphic design plays a crucial role in both commercial and personal contexts, yet creating high-quality, editable, and aesthetically pleasing graphi

MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

arXiv:2607.19235v1 Announce Type: cross Abstract: Theory of Mind (ToM), the ability to infer other's beliefs, intentions, and states of knowledge, is central to social interaction, yet remains challen

NEXUS: Structured Runtime Safety for Tool-Using LLM Agents

arXiv:2607.19356v1 Announce Type: new Abstract: Tool-using LLM agents increasingly execute high-impact actions, making runtime safety monitoring essential. We present NEXUS (Neural EXecution Utility a

Now We Know? A Systematic Comparison of TerraMind and THOR

arXiv:2607.18504v1 Announce Type: cross Abstract: Benchmarks for Geospatial Foundation Models (GFMs) increasingly rank models by aggregate score, but such rankings obscure why models differ: how much

researcharxiv-cs-cv

Solar Open 2 Technical Report

arXiv:2607.20062v1 Announce Type: new Abstract: We present Solar Open 2, a 250B-A15B Mixture-of-Experts language model built for long-horizon agentic tasks, scaled up from Solar Open 1 (Solar Open 100

Implementations of Quantum and Classical Topology-Aligned Architectures for Molecular Property Prediction

arXiv:2607.13737v1 Announce Type: new Abstract: For low-data and resource-constrained regimes typical of quantum chemistry, parameter-efficient learning is a key objective. Here, we propose a topology

Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models

arXiv:2607.13408v1 Announce Type: cross Abstract: Recent text-to-audio models generate high-quality audio, but often fail to follow instructions involving multiple sound events and temporal order. Thi

Is the Statistical Advantage Worth the Cost? An Empirical Comparison of KANs and MLPs for Structured Data Classification

arXiv:2607.13413v1 Announce Type: cross Abstract: This study presents an empirical benchmarking comparison between Kolmogorov-Arnold Networks (KANs) and Multi-Layer Perceptrons (MLPs) on structured ta

Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants

arXiv:2607.13039v1 Announce Type: cross Abstract: Safety evaluations for dual-use biology assistants often measure base-model capability, refusal behavior, or jailbreak success. These metrics miss a d

SPINE: Bridging the Cyber-Physical Gap with Agentic AI

arXiv:2607.13049v1 Announce Type: new Abstract: Foundation models have given robots a sophisticated brain for complex decision-making, yet deploying that intelligence into a physical platform still de

STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle

arXiv:2607.13618v1 Announce Type: new Abstract: LLM agents are increasingly evaluated on multi-week decision tasks in which the state that drives cost is never directly observed. On such tasks the fin

An Empirical Study for Android-to-OpenHarmony GUI Test Migration

arXiv:2607.11245v2 Announce Type: replace-cross Abstract: To reduce the substantial engineering effort required to test the corresponding applications from Android to OpenHarmony, migrating existing G

Breaking Deja Vu: Independent Auditing of Visual Place Recognition through Vision-Language Reasoning

arXiv:2607.12818v1 Announce Type: new Abstract: Visual place recognition (VPR) is a key enabler of accurate localization and long-term autonomous navigation in robotics applications, such as loop clos

Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning

arXiv:2512.17788v2 Announce Type: replace Abstract: Multi-instance partial-label learning (MIPL) is a weakly supervised framework that extends the principles of multi-instance learning (MIL) and parti

Contrastive-Collapsed Loss for Flexible and Geometrically Optimal Embeddings and Faster Convergence

arXiv:2607.12916v1 Announce Type: new Abstract: In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations. The proposed loss encourages intra-cl

Decouple and Reason: Anatomically Guided Two-Stage Voxel-Level Grounding of Free-Text Findings in 3D Chest CT

arXiv:2607.12602v1 Announce Type: new Abstract: Automatic voxel-level grounding of free-text findings in 3D chest Computed Tomography (CT) is critical for clinical interpretability. However, this task

DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for Autonomous Travel Planning Agents

arXiv:2509.21842v2 Announce Type: replace Abstract: Travel planning (TP) agent has recently worked as an emerging building block to interact with external tools/resources for travel itinerary generati

Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?

arXiv:2607.12787v1 Announce Type: new Abstract: Recent advances in multimodal large language models (MLLMs) have significantly improved the performance of multimodal emotion recognition (MER) and enab

Dynamic Resource Allocation for Ensemble Determinization MCTS

arXiv:2607.13007v1 Announce Type: new Abstract: Simulation-based algorithms are especially suited for high-uncertainty environments such as adversarial board games with significant elements of randomn

Egocentric Bias in Vision-Language Models

arXiv:2602.15892v2 Announce Type: replace-cross Abstract: Visual perspective taking--inferring how the world appears from another's viewpoint--is foundational to social cognition. We introduce FlipSet

Evidence-Grounded AI for Musculoskeletal Care

arXiv:2607.12527v1 Announce Type: new Abstract: Musculoskeletal diseases are among the leading causes of disability worldwide and create the greatest global need for rehabilitation. Because recovery,

GRID: Grammar-Railed Decoding for Enterprise SQL Generation

arXiv:2607.11951v1 Announce Type: new Abstract: Large language models can write SQL, but enterprise deployment demands more than plausible text: outputs must be syntactically valid, must respect per-r

Improved Robustness from Biologically Inspired Sparse Contrast Representations

arXiv:2509.24863v2 Announce Type: replace Abstract: Deep neural networks surpass humans on many vision benchmarks, yet remain far less robust to distribution shifts such as illumination and weather ch

researcharxiv-cs-cv

Institutional Equity Holdings Prediction Using Node Affinities of Dynamic Graphs

arXiv:2607.12067v1 Announce Type: new Abstract: Institutional equity holdings disclosed in SEC Form 13F filings provide a rich temporal record of portfolio decisions by large investment managers. Howe

KnowAct-GUIClaw: Know Deeply, Act Perfectly, Personal GUI Assistant with Self-Evolving Memory and Skill

arXiv:2607.12625v1 Announce Type: new Abstract: OpenClaw has emerged as a leading agent framework for complex task automation, yet it faces insufficient cross-platform GUI interaction support and a we

Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models

arXiv:2607.12771v1 Announce Type: cross Abstract: Reaction mechanisms consist of the step-by-step sequences of elementary reactions that explain chemical transformations. Learning the mechanism logic

Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts

arXiv:2607.12696v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) models have become an important approach for scaling Large Language Models (LLMs), but their inference efficiency depe

Optimization Is Not All You Need

arXiv:2607.11977v1 Announce Type: new Abstract: In 2019, OpenAI released two million GPT-2 outputs-ungrammatical, half broken-to aid the detection of machine-generated text. The alignment that produce

Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs

arXiv:2607.12985v1 Announce Type: new Abstract: Aligned language models routinely misreport under non-evidential incentive pressure: they agree with a confident user or overstate certainty even when t

RoboDesign1M: A Large-scale Dataset for Robot Design Understanding

model-releasesarxiv-cs-ro

arXiv:2503.06796v2 Announce Type: replace Abstract: Robot design is a complex and time-consuming process that requires specialized expertise. Gaining a deeper understanding of robot design data can en

SheetMind: An End-to-End LLM-Powered Multi-Agent Framework for Spreadsheet Automation

arXiv:2506.12339v2 Announce Type: replace-cross Abstract: We present SheetMind, a modular multi-agent framework powered by large language models (LLMs) for spreadsheet automation via natural language

TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale

arXiv:2607.13028v1 Announce Type: cross Abstract: Training robust autonomous driving agents requires a simulator that is fast enough for reinforcement learning at scale, realistic enough to ground beh

VanillaBench: The Hidden Accuracy Cost of Adversarial Robustness

arXiv:2607.12545v1 Announce Type: cross Abstract: Adversarial robustness research has produced hundreds of defended models over the past decade, yet the literature almost universally reports robustnes

WikiSTAR: A System for Shedding Light on the Hidden History of Scientific Wikipedia Articles

arXiv:2607.12441v1 Announce Type: new Abstract: Wikipedia plays a key role in shaping public understanding of science, and its openly accessible revision history is a unique record of how scientific k

Safety

Beyond Success Rates: Trainability and Extractability for Offline GCRL

arXiv:2602.05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method. This score measures a

safetyarxiv-cs-lg

Safety

Echoes: A semantically-aligned music deepfake detection dataset

arXiv:2603.23667v2 Announce Type: replace-cross Abstract: We introduce Echoes, a new dataset for music deepfake detection designed for training and benchmarking detectors under realistic and provider-

safetyarxiv-cs-ai

Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos

arXiv:2512.01803v3 Announce Type: replace Abstract: Despite rapid advances in video generative models, robust metrics for evaluating visual and temporal correctness of complex human actions remain elu

LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity

arXiv:2607.08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language

LightCrafter: PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting

arXiv:2607.08016v1 Announce Type: new Abstract: Video relighting requires balancing long-form temporal consistency with a physically grounded understanding of light transport, which depends on accurat

Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data

arXiv:2607.08522v1 Announce Type: new Abstract: The inherent rigidity of fixed-size benchmarks makes them an inefficient tool for model evaluation. Diverse evaluation objectives, including model ranki

researcharxiv-cs-lg

A Study of Commonsense Reasoning over Visual Object Properties

arXiv:2508.10956v3 Announce Type: replace-cross Abstract: Inspired by human categorization, visual reasoning about object properties, such as physical attributes and functions, involves identifying an