Search: “benchmark”

Polite on the Surface, Wrong in Practice: A Curated Dataset for Fixing Honorific Failures in Multilingual Bangla Generation

arXiv:2605.22487v1 Announce Type: new Abstract: Recent advances in Multilingual Large Language Models (MLLMs) have significantly enhanced cross-lingual conversational capabilities, yet modeling cultur

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SADGE: Structure and Appearance Domain Gap Estimation of Synthetic and Real Data

DGX agent

arXiv:2605.22467v1 Announce Type: new Abstract: We propose SADGE, a quantitative similarity metric that predicts the performance of synthetic image datasets for common computer vision tasks without do

model-releasesarxiv-cs-cv

22 May 2026

Agents

SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations

arXiv:2605.22564v1 Announce Type: new Abstract: Today, tool-calling agents are commonly evaluated or tested on static datasets of execution traces, including input commands, agent responses, and assoc

agentsarxiv-cs-cl

Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation

arXiv:2605.21491v1 Announce Type: cross Abstract: As language models accelerate scientific research by automating hypothesis generation and implementation, a new bottleneck emerges: evaluating and fil

Industry

Thanks for sharing this. The Huggingface Hub, instead of being considered as a huge dataset, should be considered as a dynamic discovery eng…

Thanks for sharing this. The Huggingface Hub, instead of being considered as a huge dataset, should be considered as a dynamic discovery engine that can be directly executed and verified. Most models

industryclem-delangue--x

Visual-Advantage On-Policy Distillation for Vision-Language Models

arXiv:2605.21924v1 Announce Type: new Abstract: On-policy knowledge distillation has proven effective for language models, yet its application to vision-language models (VLMs) remains underexplored. W

Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations

arXiv:2602.19320v2 Announce Type: replace Abstract: Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and pe

AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals

arXiv:2605.20643v1 Announce Type: cross Abstract: Self-distillation enables language models to learn on-policy from their own trajectories by using the same model as both student and teacher, with the

safetyarxiv-cs-cl

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

arXiv:2605.20247v1 Announce Type: cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs). Although Mi

DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditions

arXiv:2605.20680v1 Announce Type: new Abstract: Human Action Recognition (HAR) is a fundamental computer vision task with diverse real-world applications. Practical deployments often involve low-light

Do LLMs Know What Luxembourgish Borrows? Probing Lexical Neology in Low-Resource Multilingual Models

arXiv:2605.21227v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for writing assistance in small contact languages, yet it is unclear whether they respect community n

Hyper-V2X: Hypernetworks for Estimating Epistemic and Aleatoric Uncertainty in Cooperative Bird's-Eye-View Semantic Segmentation

arXiv:2605.21309v1 Announce Type: new Abstract: Cooperative perception enabled by Vehicle-to-Everything (V2X) communication enhances autonomous driving safety by creating a unified environmental repre

PGC: Peak-Guided Calibration for Generalizable AI-Generated Image Detection

arXiv:2605.21207v1 Announce Type: new Abstract: The rapid evolution of generative AI, from GANs to modern diffusion models, has resulted in increasingly subtle discriminative clues. These fine-grained

PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction

arXiv:2605.21414v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation by leveraging large pretrained vision-languag

safetyarxiv-cs-cv

Pseudo-Formalization for Automatic Proof Verification

arXiv:2605.20531v1 Announce Type: cross Abstract: Reliable verification of proofs remains a bottleneck for training and evaluating AI systems on hard mathematical reasoning. Fully formal proofs, in la

Quantum reservoir computing in Jaynes-Cummings models: Nonlinear memory and time-series prediction

arXiv:2510.00171v2 Announce Type: replace-cross Abstract: We investigate quantum reservoir computing (QRC) using a hybrid qubit-boson system described by the Jaynes-Cummings (JC) Hamiltonian and its d

RadProPoser: Probabilistic Radar Tensor Human Pose Estimation That Knows Its Limits

arXiv:2508.03578v2 Announce Type: replace Abstract: Radar-based human pose estimation enables privacy-preserving motion tracking for ambient intelligence, yet the noisy nature of radar sensing makes u

Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis

arXiv:2605.20277v1 Announce Type: new Abstract: Medical vision-language models (VLMs) have rapidly advanced as general-purpose multimodal assistants, yet their deployment in 3D Computed Tomography (CT

safetyarxiv-cs-cv

Reinforcing Human Behavior Simulation via Verbal Feedback

arXiv:2605.20506v1 Announce Type: cross Abstract: Humans learn social norms and behaviors from verbal feedback (e.g., a parent saying 'that was rude' or a friend explaining 'here's why that hurt'). Ye

roto 2.0: The Robot Tactile Olympiad

arXiv:2605.21429v1 Announce Type: cross Abstract: Tactile-based reinforcement learning (RL) is currently hindered by fragmented research and a focus on over-saturated orientation tasks. We introduce v

Semiparametric Efficient Bilevel Gradient Estimation

arXiv:2605.21341v1 Announce Type: cross Abstract: Functional bilevel methods estimate a lower-level function and plug it into a hypergradient, but this plug-in gradient can retain first-order bias whe

Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining

arXiv:2605.20296v1 Announce Type: new Abstract: Fine-tuning a language model for a target task routinely degrades capabilities the training data never explicitly threatened. We study this phenomenon,

Applications

TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis

arXiv:2510.06063v2 Announce Type: replace-cross Abstract: Modern enterprises generate vast streams of time series metrics when monitoring complex systems, known as observability data. Unlike conventio

applicationsarxiv-cs-lg

The Economics of Model Collapse: Equilibrium, Welfare, and Optimal Provenance Subsidies in Synthetic Data Markets

arXiv:2605.20279v1 Announce Type: cross Abstract: Generative artificial intelligence is rapidly transforming the supply side of training data: an increasing share of new tokens, images, and structured

TimeSRL: Generalizable Time-Series Behavioral Modeling via Semantic RL-Tuned LLMs -- A Case Study in Mental Health

arXiv:2605.21295v1 Announce Type: new Abstract: Longitudinal passive sensing enables continuous health prediction, yet models often fail under cross-dataset distribution shifts. Traditional ML overfit

Verifiable Provenance and Watermarking for Generative AI: An Evidentiary Framework for International Operational Law and Domestic Courts

arXiv:2605.21002v1 Announce Type: cross Abstract: Generative artificial intelligence now synthesizes photorealistic imagery, audio, and video at a cost that defeats traditional forensic intuition. The

VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment

arXiv:2605.21130v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) have shown promise for video quality assessment, but most methods still predict an absolute score for each video. Such po

Adversarial Stress Testing of SPARK Humanoid Safety Filters

model-releasesarxiv-cs-ro

arXiv:2605.19009v1 Announce Type: new Abstract: Humanoid robots are difficult to deploy safely because they have high-dimensional bodies, many collision constraints, and must operate near people and o

AgentNLQ: A General-Purpose Agent for Natural Language to SQL

arXiv:2605.19010v1 Announce Type: new Abstract: Natural language to SQL (NL2SQL) conversion is an important problem for researchers and enterprises due to the ubiquitous importance of relational datab

An LLM-Based System for Argument Mining

arXiv:2605.13793v2 Announce Type: replace Abstract: Arguments are a fundamental aspect of human reasoning, in which claims are supported, challenged, and weighed against one another. We present an end

Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning

arXiv:2605.19852v1 Announce Type: new Abstract: Tool-augmented reasoning has emerged as a promising direction for enhancing the reasoning capabilities of multimodal large language models (MLLMs). Howe

Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos

arXiv:2605.18984v1 Announce Type: new Abstract: Recent video generative models have greatly improved the realism of AI-generated videos, yet their outputs still exhibit artifacts such as temporal inco

Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negatives

arXiv:2605.19771v1 Announce Type: cross Abstract: Existing imitation learning methods for end-to-end autonomous driving predominantly learn from successful demonstrations by minimizing geometric devia

Cross-View Attention Fusion Net: A Prior-Guided Dual-View Representation Learning for Cardiac Output Estimation from Short-Term PPG Signals

arXiv:2605.19666v1 Announce Type: cross Abstract: Accurate cardiac output (CO) estimation from photoplethysmography (PPG) is promising for unobtrusive hemodynamic monitoring, but remains difficult sin

Differential-Integral Neural Operator for Long-Term Turbulence Forecasting

arXiv:2509.21196v3 Announce Type: replace-cross Abstract: Accurately forecasting the long-term evolution of turbulence represents a grand challenge in scientific computing and is crucial for applicati

Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning

arXiv:2605.18871v1 Announce Type: cross Abstract: When Large Language Models produce structured outputs such as travel plans, code solutions, or multi-step proofs, individual reasoning steps may appea

Explainable Wastewater Digital Twins: Adaptive Context-Conditioned Structured Simulators with Self-Falsifying Decision Support

arXiv:2605.19826v1 Announce Type: new Abstract: Operators of safety-critical industrial processes increasingly rely on digital twins to screen control interventions, but such simulators rarely carry c

Research

GOAL: Graph-based Objective-Aligned Diffusion Solvers for Dynamic Multi-Objective Optimization

arXiv:2605.19119v1 Announce Type: cross Abstract: Existing neural combinatorial optimization solvers frame solution search as imitation of optimal decisions, inherently limiting their utility to singl

researcharxiv-cs-ai

GoTTA be Diverse: Rethinking Memory Policies for Test-Time Adaptation

arXiv:2605.19890v1 Announce Type: new Abstract: Test-time adaptation (TTA) enables a pre-trained model to adapt online to an unlabeled test stream under distribution shift. While most TTA research foc

HeadRank: Decoding-Free Passage Reranking via Preference-Aligned Attention Heads

arXiv:2604.17237v2 Announce Type: replace-cross Abstract: Decoding-free reranking methods that read relevance signals directly from LLM attention weights offer significant latency advantages over auto

safetyarxiv-cs-ai

Lying Is Just a Phase: The Hidden Alignment Transition in Language Model Scaling

arXiv:2605.18838v1 Announce Type: cross Abstract: Scaling laws predict loss from compute but not how capabilities interact. We measure the coupling between reasoning and truthfulness across 63 base mo

Research

M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models

arXiv:2605.18774v1 Announce Type: cross Abstract: In long, multi-page industrial documents, retrieval-augmented generation (RAG) depends heavily on whether chunk boundaries follow the document's true

researcharxiv-cs-ai

Measuring Safety Alignment Effects in Autonomous Security Agents

arXiv:2605.19722v1 Announce Type: cross Abstract: Do stock safety-aligned language models and their uncensored or abliterated derivatives behave differently when run as autonomous security agents? Sin

No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models

arXiv:2603.25722v2 Announce Type: replace Abstract: Contrastive vision-language (V&L) models remain a popular choice for various applications. However, several limitations have emerged, most notably t

OpenCompass: A Universal Evaluation Platform for Large Language Models

arXiv:2605.19276v1 Announce Type: new Abstract: In recent years, the field of artificial intelligence has undergone a paradigm shift from task-specific small-scale models to general-purpose large lang

PixVerve: Advancing Native UHR Image Generation to 100MP with a Large-Scale High-Quality Dataset

arXiv:2605.20147v1 Announce Type: new Abstract: Text-to-Image (T2I) models have recently seen notable progress around 1K and 2K resolution. With the extreme desire for better visual experience and the

Research

Projecting Latent RL Actions: Towards Generalizable and Scalable Graph Combinatorial Optimization

arXiv:2605.19721v1 Announce Type: new Abstract: Graph combinatorial optimization (GCO) has attracted growing interest, as many NP-hard problems naturally admit graph formulations, yet their combinator

researcharxiv-cs-ai

ReacTOD: Bounded Neuro-Symbolic Agentic NLU for Zero-Shot Dialogue State Tracking

arXiv:2605.19077v1 Announce Type: cross Abstract: Task-oriented dialogue systems -- handling transactions, reservations, and service requests -- require predictable behavior, yet the moderately-sized