Search: “benchmark”

GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping

arXiv:2603.12275v1 Announce Type: cross Abstract: Unlearning knowledge is a pressing and challenging task in Large Language Models (LLMs) because of their unprecedented capability to memorize and dige

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Human-Inspired Memory Architecture for LLM Agents

DGX agent

arXiv:2605.08538v1 Announce Type: new Abstract: Current LLM agents lack principled mechanisms for managing persistent memory across long interaction horizons. We present a biologically-grounded memory

model-releasesarxiv-cs-ai

12 May 2026

Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care

arXiv:2605.08533v1 Announce Type: new Abstract: Clinical decision-making in emergency medicine demands rapid, accurate diagnoses under uncertainty. Despite benchmark progress, evidence for LLMs as int

Improving TMS EEG Signal Quality for Closed-Loop Neuro Stimulation via Source-Domain Denoising

arXiv:2605.08184v1 Announce Type: cross Abstract: This research addresses a validated TMS EEG cleaning pipeline and a corresponding benchmark dataset. It evaluates two widely used artifact removal pip

Investigating Anisotropy in Visual Grounding under Controlled Counterfactual Perturbations

arXiv:2605.09090v1 Announce Type: cross Abstract: Visual Grounding benchmarks assume that the object described by a referring expression is always present in the image, and grounding models are theref

safetyarxiv-cs-ai

LegalCiteBench: Evaluating Citation Reliability in Legal Language Models

arXiv:2605.10186v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into legal drafting and research workflows, where incorrect citations or fabricated precedent

LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search

arXiv:2605.09764v1 Announce Type: cross Abstract: LLM-guided evolutionary methods such as AlphaEvolve have proven effective in domains like math, systems research, and algorithmic discovery, but their

Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models

arXiv:2605.10002v1 Announce Type: new Abstract: Large vision-language models (VLMs) demonstrate strong performance in medical image understanding, but frequently generate clinically plausible yet inco

MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models

arXiv:2603.18256v2 Announce Type: replace-cross Abstract: Recent reasoning-based large language models have shown strong performance on tasks with verifiable outcomes, but their use in de novo molecul

MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring

arXiv:2605.09684v1 Announce Type: cross Abstract: We introduce a red-teaming methodology that exposes harder-to-catch attacks for coding-agent monitors, suggesting that current practices may under-eli

MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding

arXiv:2605.09703v1 Announce Type: new Abstract: Understanding human mental states from natural behavior is crucial for intelligent systems in the real world. However, most current research focuses on

Multi-layer attentive probing improves transfer of audio representations for bioacoustics

arXiv:2605.10494v1 Announce Type: cross Abstract: Probing heads map the representations learned from audio by a machine learning model to downstream task labels and are a key component in evaluating r

safetyarxiv-cs-ai

MVB-Grasp: Minimum-Volume-Box Filtering of Diffusion-based Grasps for Frontal Manipulation

arXiv:2605.09672v1 Announce Type: new Abstract: State-of-the-art 6-DoF grasp generators excel on tabletop benchmarks with overhead cameras but struggle in frontal grasping scenarios on low-cost manipu

safetyarxiv-cs-ro

Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning

arXiv:2605.08660v1 Announce Type: new Abstract: In the recent literature, Support Vector Regression (SVR) has been cited as one of the weakest performers on the California Housing benchmark dataset, w

Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs

arXiv:2605.09239v1 Announce Type: new Abstract: Large language models fail at counting repeated tokens despite strong performance on broader reasoning benchmarks. These failures are commonly attribute

SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representations

arXiv:2605.09485v1 Announce Type: new Abstract: Latent representations learned by neural networks often exhibit semantic structure, where concept similarity is reflected by geometric proximity in embe

Local Ai

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

arXiv:2605.10442v1 Announce Type: cross Abstract: Multilingual studies of social bias in open-ended LLM generation remain limited: most existing benchmarks are English-centric, template-based, or rest

local-aiarxiv-cs-ai

TIDES: Implicit Time-Awareness in Selective State Space Models

arXiv:2605.09742v1 Announce Type: cross Abstract: Selective state space models (SSMs), such as Mamba, achieve strong per-token expressivity by making the time discretization step Tilde{Delta} a learne

Useful for Exploration, Risky for Precision: Evaluating AI Tools in Academic Research

arXiv:2605.10125v1 Announce Type: new Abstract: Artificial intelligence (AI) tools are being incorporated into scientific research workflows with the potential to enhance efficiency in tasks such as d

researcharxiv-cs-ai

WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors

arXiv:2605.10434v1 Announce Type: new Abstract: Commercial video generation systems such as Seedance2.0 and Veo3.1 have rapidly improved, strengthening the view that video generators may be evolving i

Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?

arXiv:2605.07937v1 Announce Type: new Abstract: Long-horizon AI agents execute complex workflows spanning hundreds of sequential actions, yet a single wrong assumption early on can cascade into irreve

CarCrashNet: A Large-Scale Dataset and Hierarchical Neural Solver for Data-Driven Structural Crash Simulation

arXiv:2605.07098v1 Announce Type: new Abstract: Crash simulation is a cornerstone of modern vehicle development because it reduces the need for costly physical prototypes, accelerates safety-driven de

Dataset Watermarking for Closed LLMs with Provable Detection

arXiv:2605.06865v1 Announce Type: new Abstract: Large language models (LLMs) are pre-trained and post-trained on vast amounts of loosely curated data, raising the possibility that these models may hav

HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

arXiv:2605.07177v1 Announce Type: cross Abstract: Existing multimodal search agents process target entities sequentially, issuing one tool call per entity and accumulating redundant interaction rounds

ModelLens: Finding the Best for Your Task from Myriads of Models

arXiv:2605.07075v1 Announce Type: new Abstract: The open-source model ecosystem now contains hundreds of thousands of pretrained models, yet picking the best model for a new dataset is increasingly in

PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphism

arXiv:2605.07796v1 Announce Type: new Abstract: SQL dialects vary in syntax, types, and functions across database engines. Text-to-SQL benchmarks, however, predominantly support only SQLite. This crea

researcharxiv-cs-cl

RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation

arXiv:2605.07760v1 Announce Type: new Abstract: Platform content moderation applies explicit policy rules and context-dependent conditions to decide whether user content is allowed, restricted, or rem

S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models

arXiv:2503.05085v2 Announce Type: replace Abstract: Recent advances in large language models (LLMs) have fundamentally reshaped speech-to-speech (S2S) systems, enabling increasingly natural spoken int

Tracing Uncertainty in Language Model 'Reasoning'

arXiv:2605.07776v1 Announce Type: cross Abstract: Language model (LM) 'reasoning', commonly described as Chain-of-Thought or test-time scaling, often improves benchmark performance, but the dynamics u

A Comparative Study of PyCaret AutoML and CNN-BiLSTM for Binary Hate Speech Detection in Indonesian Twitter

arXiv:2605.04885v1 Announce Type: new Abstract: This paper compares a PyCaret AutoML branch and a CNN-BiLSTM branch for binary hate speech detection on Indonesian Twitter using the HS label from the c

AsymmetryZero: A Framework for Operationalizing Human Expert Preferences as Semantic Evals

arXiv:2605.04083v1 Announce Type: new Abstract: Much of the focus in RL today is on evaluation design: building meaningful evals that serve simultaneously as benchmarks and as well-defined reward sign

Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization

arXiv:2603.08022v2 Announce Type: replace Abstract: A data mixture refers to how different data sources are combined to train large language models, and selecting an effective mixture is crucial for o

Implicit Representations of Grammaticality in Language Models

arXiv:2605.05197v1 Announce Type: new Abstract: Grammaticality and likelihood are distinct notions in human language. Pretrained language models (LMs), which are probabilistic models of language fitte

researcharxiv-cs-cl

Misaligned by Reward: Socially Undesirable Preferences in LLMs

arXiv:2605.05003v1 Announce Type: new Abstract: Reward models are a key component of large language model alignment, serving as proxies for human preferences during training. However, existing evaluat

safetyarxiv-cs-cl

Privacy-Preserving Empathy Detection in Video Interactions

arXiv:2504.10808v3 Announce Type: replace Abstract: Detecting empathy from video interactions has emerging applications, yet raw videos that could be used for training AI models are rarely available d

Agents

Revisiting the Travel Planning Capabilities of Large Language Models

arXiv:2605.03308v1 Announce Type: new Abstract: Travel planning serves as a critical task for long-horizon reasoning, exposing significant deficits in LLMs. However, existing benchmarks and evaluation

agentsarxiv-cs-ai

ABD: Default Exception Abduction in Finite First Order Worlds

arXiv:2602.18843v3 Announce Type: replace Abstract: We introduce ABD, a benchmark for default-exception abduction over finite first-order worlds. Given a background theory with an abnormality predicat

Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls

arXiv:2605.03147v1 Announce Type: new Abstract: Earnings calls are a key source of financial information about public companies. However, extracting information from these calls is difficult. Unlike t

RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation

arXiv:2507.00435v2 Announce Type: replace-cross Abstract: We introduce RoboEval, a structured evaluation framework and benchmark for robotic manipulation that augments binary success with principled b

Sentiment Analysis of Indonesian Spotify Reviews Using Machine Learning and BiLSTM

arXiv:2605.03443v1 Announce Type: new Abstract: This paper benchmarks classical machine learning and deep learning approaches for three-class sentiment classification of Indonesian Spotify reviews. Us

researcharxiv-cs-cl

Agents

The Conversations Beneath the Code: Triadic Data for Long-Horizon Software Engineering Agents

arXiv:2605.02244v1 Announce Type: cross Abstract: Frontier software engineering agents have saturated short-horizon benchmarks while regressing on the work that constitutes senior engineering: long-ho

agentsarxiv-cs-ai

Valley3: Scaling Omni Foundation Models for E-commerce

arXiv:2605.01278v1 Announce Type: new Abstract: In this work, we present Valley3, an omni multimodal large language model (MLLM) developed for diverse global e-commerce tasks, with unified understandi

Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts

arXiv:2605.01882v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown considerable potential in chart understanding and reasoning tasks. However, they still struggle with

Coopetition-Gym v1: A Formally Grounded Platform for Mixed-Motive Multi-Agent Reinforcement Learning under Strategic Coopetition

arXiv:2605.02063v1 Announce Type: cross Abstract: We present Coopetition-Gym v1, a benchmark platform for mixed-motive multi-agent reinforcement learning under strategic coopetition. The platform comp

Creating and Evaluating Figurative Language Dataset for Sindhi

arXiv:2605.01323v1 Announce Type: new Abstract: In this article, we introduce SiNFluD, a novel benchmark dataset for Sindhi figurative language classification. We first collect raw text from various b

EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts

arXiv:2605.02083v1 Announce Type: new Abstract: Local factual edits in scientific manuscripts often create non-local revision obligations. If a dataset changes from 215 to 80 documents, claims such as

Enhanced LLM Reasoning by Optimizing Reward Functions with Search-Driven Reinforcement Learning

arXiv:2605.02073v1 Announce Type: new Abstract: Mathematical reasoning is a key benchmark for large language models. Reinforcement learning is a standard post-training mechanism for improving the reas

GEODE: Angle-Adaptive OOD Detection with Universal Scorer Compatibility

arXiv:2605.01063v1 Announce Type: cross Abstract: Outlier Exposure (OE) is among the strongest training-based OOD detectors on standard benchmarks but exhibits scorer-dependent tradeoffs (e.g., strong

researcharxiv-cs-cv