Search: “evaluation”

ASTRA-QA: A Benchmark for Abstract Question Answering over Documents

arXiv:2605.10168v1 Announce Type: new Abstract: Document-based question answering (QA) increasingly includes abstract questions that require synthesizing scattered information from long documents or a

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

Behavior-Centric Extraction of Scenarios from Highway Traffic Data and their Domain-Knowledge-Guided Clustering using CVQ-VAE

DGX agent

arXiv:2603.16964v2 Announce Type: replace Abstract: Approval of ADS depends on evaluating its behavior within representative real-world traffic scenarios. A common way to obtain such scenarios is to e

applicationsarxiv-cs-cv

12 May 2026

Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing

arXiv:2605.10146v1 Announce Type: new Abstract: Large language models (LLMs) increasingly rely on knowledge editing to support knowledge-intensive reasoning, but this flexibility also introduces criti

ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics

arXiv:2605.09676v1 Announce Type: cross Abstract: Spatio-temporal graph neural networks (STGNNs) are widely used for short-term forecasting in dynamic physical systems such as traffic and weather. How

Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs

arXiv:2602.11181v2 Announce Type: replace Abstract: Code-mixing and code-switching (CSW) remain challenging phenomena for large language models (LLMs). Despite recent advances in multilingual modeling

safetyarxiv-cs-cl

CUDABeaver: Benchmarking LLM-Based Automated CUDA Debugging

arXiv:2605.08455v1 Announce Type: new Abstract: Debugging CUDA programs has long been challenging because failures often arise from subtle interactions among hardware behavior, compiler decisions, mem

Agents

DRIVE-C: A Controlled Corruption Dataset for Autonomous Driving

arXiv:2605.09774v1 Announce Type: new Abstract: DRIVE-C is a controlled corruption dataset designed to evaluate visual perception robustness in autonomous driving systems. It is built from real-world

agentsarxiv-cs-cv

EpiGraph: A Knowledge Graph and Benchmark for Evidence-Intensive Reasoning in Epilepsy

arXiv:2605.09505v1 Announce Type: new Abstract: Epilepsy diagnosis and treatment require evidence-intensive reasoning across heterogeneous clinical knowledge, including biosignal patterns, genetic mec

Execution Envelopes: A Shared Admission Contract for Backend AI Execution Requests

arXiv:2605.08267v1 Announce Type: cross Abstract: Enterprise AI backends increasingly admit heterogeneous execution requests across model deployment, inference, evaluation, data movement, and agentic

safetyarxiv-cs-ai

FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecasting

arXiv:2502.18834v2 Announce Type: replace-cross Abstract: Financial time series (FinTS) record the behavior of human-brain-augmented decision-making, capturing valuable historical information that can

Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs

arXiv:2605.08905v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success on reasoning benchmarks through Reinforcement Learning with Verifiable Rewards (RLVR), exc

FragileFlow: Spectral Control of Correct-but-Fragile Predictions for Foundation Model Robustness

arXiv:2605.08896v1 Announce Type: cross Abstract: Robust adaptation of LLMs and VLMs is often evaluated by average accuracy or average consistency under perturbations. However, these averages can hide

researcharxiv-cs-ai

From Pixels to Concepts: Do Segmentation Models Understand What They Segment?

arXiv:2605.09591v1 Announce Type: new Abstract: Segmentation is a fundamental vision task underlying numerous downstream applications. Recent promptable segmentation models, such as Segment Anything M

Local Ai

From Single-Step Edit Response to Multi-Step Molecular Optimization

arXiv:2605.10035v1 Announce Type: new Abstract: Conditional molecular optimization aims to edit a molecule to realize a specified property shift. In practice, structurally similar molecule data is sca

local-aiarxiv-cs-ai

Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis

arXiv:2603.09007v2 Announce Type: replace-cross Abstract: Audio deepfake detection aims to detect real human voices from those generated by Artificial Intelligence (AI) and has emerged as a significan

safetyarxiv-cs-ai

GLEAM: A Multimodal Imaging Dataset and HAMM for Glaucoma Classification

arXiv:2603.12800v2 Announce Type: replace-cross Abstract: We propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset co

researcharxiv-cs-cv

Grounded Satirical Generation with RAG

arXiv:2605.10853v1 Announce Type: new Abstract: Humor generation remains challenging task for Large Language Models (LLMs), due to their subjective nature. We focus on satire, a form of humor strongly

researcharxiv-cs-cl

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

arXiv:2605.08348v1 Announce Type: new Abstract: The circuits framework in mechanistic interpretability aims to identify causally important sparse subgraphs of model components, typically evaluated by

researcharxiv-cs-cl

Human-Inspired Memory Architecture for LLM Agents

arXiv:2605.08538v1 Announce Type: new Abstract: Current LLM agents lack principled mechanisms for managing persistent memory across long interaction horizons. We present a biologically-grounded memory

Tutorials

Inductive Entity Representations from Text via Link Prediction

arXiv:2010.03496v4 Announce Type: replace-cross Abstract: Knowledge Graphs (KG) are of vital importance for multiple applications on the web, including information retrieval, recommender systems, and

tutorialsarxiv-cs-ai

Is Your Driving World Model an All-Around Player?

arXiv:2605.10858v1 Announce Type: new Abstract: Today's driving world models can generate remarkably realistic dash-cam videos, yet no single model excels universally. Some generate photorealistic tex

Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident

arXiv:2602.01015v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly embedded in AI-based tutoring systems. Can they faithfully model novice reasoning and metacognitive ju

Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management

arXiv:2605.09818v1 Announce Type: new Abstract: Reinforcement learning (RL) in healthcare has had mixed results, with reward sparsity, unreliable off-policy evaluation, and deployment-simulation gap a

safetyarxiv-cs-lg

LogitTrace: Detecting Benchmark Contamination via Layerwise Logit Trajectories

arXiv:2509.20909v2 Announce Type: replace Abstract: Large language models (LLMs) are commonly evaluated on challenging benchmarks such as AIME and Math500, where benchmark contamination can make memor

MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks

arXiv:2507.23511v3 Announce Type: replace-cross Abstract: While large audio-language models have advanced open-ended audio understanding, they still fall short of nuanced human-level comprehension. Th

MedThink: Enhancing Diagnostic Accuracy in Small Models via Teacher-Guided Reasoning Correction

arXiv:2605.08094v1 Announce Type: cross Abstract: Accurate clinical diagnosis requires extensive domain knowledge and complex clinical reasoning capabilities. Although large language models (LLMs) hol

researcharxiv-cs-ai

ML-CLIPSim: Multi-Layer CLIP Similarity for Machine-Oriented Image Quality

arXiv:2605.09479v1 Announce Type: cross Abstract: We study full-reference image quality assessment from a machine-centric perspective, where images are evaluated by how well they preserve information

researcharxiv-cs-cv

Model-Free Neural Filtering: A Comparison with Classical Filters in Nonlinear Systems

arXiv:2601.21266v3 Announce Type: replace Abstract: Neural network models are increasingly used for state estimation in control and decision-making, yet it remains unclear to what extent they behave a

Applications

Nectar: Neural Estimation of Cached-Token Attention via Regression

arXiv:2605.09778v1 Announce Type: cross Abstract: Evaluating softmax attention over a fixed long context requires reading every cached key-value pair for each new query token. For a given context (a b

applicationsarxiv-cs-cl

NeuralBench: A Unifying Framework to Benchmark NeuroAI Models

arXiv:2605.08495v1 Announce Type: new Abstract: Deep learning and large public datasets have recently catalyzed the proliferation of AI models for processing brain recordings. However, systematically

On the Generation and Mitigation of Harmful Geometry in Image-to-3D Models

arXiv:2605.09606v1 Announce Type: cross Abstract: Recent advances in image-to-3D models have significantly improved the fidelity and accessibility of 3D content creation. Such a powerful reconstructio

safetyarxiv-cs-cv

Applications

On the Overscaling Curse of Parallel Thinking: System Efficacy Contradicts Sample Efficiency

arXiv:2601.21619v2 Announce Type: replace-cross Abstract: Parallel thinking improves LLM reasoning through multi-path sampling and aggregation. In standard evaluations, due to a lack of sample-specifi

applicationsarxiv-cs-ai

Optimal FALQON for Quantum Approximate Optimization via Layer-wise Parameter Tuning

arXiv:2605.08332v1 Announce Type: cross Abstract: Feedback-based adaptive quantum optimization (FALQON) is a promising approach for solving combinatorial problems on noisy intermediate-scale quantum (

Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning

arXiv:2605.09822v1 Announce Type: cross Abstract: We define Oracle Poisoning, an attack class in which an adversary corrupts a structured knowledge graph that AI agents query at runtime via tool-use p

PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generation

arXiv:2605.09636v1 Announce Type: new Abstract: PDE-to-solver code generation aims to automatically synthesize executable numerical solvers from partial differential equation (PDE) specifications. Thi

Process Matters more than Output for Distinguishing Humans from Machines

arXiv:2605.06524v2 Announce Type: replace Abstract: Reliable human-machine discrimination is becoming increasingly important as large language models and autonomous agents are deployed in online setti

Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions

arXiv:2605.09893v1 Announce Type: cross Abstract: Large language models (LLMs) are often evaluated based on their stated values, yet these do not reliably translate into their actions, a discrepancy t

safetyarxiv-cs-ai

Reasoning emerges from constrained inference manifolds in large language models

arXiv:2605.08142v1 Announce Type: cross Abstract: Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inf

RelBench v2: A Large-Scale Benchmark and Repository for Relational Data

arXiv:2602.12606v2 Announce Type: replace Abstract: Relational deep learning (RDL) has emerged as a powerful paradigm for learning directly on relational databases by modeling entities and their relat

Reward Auditor: Inference on Reward Modeling Suitability in Real-World Perturbed Scenarios

arXiv:2512.00920v4 Announce Type: replace Abstract: Reliable reward models (RMs) are critical for ensuring the safe alignment of large language models (LLMs). However, current RM evaluation methods fo

safetyarxiv-cs-cl

SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning

arXiv:2605.09266v1 Announce Type: new Abstract: We introduce SeePhys Pro, a fine-grained modality transfer benchmark that studies whether models preserve the same reasoning capability when critical in

SEIS: Subspace-based Equivariance and Invariance Scores for Neural Representations

arXiv:2602.04054v2 Announce Type: replace-cross Abstract: Understanding how neural representations respond to geometric transformations is essential for evaluating whether learned features preserve me

researcharxiv-cs-cv

SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models

arXiv:2605.10576v1 Announce Type: cross Abstract: Low-level visual perception underpins reliable remote sensing (RS) image analysis, yet current image quality assessment (IQA) methods output uninterpr

SleepWalk: A Three-Tier Benchmark for Stress-Testing Instruction-Guided Vision-Language Navigation

arXiv:2605.10376v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have advanced rapidly in multimodal perception and language understanding, yet it remains unclear whether they can reliabl

SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?

arXiv:2602.03916v3 Announce Type: replace-cross Abstract: Spatial reasoning is a fundamental aspect of human cognition, yet it remains a major challenge for contemporary vision-language models (VLMs).

Local Ai

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

arXiv:2605.10442v1 Announce Type: cross Abstract: Multilingual studies of social bias in open-ended LLM generation remain limited: most existing benchmarks are English-centric, template-based, or rest

local-aiarxiv-cs-ai

Sufficient conditions for a Heuristic Rating Estimation Method application

arXiv:2605.08991v1 Announce Type: new Abstract: A series of papers has introduced the Heuristic Rating Estimation method, which evaluates a set of alternatives based on pairwise comparisons and the we

researcharxiv-cs-ai

SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding

arXiv:2605.08412v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have made rapid progress in single-video understanding, yet their ability to reason across multiple independent