Search: “evaluation”

Search Discipline for Long-Horizon Research Agents

arXiv:2606.11522v1 Announce Type: new Abstract: Autoresearch agents now propose, evaluate, and select scientific candidates against a metric, and that metric is usually an aggregate reduced over a het

11 Jun 2026

Understanding Cross-Sensor Feature Variations for Generalizable 3D Perception

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2606.11573v1 Announce Type: new Abstract: Radar-camera BEV perception often suffers from degraded performance when evaluated across datasets, as changes in driving scenes, sensor configurations,

researcharxiv-cs-cv

11 Jun 2026

uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking

arXiv:2606.11945v1 Announce Type: new Abstract: This report describes our participation in SemEval-2026 Task 8 on multi-turn retrieval and question answering. The task evaluates conversational systems

applicationsarxiv-cs-cl

11 Jun 2026

A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks

arXiv:2606.10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for year

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

arXiv:2606.09864v1 Announce Type: cross Abstract: Key-value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measu

Benchmarking Knowledge Editing using Logical Rules

arXiv:2606.10554v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed in real-world applications that require access to up-to-date knowledge. However, retraining LLM

Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agents

arXiv:2606.10315v1 Announce Type: cross Abstract: LLM-as-judge is the default instrument for evaluating conversational agents, yet its reliability is almost always reported as agreement with human rat

Constructing coherent spatial memory in LLM agents through graph rectification

arXiv:2510.04195v2 Announce Type: replace Abstract: Given a map description through global traversal navigation instructions, an LLM can often infer the implicit spatial layout and answer user queries

Disjoint or Overlapping? Inference Windowing for Reconstruction-Based Time Series Anomaly Detection

arXiv:2606.09874v1 Announce Type: new Abstract: Reconstruction-based methods are widely used for time series anomaly detection, where models are trained to reconstruct subsequences, and anomalies are

FG-Attn: Leveraging Fine-Grained Sparse Attention in Video Diffusion Models

arXiv:2509.16518v2 Announce Type: replace Abstract: Using diffusion transformers for media generation may require evaluating attention over extremely long sequences, with attention layers accounting f

researcharxiv-cs-cv

Importance-Aware Scheduling for High-Dimensional Hyperparameter Optimization

arXiv:2606.10068v1 Announce Type: cross Abstract: Hyperparameter Optimization (HPO) is essential for building high-performing ML/DL models, yet conventional optimizers often struggle in high-dimension

Janus: A Benchmark for Goal-Conditioned Information Distortion in LLMs

arXiv:2606.10852v1 Announce Type: cross Abstract: LLM deception is often evaluated through direct markers such as fabricated claims, explicit lies, or strategic concealment. However, many real-world m

Limitations of Learning Tanh Neural Networks with Finite Precision

arXiv:2606.11104v1 Announce Type: new Abstract: We investigate limitations of learning anh neural networks from point evaluations under finite-precision computations and L^p accuracy guarantees, build

researcharxiv-cs-lg

Mechanistic Analysis of Alignment Algorithms in Language Models

arXiv:2606.09850v1 Announce Type: cross Abstract: Post-training alignment algorithms are predominantly evaluated as black boxes, obscuring how they reshape language models' internal computations. We p

safetyarxiv-cs-cl

NOVA: Symbolic Regression Discovery of Interpretable Car-Following and Lane-Change Models with Driver Heterogeneity

arXiv:2606.10583v1 Announce Type: cross Abstract: We present NOVA, an autonomous symbolic regression framework that identifies interpretable car-following and lane-change structures from raw trajector

OncoTraj: a public benchmark for longitudinal resistance prediction in EGFR-mutant non-small-cell lung cancer on osimertinib

arXiv:2606.11144v1 Announce Type: new Abstract: Resistance to first-line osimertinib in EGFR-mutant non-small-cell lung cancer (NSCLC) is the canonical example of predictable clonal evolution under th

POPSICLE: Benchmark Datasets for Segmentation and Localization in CryoET

model-releasesarxiv-cs-cv

arXiv:2606.10255v1 Announce Type: cross Abstract: Cryo-electron tomography (cryoET) has emerged as a powerful tool in structural and cellular biology by enabling direct visualization of macromolecular

Sources: Microsoft is restricting employees from using Claude Fable 5 because of Anthropic's new 30-day data retention requirements (Tom Warren/The Verge)

Tom Warren / The Verge: Sources: Microsoft is restricting employees from using Claude Fable 5 because of Anthropic's new 30-day data retention requirements — Microsoft's legal teams are evaluating Ant

model-releasestechmeme

SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech

arXiv:2606.06037v2 Announce Type: cross Abstract: Large audio language models (LALMs) are increasingly deployed in real-world applications, yet their safety alignment is still primarily evaluated on m

safetyarxiv-cs-cl

TaCarla: A comprehensive benchmarking dataset for end-to-end autonomous driving

arXiv:2602.23499v4 Announce Type: replace-cross Abstract: Collecting a high-quality dataset is a critical task that demands meticulous attention to detail, as overlooking certain aspects can render th

V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions

arXiv:2512.11995v2 Announce Type: replace-cross Abstract: While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in

VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models

arXiv:2606.10568v1 Announce Type: new Abstract: Vision-language-action (VLA) models have shown strong promise for robotic manipulation, but their reliability at test time remains limited by one-shot a

applicationsarxiv-cs-ro

What Demonstration Curation Metrics Do to Your Policy

arXiv:2606.10229v1 Announce Type: cross Abstract: We study whether demonstration-curation metrics that detect defective training episodes also improve the downstream behavior-cloning policy that train

AI Scientists Are Only as Good as Their Evidence: A Stratified Ablation of Proprietary Data and Reasoning Skills in Drug-Asset Valuation

arXiv:2606.09556v1 Announce Type: new Abstract: AI Scientist agents are often evaluated as if capability were mainly a function of model quality, prompting, or reasoning scaffolds. We test a different

ATM: Action-Consistency Transfer Matrix for Diagnosing and Improving Latent World Models

arXiv:2606.09028v1 Announce Type: cross Abstract: Latent world models are increasingly used for control and goal-conditioned planning, yet assessing whether their learned representations are useful fo

AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

arXiv:2606.07643v1 Announce Type: cross Abstract: Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their a

Beyond Accuracy: Interpreting Topic Representation in Suicide Ideation Detection Models

arXiv:2606.07714v1 Announce Type: cross Abstract: Suicide ideation detection models are typically evaluated using aggregate performance metrics, yet little is known about how they internally represent

safetyarxiv-cs-ai

Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems

arXiv:2606.08285v1 Announce Type: new Abstract: Large language models (LLMs) and agentic systems are increasingly proposed for financial trading, yet their reported performance remains difficult to co

Beyond Pass/Fail: Using Process Mining to Understand How LLMs Resist (and Fail) Red Team Attacks

arXiv:2606.07833v1 Announce Type: cross Abstract: Standard AI red teaming evaluations reduce adversarial campaigns to a single binary outcome, attack success rate (ASR), not taking into account the se

CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design

arXiv:2603.13431v3 Announce Type: replace-cross Abstract: Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years,

CTS-Bench: Benchmarking Graph Coarsening Trade-offs for GNNs in Clock Tree Synthesis

arXiv:2602.19330v2 Announce Type: replace Abstract: Graph Neural Networks (GNNs) are increasingly explored for physical design analysis in Electronic Design Automation, particularly for modeling Clock

Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

arXiv:2512.01467v2 Announce Type: replace Abstract: Controlling autonomous systems under real-world conditions often requires policies that can be evaluated with low latency and minimal energy consump

agentsarxiv-cs-lg

DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving

model-releasesarxiv-cs-cv

arXiv:2606.08525v1 Announce Type: new Abstract: Reward models play a pivotal role in reinforcement learning (RL) and multi-modal trajectory selection for autonomous driving. However, acquiring such re

Echo-Memory: A Controlled Study of Memory in Action World Models

arXiv:2606.09803v1 Announce Type: new Abstract: We present extbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos fr

researcharxiv-cs-cv

ERBench: A Benchmark and Testsuite for Equation Discovery Algorithms

arXiv:2606.09276v1 Announce Type: new Abstract: Equation discovery aims to automate the discovery of scientific models in the form of mathematical equations from data. Technically, equation discovery

Executable World Models for ARC-AGI-3 in the Era of Coding Agents

arXiv:2605.05138v2 Announce Type: replace Abstract: We evaluate an initial coding-agent system for ARC-AGI-3 in which the agent maintains an executable Python world model, verifies it against previous

Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method

arXiv:2606.08090v1 Announce Type: cross Abstract: Evaluating a natural-language yes/no predicate over a document corpus under an accuracy target - the semantic filter - is a cornerstone of LLM-based d

safetyarxiv-cs-ai

From 0-to-1 to 1-to-N: Reproducible Engineering Evidence for MetaAI Recursive Self-Design

arXiv:2606.09663v1 Announce Type: new Abstract: Recursive self-design refers to AI-assisted modification of the mechanisms by which an AI system is built, evaluated, and improved. This paper treats Me

How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap

arXiv:2606.08594v1 Announce Type: new Abstract: Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model ca

IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation

model-releasesarxiv-cs-cv

arXiv:2601.04498v2 Announce Type: replace-cross Abstract: Infographics are composite visual artifacts that combine data visualizations with textual and illustrative elements to communicate information

Larch: Learned Query Optimization for Semantic Predicates

arXiv:2606.07923v1 Announce Type: cross Abstract: With the advent of Large Language Models (LLMs), many database systems introduced semantic operators that enabled analytical queries over unstructured

applicationsarxiv-cs-ai

LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty

arXiv:2503.18314v5 Announce Type: replace-cross Abstract: We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding r

applicationsarxiv-cs-ai

MedicalRec: Medical recommender system for image classification without retraining

arXiv:2606.07553v1 Announce Type: cross Abstract: The emergence of machine learning and deep learning has revolutionized the efficiency of diagnostic, therapeutic, and administrative systems in health

applicationsarxiv-cs-ai

No Free Lunch for Synthetic Images under Data Scarcity Conditions

arXiv:2606.07640v1 Announce Type: cross Abstract: This study investigates the trade-offs between fidelity, privacy, and utility in synthetic data generation under conditions of data scarcity and priva

PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

arXiv:2606.08106v1 Announce Type: new Abstract: Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small h

Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models

arXiv:2602.16061v2 Announce Type: replace-cross Abstract: Estimating population quantities such as mean outcomes from user feedback is fundamental to platform evaluation and social science, yet feedba

researcharxiv-cs-lg

Principled Agent Debate: Adversarial Arbitration for Sycophancy Reduction in Large Language Models

arXiv:2606.07532v1 Announce Type: cross Abstract: RLHF-trained models are systematically biased toward agreement over accuracy, a structural property of the training process. We present Principled Age

Sci-Rho: A Multilingual Visually-Grounded Symbolic Benchmark for STEM Problems

arXiv:2606.08034v1 Announce Type: cross Abstract: Symbolic benchmarks have emerged as a key approach to assess model robustness under minor modifications to STEM-related questions. However, existing s