Search: “benchmark”

CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution

arXiv:2608.12629v1 Announce Type: new Abstract: GPU kernel agents and GPU programming languages have advanced separately, leaving expert kernels difficult to reproduce. Agents usually treat the compil

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

CAS: A Causal Attribution Score for Local and Global Explainable Artificial Intelligence

DGX agent

arXiv:2608.12555v1 Announce Type: new Abstract: Predictive explanation methods attribute a model output; they do not, by themselves, attribute an intervention effect on the real-world outcome. We intr

model-releasesarxiv-cs-ai

14 Aug 2026

Diagnostic Foundation for Evaluating LLMs' Research Integrity as Co-Scientists

arXiv:2608.12345v1 Announce Type: new Abstract: Language models are increasingly deployed as co-scientists, yet their ability to uphold research integrity under institutional pressure remains unmeasur

Dual Spatial-Temporal Attribution: Architecture-Aligned Post-Hoc Explainability for Recurrent Graph Anomaly Detection

arXiv:2608.12441v1 Announce Type: cross Abstract: Deep learning detectors for anomalies in dynamic graphs have reached strong accuracy, yet they remain opaque: when an edge is flagged, the analyst rec

ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval

arXiv:2608.12720v1 Announce Type: cross Abstract: While Large Language Model (LLM) agents increasingly rely on long-term memory for persistent interactions, the retrieval mechanisms governing this mem

From Visual Widgets to UI Code: Efficient Tool-Grounded Generation

arXiv:2608.12611v1 Announce Type: new Abstract: Existing screenshot-to-code systems face a trade-off between flexibility and controllability. Direct multimodal generation can hallucinate visible detai

FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisition

arXiv:2608.12683v1 Announce Type: cross Abstract: Embodied agents must often identify and interact with objects based on their function rather than their identity, requiring them to actively acquire o

Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval

arXiv:2608.12843v1 Announce Type: cross Abstract: Text-based person anomaly retrieval aims to retrieve pedestrians exhibiting anomalous behaviors from a large image gallery using natural language desc

MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning

arXiv:2608.12724v1 Announce Type: new Abstract: Few-shot in-context learning (ICL) with multi-modal large language models (MLLMs) enables task adaptation without parameter updates, but its performance

Safety

Accuracy and Order Sensitivity Diverge Under Label-Free Strategies

arXiv:2608.11947v1 Announce Type: cross Abstract: Multiple-choice benchmarks are widely used to evaluate large language models, but MCQ scores conflate knowledge with sensitivity to option order, whic

safetyarxiv-cs-ai

BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents

arXiv:2511.20597v2 Announce Type: replace-cross Abstract: The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web applica

Cross-Corpus Evaluation of Generalizable Vulnerability Detection in IoT Firmware

arXiv:2608.11492v1 Announce Type: cross Abstract: IoT firmware vulnerability detection remains challenging due to heterogeneous firmware ecosystems, resource-constrained platforms, and limitations in

Research

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

arXiv:2608.11922v1 Announce Type: new Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidat

researcharxiv-cs-cl

OEIS Open: How many conjectures can language models turn into theorems?

arXiv:2608.11941v1 Announce Type: new Abstract: We construct OEIS Open, a benchmark based on 492 open mathematical conjectures from the OEIS, formalized in Lean by Tsoukalas et al. Whereas these conje

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505.03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-triv

Research

CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting

arXiv:2608.11150v1 Announce Type: new Abstract: While 3D Gaussian Splatting (3DGS) has advanced open vocabulary scene understanding, existing methods remain confined to explicit queries. They struggle

researcharxiv-cs-cv

DACRI: Decision-Aware Causal Intervention Ranking for Critical Supply Chains

arXiv:2608.11154v1 Announce Type: new Abstract: Detecting or attributing a supply-chain disruption is not the same as selecting the intervention that maximizes recoverable net value. We present Critic

FADE: From Passive Verification to Active Discovery in Counterfactual Video Understanding

arXiv:2608.10764v1 Announce Type: new Abstract: Counterfactual video understanding evaluates whether models grasp physical and commonsense regularities. However, existing multiple-choice question (MCQ

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

arXiv:2509.13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models. Whi

HNDiff: Haze-Noise Diffusion for Image Dehazing

arXiv:2608.10995v1 Announce Type: new Abstract: Existing diffusion-based methods have recently made significant progress in image dehazing. However, they typically neglect the physics of haze formatio

Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibility

arXiv:2608.10315v1 Announce Type: cross Abstract: Large language models (LLMs) are powerful black-box systems, making it difficult to discern whether their answers reflect stable internal beliefs or s

Mapping and Measuring the Behavioral Evolution of Large Language Models

model-releasesarxiv-cs-cl

arXiv:2608.11027v1 Announce Type: cross Abstract: Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across genera

No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding

model-releasesarxiv-cs-cl

arXiv:2503.05061v3 Announce Type: replace Abstract: Reliable evaluation of large language models (LLMs) is critical as their deployment rapidly expands, particularly in high-stakes domains such as bus

Nutrition Data Infrastructure for the AI Era: Operationalizing FAIR for Agent-Mediated Research

arXiv:2608.10363v1 Announce Type: new Abstract: AI agents can accelerate nutrition research, but their analyses inherit the identity, semantic, and release ambiguities of the underlying data. We prese

Optimal Stopping of Self-Refining Foundation Models

arXiv:2608.10729v1 Announce Type: cross Abstract: Foundation models can improve their outputs through a self-refinement process driven by external feedback. In this process, the model is embedded in a

Safety

Physics-Informed Machine Learning in Prognostics and Health Management: A Systematic Literature Review

arXiv:2608.10047v1 Announce Type: cross Abstract: In modern industry, keeping complex systems reliable, safe, and efficient hinges on Prognostics and Health Management (PHM). Machine Learning (ML) has

safetyarxiv-cs-ai

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608.10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interact

Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation

arXiv:2608.11191v1 Announce Type: cross Abstract: GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ab

AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images

arXiv:2608.08874v1 Announce Type: new Abstract: Spoken language offers a natural, hands-free interface for specifying an arbitrary target in dense remote-sensing imagery, yet existing referring remote

Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction

arXiv:2608.08459v1 Announce Type: cross Abstract: Practical AI systems increasingly need to turn long, heterogeneous documents into queryable relational databases, not isolated spreadsheets. In domain

Control-Diverse Reinforcement Fine-Tuning: Decoupling the Shared Control Bottleneck of RL Post-Training

arXiv:2608.08224v1 Announce Type: new Abstract: Reinforcement learning post-training unlocks complex reasoning in LLMs. Yet benchmark scores reveal only whether a model improved, not what changed insi

DarwinX: Evolving Agent Harnesses Through Natural Selection

arXiv:2608.07545v1 Announce Type: cross Abstract: An LLM agent's capability depends not only on model weights but on its harness: prompts, tools, skills, and control flow. Self-improvement loops alrea

Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

arXiv:2608.08029v1 Announce Type: cross Abstract: Khatri et al. (2026) [DOI: 10.1109/DSN-W70714.2026.00027] show that lightweight MLP probes on final-layer activations of a single 8B model (LLaMA-3.1-

Safety

Evaluation of Motivational Interviewing Counsellors with Task-Aware Multi-Stage LLM-Based Simulated Clients

arXiv:2608.07499v1 Announce Type: cross Abstract: The development and benchmarking of Large Language Model (LLM)-based Motivational Interviewing (MI) counsellors now often rely on LLM-based simulated

safetyarxiv-cs-ai

Tutorials

Experience-Sensitive Game Learning: A Behavioral Study of Humans and Language Agents

arXiv:2608.07490v1 Announce Type: cross Abstract: Large language model agents are increasingly evaluated through games, but most benchmarks emphasize final outcomes rather than how players learn from

tutorialsarxiv-cs-ai

Floating-Point Neural Network Verification at the Software Level

arXiv:2510.23389v2 Announce Type: replace-cross Abstract: The behaviour of neural network components must be proven correct before deployment in safety-critical systems. Unfortunately, existing neural

ForestBench: A Unified Graph Framework for Evaluating Multi-Agent Collaboration

arXiv:2608.08605v1 Announce Type: new Abstract: Multi-agent systems (MAS) built on Large Language Models (LLMs) are proliferating rapidly, but their heterogeneous execution traces provide no common ba

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

arXiv:2608.09435v1 Announce Type: new Abstract: Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time. Yet exi

Research

MDB-Link: Hierarchical Schema Linking for Multi-Database Text-to-SQL

arXiv:2608.09588v1 Announce Type: new Abstract: Traditional Text-to-SQL research and benchmarks assume a known target database, overlooking settings in which a query must be routed within a large, het

researcharxiv-cs-cl

Applications

Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses

arXiv:2608.07946v1 Announce Type: cross Abstract: Text-to-SQL benchmarks ship schemas whose column names already say what the columns mean. Production warehouses are the inverse: cryptic identifiers,

applicationsarxiv-cs-ai

Shattered Compositionality: Counterintuitive Learning Dynamics of Transformers for Arithmetic

arXiv:2601.22510v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often achieve strong benchmark accuracy yet remain brittle under small distribution shifts. While recent mechanis

SkillConsist: Detecting Inconsistencies in Agent Skills via Bidirectional Graph Alignment

arXiv:2608.07639v1 Announce Type: cross Abstract: Agent Skills provide reusable capabilities to LLM agents. Agent Skill inconsistencies can expose undisclosed dangerous behavior or cause wrong Skill s

SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests

arXiv:2608.08640v1 Announce Type: new Abstract: Large language model agents increasingly rely on reusable skills to extend their capabilities beyond parametric knowl- edge. However, retrieving the app

TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis?

arXiv:2608.07899v1 Announce Type: new Abstract: Agent systems increasingly expose execution traces, yet telemetry that reveals a failure may still be inadequate for identifying where that failure orig

Addressable Memory for Video World Models

arXiv:2608.07408v1 Announce Type: new Abstract: We study visual persistence in interactive video world models. These models rely on a Key-Value (KV) cache as a growing visual memory to carry forward p

Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory

arXiv:2608.07169v1 Announce Type: new Abstract: Memory systems have shown promise for improving agent performance, but their potential remains largely unexplored for small language models, which strug

Beyond Starry Night: Shortcut-Aware Control-State Planning for Artist-Grounded Text to Image Generation

arXiv:2608.06751v1 Announce Type: cross Abstract: Artist-grounded image generation requires more than appending an artist name to a prompt. Image models often respond to artist names through canonical

Beyond the Black Box: Interpretable Models of Human Randomisation Failures

arXiv:2608.07220v1 Announce Type: new Abstract: Mixed strategy equilibrium predicts i.i.d play: past actions should not help predict future decisions. Human players, however, systematically depart fro