Search: “benchmark”

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say

arXiv:2606.00152v1 Announce Type: cross Abstract: LLM-based agents are rapidly advancing, autonomously invoking external tools to complete multi-step tasks for users. However, agents often acquire mor

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ProtStructQA: A Denotation Threshold in Protein Structural Reasoning

DGX agent

arXiv:2606.00451v1 Announce Type: new Abstract: Protein-language systems are often evaluated by whether they generate plausible biological text, but a structural question has a sharper semantics: it d

model-releasesarxiv-cs-cl

2 Jun 2026

Query-Limited Community Recovery in Stochastic Block Models

arXiv:2606.02055v1 Announce Type: cross Abstract: We study exact community recovery in the two-community stochastic block model on n vertices under limited and noisy access to network data. The learne

RealityTest: How People Probe AI Identity and Whether Models Disclose It

arXiv:2606.00168v1 Announce Type: new Abstract: AI systems are increasingly deployed in conversational settings where users may be uncertain whether they are speaking with a human or an AI. Despite mo

RescueBench: Can Embodied Agents Save Lives in the Wild ?

arXiv:2606.01848v1 Announce Type: new Abstract: Search-and-rescue (SAR) requires embodied agents to explore unfamiliar environments under multimodal uncertainty, perform multi-stage interactions, and

Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints

arXiv:2604.01841v2 Announce Type: replace Abstract: Clinical prediction from structured electronic health records (EHRs) is challenging due to high dimensionality, heterogeneity, class imbalance, and

Applications

Robust Asynchronous Planning via Auto-Formalization

arXiv:2606.00981v1 Announce Type: new Abstract: LLMs can plan by either generating action sequences directly as a Planner or translating tasks into domain specific language for an external solver as a

applicationsarxiv-cs-cl

Safety

SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment

arXiv:2606.02530v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, termed the alignment tax. Existing methods mitigate t

safetyarxiv-cs-ai

Sandboxed Coding Agents are Competitive Omni-modal Task Solvers

arXiv:2606.00579v1 Announce Type: new Abstract: As multimodal LLMs increasingly target video and audio, it is often assumed that such tasks require native omnimodal models. We show that this is not al

Scaling Pre-training to One Hundred Billion Data for Vision Language Models

arXiv:2502.07617v2 Announce Type: replace Abstract: We provide an empirical investigation of the potential of pre-training vision-language models on an unprecedented scale: 100 billion examples. We fi

researcharxiv-cs-cv

SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents

arXiv:2606.02302v1 Announce Type: cross Abstract: Autonomous LLM agents increasingly operate in stateful environments where they access tools, files, memory, and external services. While such capabili

See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation

arXiv:2603.09292v2 Announce Type: replace-cross Abstract: Measurement of task progress through explicit, actionable milestones is critical for robust robotic manipulation. This progress awareness enab

SentGuard: Sentence-Level Streaming Guardrails for Large Language Models

arXiv:2606.02041v1 Announce Type: new Abstract: Large language models increasingly stream long, reasoning-intensive responses in real time, making when to moderate as critical as whether to moderate.

SindBERT, the Sailor: Charting the Seas of Turkish NLP

arXiv:2510.21364v2 Announce Type: replace Abstract: Transformer models have revolutionized NLP, yet many morphologically rich languages remain underrepresented in large-scale pre-training efforts. Wit

SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction

arXiv:2606.02540v1 Announce Type: new Abstract: Agent skills occupy a privileged position in the agent workflow, as agents are expected to implicitly follow and execute them, rendering third-party ski

SkyShield: Occupancy as a Safety Interface for Low-Altitude UAV Autonomy

arXiv:2606.00747v1 Announce Type: cross Abstract: For low-altitude Unmanned Aerial Vehicle (UAV) autonomy, 3D spatial understanding is not merely a perception objective, but the safety interface betwe

SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence

arXiv:2606.02380v1 Announce Type: cross Abstract: As LLM-based agents expand their operational scope, reliability becomes a prerequisite for real-world deployment. However, in practical applications,

StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning

arXiv:2606.00148v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) often know the rule but pick the wrong answer: on abstract visual reasoning (AVR) tasks, a model can describe

T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining

arXiv:2606.00673v1 Announce Type: new Abstract: Thermal imaging offers a powerful alternative to visible-spectrum vision under challenging conditions such as low illumination and adverse weather, yet

researcharxiv-cs-cv

Taming System Complexity: Demystifying Software Engineering Agents in Diagnosing Linux Kernel Faults

arXiv:2505.19489v2 Announce Type: replace Abstract: The Linux kernel is a critical system, serving as the foundation for numerous systems. Bugs in the Linux kernel can cause serious consequences, affe

Test-Time Training for Zero-Resource Dense Retrieval Reranking

arXiv:2606.01070v1 Announce Type: cross Abstract: Dense retrievers excel at first-stage candidate generation but lack effective reranking in zero-resource settings. Existing approaches face a fundamen

researcharxiv-cs-ai

Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designs

arXiv:2606.00592v1 Announce Type: new Abstract: Effective visual communication stems from the harmony of multiple design principles, such as readability, contrast, alignment, overlap, and coherence, w

Toward accurate RUL and SoH estimation using reinforced graph-based physics-informed neural networks enhanced with dynamic weights

arXiv:2507.09766v2 Announce Type: replace-cross Abstract: Accurate estimation of Remaining Useful Life (RUL) and State of Health (SoH) is essential for reliable Prognostics and Health Management (PHM)

TVIR: Building Deep Research Agents Towards Text--Visual Interleaved Report Generation

arXiv:2606.02320v1 Announce Type: new Abstract: Deep Research Agents have shown strong capability in multi-step information retrieval, reasoning, and long-form report generation, but existing benchmar

Understanding LLM Behavior in Multi-Target Cross-Lingual Summarization

arXiv:2606.01252v1 Announce Type: cross Abstract: Multi-target cross-lingual text summarization (MTXLS), which summarizes a source document into multiple target languages, is increasingly important as

Verifying Meta-Awareness via Predictive Rewards in Reasoning Models

arXiv:2510.03259v2 Announce Type: replace-cross Abstract: Recent research on reasoning models explores the meta-awareness of language models, including their ability to determine optimal thinking dura

researcharxiv-cs-ai

Agents

VideoBrain: Learning Adaptive Frame Sampling for Long Video Understanding

arXiv:2602.04094v2 Announce Type: replace Abstract: Long-form video understanding remains challenging for Vision-Language Models (VLMs) due to the inherent tension between computational constraints an

agentsarxiv-cs-cv

Vision-language Models for Driver Monitoring Systems: A Driver Activity Description Dataset

arXiv:2606.02273v1 Announce Type: new Abstract: Understanding subtle driver actions is essential for building reliable driver monitoring systems. Existing visionlanguage models (VLMs) are trained on g

VLBM: Variational Latent Basis Modeling for OOD Robust Multivariate Time Series Forecasting

arXiv:2606.02138v1 Announce Type: cross Abstract: Out of distribution (OOD) events in multivariate time series forecasting are rare but often dominate real world risk, making average case forecasting

WAON: A Large-Scale Japanese Image-Text Dataset for Cultural Adaptation in Contrastive Vision-Language Models

arXiv:2510.22276v3 Announce Type: replace-cross Abstract: Contrastive vision-language models have achieved remarkable progress through large-scale pretraining. Recent work has shown that removing Engl

When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning

arXiv:2602.08236v2 Announce Type: replace-cross Abstract: Despite rapid progress in MLLMs, visual spatial reasoning remains unreliable when correct answers depend on how a scene would appear under uns

When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems

arXiv:2606.00448v1 Announce Type: cross Abstract: LLM agents increasingly rely on community-contributed skills that expand an agent's operational capability set. We study a core safety problem in agen

Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories

arXiv:2606.02060v1 Announce Type: new Abstract: Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final ans

WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World

arXiv:2512.10958v2 Announce Type: replace Abstract: Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fa

X-Stream: Exploring MLLMs as Multiplexers for Multi-Stream Understanding

arXiv:2606.02482v1 Announce Type: new Abstract: While video streaming understanding has made significant strides, real-world applications, such as live sports broadcasting, autonomous driving, and mul

Auto-Discovery-Bench: Diagnosing Structured State Tracking in Oracle-Guided Discovery

arXiv:2502.15224v2 Announce Type: replace-cross Abstract: Interactive discovery requires agents to maintain and update structured beliefs over many rounds of feedback. Before evaluating agents in nois

Automating Formal Verification with Reinforcement Learning and Recursive Inference

arXiv:2605.30914v1 Announce Type: new Abstract: Automated formal verification remains challenging for large language models because data for proof assistants and verification-aware languages is scarce

Bandwidth Allocation with Device Partitioning for Federated Learning over Industrial IoT networks

arXiv:2605.30892v1 Announce Type: new Abstract: We consider a federated learning (FL) system in which Industrial Internet-of-Things (IIoT) devices collaboratively train a global model over wireless ch

Beyond Agreement: Scoring Panel-Surfaced Biomedical Entity Candidates for Curator Triage

arXiv:2605.30826v1 Announce Type: cross Abstract: Biomedical NER is deceptively simple for modern LLMs: plausible biomedical mentions are easy to surface, but corpus-convention correctness depends on

Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs

arXiv:2605.30611v1 Announce Type: cross Abstract: Scientific figures are among the most effective means of communicating complex research ideas, yet producing publication-quality illustrations remains

Error Amplification Limits ANN-to-SNN Conversion in Continuous Control

arXiv:2601.21778v2 Announce Type: replace-cross Abstract: Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANN

Evaluating Factual Density in Multi-Source RAG: A Study in Medical AI Accuracy

arXiv:2605.31506v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) is the current industry standard for grounding AI in real-world facts. Traditional retrieval methods rely on keyw

Go-UT-Bench: A Fine-Tuning Dataset for LLM-Based Unit Test Generation in Go

arXiv:2511.10868v2 Announce Type: replace Abstract: Training data imbalance poses a major challenge for code LLMs. Most available data heavily over represents raw opensource code while underrepresenti

How Far Can You Grow? Characterizing the Extrapolation Frontier of Graph Generative Models for Materials Science

arXiv:2602.09309v2 Announce Type: replace-cross Abstract: Every generative model for crystalline materials harbors a critical structure size beyond which its outputs become unreliable; we call this th

MLIPilot: LLM-Driven Auto-Research for Machine-Learned Interatomic Potentials

arXiv:2605.30889v1 Announce Type: cross Abstract: Constructing production-quality machine-learned interatomic potentials (MLIPs) requires balancing accuracy, dynamical stability, and computational thr

Safety

Multi-Agent Teams Hold Experts Back

arXiv:2602.01011v4 Announce Type: replace-cross Abstract: Multi-agent LLM systems are increasingly deployed as autonomous collaborators, where agents interact freely rather than execute fixed, pre-spe

safetyarxiv-cs-ai

Not All Synthetic Data Is Yours to Learn From

arXiv:2605.31126v1 Announce Type: cross Abstract: Can a language model improve from plain text sampled from itself, with no prompts, no teacher, no verifier, and no reward model? Yes, but only when th

Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration

arXiv:2605.31196v1 Announce Type: cross Abstract: Safe human--robot collaboration requires more than visual description: a monitor must determine whether the robot body is safely separated, already co