Search: “evaluation”

arXiv:2606.07523v1 Announce Type: cross Abstract: Legal domains in high-resource languages like English have widely adopted artificial intelligence for legal question answering. However, data scarcity

applicationsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Reward Shaping for (Inference-Time) Alignment: A Stackelberg Game Perspective

DGX agent

arXiv:2602.02572v2 Announce Type: replace-cross Abstract: Existing alignment methods directly use the reward model learned from user preference data to optimize an LLM policy, subject to KL regulariza

safetyarxiv-cs-ai

9 Jun 2026

RTL-BenchLS: A Large-Scale Benchmark for RTL Reasoning and Generation with Large Language Models

arXiv:2606.08976v1 Announce Type: new Abstract: LLM-based RTL generation and reasoning is a promising direction for hardware design automation. High-quality benchmarks are critical infrastructure for

Tutorials

Sample-Efficient LLM-Based Detection of Malicious Web Server Logs with Forensically Explainable Reasoning

arXiv:2606.08649v1 Announce Type: cross Abstract: Forensic analysis of web server logs demands both accurate detection and human-readable explanations that can satisfy legal requirements. We present C

tutorialsarxiv-cs-ai

SecureClaw: Clawing Back Control of LLM Agents

arXiv:2606.09549v1 Announce Type: cross Abstract: Tool-using large language model (LLM) agents face two distinct security failures: unauthorized external actions and exposure of sensitive plaintext in

Self-Explainability in Self-Adaptive and Self-Organising Systems: Status and Research Directions

arXiv:2606.09568v1 Announce Type: new Abstract: The growing complexity of self-adaptive and self-organising systems, fuelled by advances in Artificial Intelligence (AI), has made them increasingly dif

Seq103: A Unified Neuroevolution Framework for Compact Sequence Architecture Discovery

arXiv:2606.07664v1 Announce Type: cross Abstract: Neuroevolution is a representative neural architecture search paradigm that evolves both network topology and weights through evolutionary algorithms.

Local Ai

Sound and Complete Neurosymbolic Reasoning with LLM-Grounded Interpretations

arXiv:2507.09751v3 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but exhibit problems with l

local-aiarxiv-cs-ai

Streaming Interventions: Can Video Large Language Models Correct Mistakes as They Occur?

arXiv:2606.09547v1 Announce Type: new Abstract: Learning everyday skills, like cooking a dish, relies increasingly on instructional media such as online videos. This opens the door to the use of video

Stress-testing medical large language models reveals latent safety pathology beyond benchmark accuracy

arXiv:2606.07929v1 Announce Type: new Abstract: Large language models (LLMs) are entering clinical practice based on benchmark accuracy that may fail to detect safety-relevant failure modes. Here we p

Structural Grid Descriptors Predict Within-Task Solver Success on ARC-AGI

arXiv:2606.09026v1 Announce Type: new Abstract: We ask whether structural properties of intermediate grid states predict whether a symbolic ARC-AGI solver will succeed, framed as a test of conditional

Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER)

arXiv:2606.07608v1 Announce Type: cross Abstract: We present a systematic study of fine-tuning OpenAI's Whisper large-v3 for Swiss German ASR, using 1,367 hours of broadcast speech paired with Standar

SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?

arXiv:2606.07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex env

Systems-Level Planning and Coordination of Truck-Drone Collaborative Delivery Networks

arXiv:2606.08738v1 Announce Type: cross Abstract: Urban last-mile parcel delivery increasingly relies on heterogeneous fleets whose performance depends on timely coordination, reliable communication,

safetyarxiv-cs-ro

Tutorials

The CIFAR Synthetic Evidence Corpus for Detecting AI-Generated Evidence

arXiv:2606.07916v1 Announce Type: new Abstract: The growing ability of generative models to produce realistic documents poses a direct challenge to evidentiary workflows in the justice system and the

tutorialsarxiv-cs-ai

The Governance of Human-LLM Interaction: Safety Gating, Civility Steering, and Affective Default Lock-In

arXiv:2606.08172v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly mediate high-stakes interactions in finance, medicine, and mental-health support, yet users have limited con

TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles

arXiv:2606.07520v1 Announce Type: cross Abstract: Instruction Following (IF) is a core capability of LLMs, requiring strict adherence to diverse constraints, ranging from verifiable ones (e.g., output

safetyarxiv-cs-lg

Titans-as-a-Layer: Test-Time Memory for Conversational Speech Emotion Recognition

arXiv:2606.08573v1 Announce Type: new Abstract: Speech emotion recognition (SER) is commonly formulated as utterance-level classification, although conversational emotion depends on a speaker's usual

Towards Long-Horizon Vessel Trajectory and Destination Forecasting with Reasoning Large Language Models

arXiv:2606.08633v1 Announce Type: new Abstract: Long-horizon maritime trajectory prediction is important for shipping management, logistics planning, and maritime risk analysis, yet month-level foreca

Towards Personalized Bangla Book Recommendation: A Large-Scale Heterogeneous Book Graph Dataset

model-releasesarxiv-cs-lg

arXiv:2602.12129v2 Announce Type: replace-cross Abstract: Personalized book recommendation in Bangla literature has been constrained by the lack of structured, large-scale, and publicly available data

Training-Free Generalized Few-Shot Segmentation through Open-Vocabulary Semantic Arbitration

arXiv:2606.09474v1 Announce Type: new Abstract: Generalized Few-Shot Semantic Segmentation (GFSS) has traditionally been approached as a representation-learning problem, requiring task-specific adapta

Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning

arXiv:2606.07631v1 Announce Type: cross Abstract: Emergent misalignment (EM) occurs when narrow finetuning causes a model to behave dangerously outside the finetuning task. Standard training signals c

VATS: Exploiting Implicit Authority in Error-Path Injection via Systematic Mutation

arXiv:2606.07992v1 Announce Type: new Abstract: As the Model Context Protocol (MCP) standardizes tool-calling for autonomous agents, it introduces a critical, unexamined attack surface: the error-hand

Vision-Language Work Zone Intelligence for Safety-Critical Speed Regulation of Mixed-Autonomy Vehicles in Dynamic Environments

arXiv:2606.08860v1 Announce Type: new Abstract: Temporary work-zone speed limits are communicated through visually inconsistent signage and are often missing from digital maps, creating safety risks f

safetyarxiv-cs-cv

What the Eyes See, the LLMs Miss: Exploiting Human Perception for Adversarial Text Attacks

arXiv:2606.09700v1 Announce Type: cross Abstract: Large language model (LLM)-powered content moderation systems have become a critical defense against harmful online content. However, these systems pr

What's the Point? Spatial Grammar & Index Resolution for Sign Language Processing

arXiv:2606.08056v1 Announce Type: cross Abstract: Sign language models are predominantly trained with gloss-sequence or text supervision, thereby under-modeling non-lexical and productive construction

When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding

arXiv:2606.08239v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains under

XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triad

arXiv:2601.14063v2 Announce Type: replace-cross Abstract: Cross-cultural competence in large language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cul

Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study

arXiv:2606.09362v1 Announce Type: new Abstract: Re-Identification (ReID) in autonomous driving is typically formulated as a visual matching problem, where observations of vehicles, pedestrians, and cy

ZIPP:Zero-shot Image Personalization from Personas

arXiv:2606.08841v1 Announce Type: new Abstract: Text-to-image diffusion models are increasingly deployed in open-ended creative contexts, yet their outputs remain impersonal, optimized for aggregate a

Accounting for Context: Shaping Moral Credences for Value Alignment

arXiv:2606.06972v1 Announce Type: new Abstract: Ensuring that agent behaviours are aligned with human moral values inevitably raises the problem of how to account for the plurality of moral perspectiv

AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPO

arXiv:2606.06828v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has demonstrated remarkable success in aligning text-to-image (T2I) flow models with human preferences. Howeve

safetyarxiv-cs-cv

Architecture-Adaptive Uncertainty Fusion for Deepfake Detection

arXiv:2606.06666v1 Announce Type: new Abstract: Deepfake detection systems achieve near-perfect accuracy on benchmarks, yet forensic deployment demands reliable prediction uncertainty. Existing uncert

researcharxiv-cs-cv

Coordinated optimization of departure sequencing and section-track allocation in railway short-term concentrated departure scenarios based on qubo and hybrid quantum algorithms

arXiv:2606.06543v1 Announce Type: cross Abstract: This study examines the coordinated optimization of departure sequencing and section-track allocation in railway short-term concentrated departure sce

CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval

arXiv:2506.11066v3 Announce Type: replace-cross Abstract: Code retrieval is essential in modern software development, as it boosts code reuse and accelerates debugging. However, current benchmarks pri

DaX: Learning General Pathology Representations Across Scales

arXiv:2606.06983v1 Announce Type: cross Abstract: Computational pathology requires visual representations that transfer across diverse clinical endpoints and remain robust to variation in magnificatio

Diagnosing Visual Ignorance in Vision-Language Models

arXiv:2606.06890v1 Announce Type: new Abstract: Vision-Language Models (VLMs) frequently rely on language priors, producing confident answers that are weakly grounded in visual evidence. While this be

Evidence-Grounded Ensemble Diagnosis of 802.11 Packet Captures: A Multi-Stage Pipeline with Deterministic Reliability Scoring

arXiv:2606.06871v1 Announce Type: new Abstract: Diagnosing 802.11 packet captures requires expert protocol knowledge, is slow, inconsistent across engineers, and unscalable. LLM-based approaches sound

safetyarxiv-cs-lg

Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software

arXiv:2602.04894v4 Announce Type: replace-cross Abstract: LLMs are increasingly used for code generation, but their outputs often follow recurring templates that can induce predictable vulnerabilities

GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratios

arXiv:2606.06967v1 Announce Type: new Abstract: Generative policies provide expressive and multimodal action distributions, making them attractive for reinforcement learning (RL) in complex continuous

safetyarxiv-cs-lg

Agents

IDDMBSE: Integrating Data-Driven and Model-Based Systems Engineering for Trusted Autonomous Cyber-Physical Systems

arXiv:2606.06727v1 Announce Type: new Abstract: Autonomous cyber-physical systems (CPS) sit at the intersection of Model-Based Systems Engineering (MBSE) and data-driven Machine Learning and Artificia

agentsarxiv-cs-ro

LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization

arXiv:2603.02970v2 Announce Type: replace Abstract: We introduce LAGO, a LocAl-Global Optimization framework coupling Bayesian Optimization (BO) and gradient-based trust region local refinement throug

M^3Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions

model-releasesarxiv-cs-cl

arXiv:2606.07402v1 Announce Type: new Abstract: Language agents are increasingly deployed over accumulating multimodal information, yet existing benchmarks assume a human-human form with sparse visual

MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring

arXiv:2606.06754v1 Announce Type: cross Abstract: We present MADRAG, a training-free framework for analytic essay scoring that combines multi-agent reasoning with retrieval-augmented grounding. Unlike

safetyarxiv-cs-cl

Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation

model-releasesarxiv-cs-lg

arXiv:2606.07387v1 Announce Type: new Abstract: State-of-the-art text-to-music generation systems rely on massive proprietary datasets and industrial-scale compute, making it impossible to disentangle

Agents

Measuring Agents in Production

arXiv:2512.04123v4 Announce Type: replace-cross Abstract: LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments

agentsarxiv-cs-ai

MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models

arXiv:2606.06696v1 Announce Type: cross Abstract: Vision and language models (VLMs) hold immense promise to transform biomedical imaging workflows, from detecting lesions in chest X-rays to profiling

Applications

Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis

arXiv:2606.06867v1 Announce Type: new Abstract: Modern medicine relies on heterogeneous data sources spanning radiology, pathology, text reports, and structured clinical information. However, real-wor

applicationsarxiv-cs-cv