Search: “evaluation”

SPECTRA: Synthetic IR Test Collections with Relevance Oracles and Controlled Distractor Diagnostics

arXiv:2605.31575v1 Announce Type: cross Abstract: Scalable information retrieval testing needs corpora that are large enough to stress index construction, ranking latency, query routing, and evaluatio

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Synthetic Stimuli, Real Gains: Rethinking VLM Fine-Tuning Through Fully Controlled Data Generation

DGX agent

arXiv:2511.11440v3 Announce Type: replace-cross Abstract: Performance gains of Vision Language Models (VLMs) obtained by fine-tuning are generally based on ad hoc data collection and annotation of rea

safetyarxiv-cs-cl

1 Jun 2026

Targeted Speaker Poisoning Framework in Zero-Shot Text-to-Speech

arXiv:2603.07551v2 Announce Type: replace-cross Abstract: Zero-shot Text-to-Speech (TTS) voice cloning poses severe privacy risks, demanding the removal of specific speaker identities from trained TTS

Local Ai

The Long-Term Effects of Data Selection in LLM Fine-Tuning

arXiv:2605.30537v1 Announce Type: new Abstract: Data selection is increasingly used to reduce the cost of large language model (LLM) fine-tuning, with recent methods prioritizing samples by current ut

local-aiarxiv-cs-lg

The Surface You Test Is Not the Surface That Breaks

arXiv:2605.30454v1 Announce Type: cross Abstract: Tool-augmented LLM agents are vulnerable to prompt injection: a third party who controls part of the agent's context can plant instructions that the a

Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows

arXiv:2605.31452v1 Announce Type: new Abstract: Building on our previous work, this paper develops practical, low-barrier methods for freelance translators and smaller language service providers to ev

TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices

arXiv:2605.31113v1 Announce Type: new Abstract: Automatically detecting machine-generated text (MGT) is critical to maintaining the knowledge integrity of user-generated content (UGC) platforms such a

Your Multimodal Speech Model Says I Have a Face for Radio

arXiv:2605.30472v1 Announce Type: new Abstract: As large neural models have become better at language tasks, researchers are increasingly building multi- and omnimodal models that handle more modaliti

A Geometric View of SRC: Learning Representations for Stable Residual Inference

arXiv:2605.29673v1 Announce Type: cross Abstract: Reconstruction-based inference assigns a class by comparing class-wise reconstruction residuals; Sparse Representation Classification (SRC) is a canon

safetyarxiv-cs-cv

A Progress-Aware Leader-Follower Midair Docking System for Dual-Drone Aerial Manipulation

model-releasesarxiv-cs-ro

arXiv:2605.29410v1 Announce Type: new Abstract: Reliable midair docking between small unmanned aerial vehicles (UAVs) is essential for modular aerial cooperation and manipulation, but it requires prec

ActTraitBench: Quantifying the Knowledge-Decision Gap in Large Language Models via Human-Grounded Behavioral Validation

arXiv:2605.29791v1 Announce Type: new Abstract: While Large Language Models (LLMs) can convincingly simulate personas in explicit self-reports, they often deviate in implicit behavioral decisions, rev

safetyarxiv-cs-cl

Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization

arXiv:2605.29396v1 Announce Type: new Abstract: Safety alignment for large language models (LLMs) aims to reduce harmful or unsafe behavior while preserving general utility. However, recent findings r

Agents

An Approach for Thyroid Nodule Analysis Using Thermographic Images

arXiv:2605.29221v1 Announce Type: new Abstract: Thyroid cancer is said to be the second most common type of cancer in female individuals and the third in males by 2030, according to projections. In ge

agentsarxiv-cs-cv

Auditing Training-Free 3D Shape Retrieval with Diffused Geodesic Moments

arXiv:2605.29004v1 Announce Type: new Abstract: Reported retrieval scores for training-free shape descriptors conflate local signal design, normalization, aggregation, codebook fitting, and metric cho

Bayesian model selection and misspecification testing in imaging inverse problems only from noisy and partial measurements

arXiv:2510.27663v3 Announce Type: replace-cross Abstract: Modern imaging techniques heavily rely on Bayesian statistical models to address difficult image reconstruction and restoration tasks. This pa

researcharxiv-cs-lg

Benchmarking Single-Factor Physical Video-to-Audio Generation

arXiv:2605.30339v1 Announce Type: new Abstract: Generative video-to-audio (V2A) models produce highly plausible soundtracks, but it remains unclear whether they capture the underlying physical process

Beyond Transcripts: A Renewed Perspective on Audio Chaptering

arXiv:2602.08979v2 Announce Type: replace-cross Abstract: Audio chaptering, the task of segmenting long-form audio into coherent sections, is increasingly important for navigating podcasts, lectures,

researcharxiv-cs-cl

BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models

arXiv:2605.30226v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for grounding visual-language understanding into real-world robotic manipulat

safetyarxiv-cs-ai

Bridging the Sim-to-Real Gap in Reinforcement Learning-Based Industrial Dispatching through Execution Semantics

arXiv:2605.29078v1 Announce Type: new Abstract: Event-driven scheduling policies are increasingly deployed in industrial environments, where decisions are made under asynchronous and partially observe

safetyarxiv-cs-ai

CommunityFact: A Dynamic, Multilingual, Multi-domain Benchmark for Misinformation Detection in the Wild

arXiv:2605.30241v1 Announce Type: new Abstract: Misinformation verification increasingly occurs in public, fast-moving, and multilingual online settings, where static benchmarks provide an incomplete

Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence

arXiv:2605.29054v1 Announce Type: cross Abstract: Coding agents increasingly act as codebase-scale collaborators that can assist with codebase conversion, but this progress has exposed a critical weak

CrystalXRD-Bench: Benchmarking Vision-Language Models for XRD Peak Indexing Across Diverse Crystalline Materials

arXiv:2605.29446v1 Announce Type: new Abstract: Miller-index identification from powder XRD patterns requires capabilities untested by existing multimodal benchmarks: the model must read a narrow peak

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning

arXiv:2508.19202v3 Announce Type: replace Abstract: Scientific problem solving poses unique challenges for LLMs, requiring both deep domain knowledge and the ability to apply such knowledge through co

Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking

arXiv:2605.30107v1 Announce Type: new Abstract: Creating spoken dialogue datasets is methodologically challenging, and these challenges are amplified when the goal is to build multilingual, multi-para

Do Language Models Track Entities Across State Changes?

arXiv:2605.30233v1 Announce Type: cross Abstract: Entity tracking (ET), the ability to keep track of states, is a fundamental skill that underlies complex reasoning. An increasing amount of work inves

DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark

arXiv:2605.30027v1 Announce Type: new Abstract: Multimodal documents contain diverse elements, such as tables, figures, and layouts, which can complicate retrieval tasks. While current approaches typi

Efficient, Validation-Free Intrinsic Quality Estimation for Large-Scale Face Recognition Datasets

arXiv:2605.29720v1 Announce Type: new Abstract: We propose Intrinsic Quality (IQ), a validation-free metric designed to estimate the inherent potential of face recognition (FR) datasets to produce hig

researcharxiv-cs-cv

Applications

Enhancing Reinforcement Learning in 3D Environments through Semantic Segmentation: A Case Study in ViZDoom

arXiv:2511.11703v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) in 3D environments with high-dimensional sensory input poses two major challenges: (1) the high memory consumption

applicationsarxiv-cs-ai

Fairness-Aware Federated Learning with Trajectory Shapley Value

model-releasesarxiv-cs-lg

arXiv:2605.30336v1 Announce Type: new Abstract: Federated learning is an emerging distributed paradigm that addresses the challenges posed by heterogeneous, privacy-sensitive data. It enables multiple

Frontier LLM-based agents can overcome the ontology curation bottleneck for natural phenotypes

arXiv:2605.28965v1 Announce Type: new Abstract: Linking free-text phenotype descriptions to ontology terms, typically referred to as phenotype annotation, is essential for the cross-study integration

Getting to the Point: Pointing Improves LVLMs at Counting

arXiv:2603.21746v2 Announce Type: replace Abstract: Pointing-based methods decompose complex tasks as sequential grounding and reasoning steps. Given a query, the model first grounds the relevant obje

researcharxiv-cs-cv

Applications

GMOS: Grounding Moving Object Segmentation in 3D Space and Time

arXiv:2605.30352v1 Announce Type: new Abstract: Moving Object Segmentation (MOS) aims to discover, segment, and track objects that move independently of the camera. Current MOS methods, however, exhib

applicationsarxiv-cs-cv

Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learning

arXiv:2602.01058v2 Announce Type: replace-cross Abstract: Post-training of reasoning LLMs is a holistic process that typically consists of an offline SFT stage followed by an online reinforcement lear

GTA: Generating Long-Horizon Tasks for Web Agents at Scale

arXiv:2605.29218v1 Announce Type: new Abstract: Web agents, which couple language models with browsing and tool-use capabilities, show promise as open web assistants. Yet progress is increasingly limi

HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering

arXiv:2512.24562v2 Announce Type: replace Abstract: Large language models (LLMs) achieve strong question answering (QA) performance but can produce fluent answers unsupported by available evidence. Ex

researcharxiv-cs-cl

HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?

arXiv:2605.30058v1 Announce Type: new Abstract: While LLM agents have demonstrated remarkable task-oriented abilities such as planning, reasoning, and action, few works have treated them as complete h

Agents

Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interaction

arXiv:2605.29960v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly leverage long term memory to support persistent and autonomous task execution. However, this capability

agentsarxiv-cs-ai

How LoRA Remembers? A Parametric Memory Law for LLM Finetuning

arXiv:2605.30260v1 Announce Type: cross Abstract: Large Language Models (LLMs) must continuously learn and update knowledge to remain effective in dynamic real-world environments. While Low-Rank Adapt

Agents

Human-in-the-Loop Swarms: A Bionic Swarm Approach to Real-World Soil Mapping

arXiv:2605.29091v1 Announce Type: new Abstract: Swarm and field robotics face significant barriers to real-world validation due to the high cost and development time to deploy hardware. This paper int

agentsarxiv-cs-ro

Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content

arXiv:2605.29245v1 Announce Type: cross Abstract: This paper presents a survey and taxonomy of LLM fingerprinting and watermarking for identity, ownership verification, provenance, and generated-conte

researcharxiv-cs-cl

InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data Agents

arXiv:2511.22884v2 Announce Type: replace Abstract: Data analysis has become an indispensable part of scientific research. To discover the latent knowledge and insights hidden within massive datasets,

Large-Scale AI and Foundation Models for Neuroscience: A Comprehensive Review

arXiv:2510.16658v3 Announce Type: replace Abstract: The development of large-scale artificial intelligence (AI) models is influencing neuroscience research by enabling end-to-end learning from raw bra

Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding

model-releasesarxiv-cs-lg

arXiv:2511.04934v3 Announce Type: replace Abstract: Unlearning in large language models (LLMs) is critical for regulatory compliance and for building ethical generative AI systems that avoid producing

Learning Representations from 3D Gaussian Splats

arXiv:2605.29549v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) is a recent approach for scene rendering. Although primarily designed for view synthesis, its potential for scene understan

LLMSurgeon: Diagnosing Data Mixture of Large Language Models

arXiv:2605.30348v1 Announce Type: cross Abstract: The pretraining data mixture of Large Language Models (LLMs) constitutes their 'digital DNA', shaping model behaviors, capabilities, and failure modes

LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback

arXiv:2605.30273v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empathy, and s

safetyarxiv-cs-ai

Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selection

arXiv:2605.30274v1 Announce Type: cross Abstract: Document-level translation remains one of the most challenging tasks for large language models, which are constrained by limited context windows that

MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data

arXiv:2605.29647v1 Announce Type: new Abstract: Aerial navigation on Mars requires vision-based pipelines that are robust to the diverse illumination conditions and terrain morphology of the Martian s

researcharxiv-cs-cv