Search: “evaluation”

Query-Side Attacks on GNN-Based KGQA: Tracing Failures from Entity Linking to Answer Generation

arXiv:2608.25922v1 Announce Type: new Abstract: GNN-based Knowledge Graph Question Answering (KGQA) pipelines process queries through four discrete stages: entity linking, subgraph retrieval, GNN reas

27 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

The 'Curse of Knowledge' in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion

DGX agent

arXiv:2608.25245v1 Announce Type: cross Abstract: LLM-generated search queries are widely used to augment IR evaluation, yet they may contain concepts that presuppose answer-side document knowledge, v

researcharxiv-cs-cl

27 Aug 2026

Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty

arXiv:2608.25660v1 Announce Type: new Abstract: Automated novelty judgment can accelerate scientific discovery by enabling efficient evaluation, refinement, and comparison of research ideas. While lar

safetyarxiv-cs-cl

27 Aug 2026

What Do Medical Vision-Language Models Learn in Radiology? Transfer, Alignment, and Source-Proxy Leakage Under Distribution Shift

arXiv:2608.25251v1 Announce Type: new Abstract: Medical vision-language models (VLMs) can appear reliable in-domain while failing when acquisition domain, paired supervision, or evaluation protocol ch

safetyarxiv-cs-cv

27 Aug 2026

Comparative Assessment of Deep Learning Architectures for Underwater Subsurface Kelp Forest Segmentation with The Kelp-o-Tron

arXiv:2608.24594v1 Announce Type: new Abstract: Submerged kelp forests are vital coastal ecosystems that support marine biodiversity and ecosystem dynamics, yet accurate underwater kelp segmentation r

researcharxiv-cs-cv

Applications

FLARE: A Systematic, Uncertainty-Aware Framework for Evidence-Based Adoption of Artificial Intelligence in Healthcare

arXiv:2608.23643v1 Announce Type: new Abstract: Artificial intelligence is increasingly being introduced into healthcare workflows, yet most evaluations emphasize model accuracy rather than whether ad

applicationsarxiv-cs-ai

Gated Activation Steering for Reducing Sycophancy & Hallucination in Medical Question Answering

arXiv:2608.23666v1 Announce Type: new Abstract: Sycophancy and hallucination are persistent failure modes of Large Language Models (LLMs) across domains. However, it becomes particularly consequential

StateTune: Transforming LLM-Assisted EDA Flow Tuning into a Stateful, Closed-Loop Process

arXiv:2608.23601v1 Announce Type: cross Abstract: EDA flow parameter tuning is critical for quality-of-results~(QoR), yet the parameter space is large, tightly coupled, and full evaluations are prohib

SyPS: Measuring Sycophancy Prompt Sensitivity in Large Language Models

arXiv:2608.23837v1 Announce Type: new Abstract: Large language models (LLMs) are known to exhibit social sycophancy, often validating or agreeing with users in socially sensitive contexts. Existing ev

TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

arXiv:2608.24674v1 Announce Type: new Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal

BanglaVeilGuard: Cross-Script Safety Benchmarking and Lightweight Guardrails for Bangla Large Language Models

arXiv:2608.21880v1 Announce Type: new Abstract: Bangla large language model (LLM) safety is difficult to evaluate with English-centric or standard-script benchmarks because Bangla users routinely writ

Evidence-State Reliability Under Controlled Degradation: Parser-Validity Divergence in a Multi-Stage LLM Pipeline

arXiv:2608.21559v1 Announce Type: new Abstract: Multi-stage LLM pipelines can remain structurally valid even when evidence available to downstream stages becomes incomplete, compressed, or conflicting

researcharxiv-cs-cl

Measuring Activation Control in Large Language Models

arXiv:2608.21664v1 Announce Type: new Abstract: Safe deployment of increasingly capable models will likely come to rely on latent-space monitoring as a complement to behavioral evaluations, especially

Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms

arXiv:2608.22335v1 Announce Type: cross Abstract: Bengali is the seventh-most-spoken language globally, yet LLM safety evaluation remains overwhelmingly English-centric. We introduce BanglaSafe, a ben

Rethinking Communication Metrics: How Should We Measure Meaning?

arXiv:2608.21626v1 Announce Type: new Abstract: Semantic communication shifts the objective of communication systems from accurate symbol reconstruction toward meaning preservation, task accomplishmen

researcharxiv-cs-lg

Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning

arXiv:2608.23100v1 Announce Type: cross Abstract: Robot co-design via bi-level optimization couples within-lifetime controller learning for fitness evaluation with cross-generational morphological evo

safetyarxiv-cs-ai

Software Frameworks for Explainable AI in Time Series Classification: A Systematic Review

arXiv:2608.21449v1 Announce Type: new Abstract: Time series arise in a wide range of application domains and are analyzed using machine learning in decision-critical settings. Time series classificati

Text-ADBench: Text Anomaly Detection Benchmark Based on LLM Embeddings

arXiv:2507.12295v2 Announce Type: replace-cross Abstract: Text anomaly detection is a critical task in natural language processing (NLP), with applications spanning fraud detection, misinformation ide

What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces

arXiv:2608.22985v1 Announce Type: new Abstract: Activation steering is often evaluated under the answer encoding used to construct the direction. A reported gain may reflect the intended judgment or c

researcharxiv-cs-cl

A Dataset-Centric Benchmark of Deep Learning Methods for Grape Leaf Disease Classification and Detection

arXiv:2608.20608v1 Announce Type: new Abstract: Grape leaf disease recognition is important for precision agriculture, enabling early diagnosis, timely intervention, and improved vineyard management.

24 Aug 2026

Clarify-Then-Search: A Clarification Benchmark for Deep Search with End-to-End Nugget Restoration

arXiv:2608.20357v1 Announce Type: cross Abstract: Deep search is brittle on underspecified user queries: missing constraints such as time, location, scope, or definitions can lead to retrieval drift a

24 Aug 2026

OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMs

arXiv:2608.21360v1 Announce Type: new Abstract: Recent omni-modal large language models (Omni-LLMs) show great potential as real-time video assistants, which continuously perceive environments and gui

24 Aug 2026

A Standardized Framework for Machine Learning in Power System Protection

arXiv:2608.20181v1 Announce Type: cross Abstract: Studies of machine-learning-based power-system protection increasingly report near-perfect scores, yet the meaning of those scores depends strongly on

ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation

arXiv:2608.20122v1 Announce Type: new Abstract: Large multimodal models (LMMs) have demonstrated strong OCR recognition capabilities, yet remain vulnerable to adversarial visual text that is readable

LLMs as Acquisition Policies for Finite-Pool Materials Optimization: A Controlled Study

arXiv:2608.19790v1 Announce Type: new Abstract: Discovering materials with desirable properties often requires searching large candidate spaces while experimental or computational evaluations remain c

PersonalBench: Measuring the Authorship Gap in LLM Personalization

arXiv:2608.19746v1 Announce Type: new Abstract: Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference align

A FEM-Based Surrogate Modelling and Optimization Framework for Physics-Constrained Electromagnetic Coil Design

arXiv:2608.18903v1 Announce Type: new Abstract: This work evaluates surrogate-assisted optimization of a seven-parameter current-excited coil--core benchmark subject to geometric, manufacturing, and s

Automated Computational Energy Minimization of ML Algorithms using Constrained Bayesian Optimization

arXiv:2407.05788v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) is an efficient framework for optimization of black-box objectives when function evaluations are costly and gradien

LabDex: A Hierarchical Benchmark for Dexterous Manipulation in Laboratories

model-releasesarxiv-cs-ro

arXiv:2608.18618v1 Announce Type: new Abstract: Autonomous laboratories hold great promise for accelerating scientific discovery. To achieve this vision, robots are supposed to dexterously manipulate

Multimodal Rapport Estimation in Real-World HRI

arXiv:2608.18401v1 Announce Type: cross Abstract: Evaluating interaction quality in real-world HRI is an important challenge. If interaction quality can be estimated reliably, the results can be used

Agents

D^2ACCI: A Dual-Loop Diagnostic Protocol for Evidence-Preserving Agent Memory

arXiv:2608.17756v1 Announce Type: new Abstract: Memory is a key capability of LLM agents. Persistent memory extends this across sessions---enabling recall, revision, and personalization. Yet its multi

agentsarxiv-cs-ai

Do LLMs Know a Good Hypothesis When They See One? Logit-Based Energy Scoring Outperforms Prompted LLM-as-Judge for Scientific Hypothesis Ranking

arXiv:2608.17270v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for scientific hypothesis generation. However, evaluating generated hypotheses remains a challenge fo

Effects of Answer Format Variation on Gender Bias in Large Language Models

arXiv:2608.17516v1 Announce Type: new Abstract: Gender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM n

OOD Detection for EEG-based Machine Learning in High-Risk Environments

arXiv:2608.17620v1 Announce Type: new Abstract: Machine learning models for electroencephalography (EEG) analysis show great promise across a wide range of applications, but their deployment in high-r

PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX

arXiv:2608.17379v1 Announce Type: cross Abstract: We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimizati

Quantifying Memorization and Privacy Risks in Genomic Language Models

arXiv:2603.08913v2 Announce Type: replace Abstract: Genomic language models (GLMs) have emerged as powerful tools for learning representations of DNA sequences, enabling advances in variant prediction

researcharxiv-cs-lg

Hardware

Structured Driving-State Narratives for Small Language Model-Based GNSS Spoofing Detection

arXiv:2608.17092v1 Announce Type: cross Abstract: Autonomous vehicles (AVs) depend on reliable Global Navigation Satellite System (GNSS) positioning. However, spoofed GNSS signals can induce plausible

hardwarearxiv-cs-ai

Agentic Data Cleaning Without a Clean Reference: An Experimental Study of Capabilities and Trade-offs

arXiv:2608.14765v1 Announce Type: new Abstract: Data cleaning without a trusted clean reference is challenging because unusual values may represent either genuine errors or valid observations. This pa

safetyarxiv-cs-ai

Agents

Bounded Agents: Delegation Security for Multi-Agent AI Systems

arXiv:2608.15888v1 Announce Type: new Abstract: LLM-based agents can act on behalf of a user to access cloud services, call tools, or invoke agents. At session start, the agent's permissions are set b

agentsarxiv-cs-ai

CFR without Unbiasedness: Deterministic Guarantees for Persistent Public-Chance Schedules

arXiv:2608.14761v1 Announce Type: cross Abstract: At a finite public-chance cut, counterfactual regret minimization (CFR) must choose how many outcomes to evaluate before each regret update. Exact eva

DeepInsight II: One Trace from Benchmark to Robot

arXiv:2608.16556v1 Announce Type: new Abstract: Across a Physical AI stack, evaluation maturity is inversely aligned with deployment risk: foundation models enjoy mature, standardized harnesses, while

Dense Expands, Sparse Anchors: Channel-Asymmetric Query Expansion for Hybrid Retrieval

arXiv:2608.15851v1 Announce Type: cross Abstract: LLM-based query expansion improves retrieval by generating document-like passages. In hybrid retrieval, however, most evaluations fuse fixed top-L den

safetyarxiv-cs-cl

Agents

DriveCache: Action-Aware Caching for Driving World Model Inference

arXiv:2608.16354v1 Announce Type: new Abstract: Driving video generation models support autonomous-driving development by predicting controllable future scenes for simulation, planning evaluation, and

agentsarxiv-cs-ai

Foresight-England: Development of a National-Scale Generative AI Model of Electronic Health Records for Medical Event Prediction across the COVID-19 Pandemic

arXiv:2608.16273v1 Announce Type: cross Abstract: Foresight-England (Foresight-E) is the first national-scale generative foundation model of electronic health records (EHRs), developed as a research p

Generation of Synthetic Fingerphotos with GANs

arXiv:2608.15029v1 Announce Type: new Abstract: Contactless fingerprinting is an emerging approach to biometric authentication that allows users to scan their fingerprints without touching a scanner.

researcharxiv-cs-cv

Applications

Grounding Healthcare LLMs in a Causal Knowledge Graph: Framework, Metrics, and a Cardiovascular Pilot

arXiv:2608.15382v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly proposed for healthcare decision support, but their evaluations still reward single-answer accuracy rather

applicationsarxiv-cs-ai

Grounding Robot Generalization in Training Data via Retrieval-Augmented VLMs

model-releasesarxiv-cs-ro

arXiv:2603.11426v3 Announce Type: replace Abstract: Recent work on robot manipulation has advanced policy generalization to novel scenarios. However, it is often difficult to characterize how differen

PLSQLBench: Benchmarking LLM Systems for Executable Procedural Database Programming

arXiv:2608.15931v1 Announce Type: new Abstract: We present PLSQLBench, to our knowledge the first benchmark for evaluating whether LLMs can write executable PL/SQL programs, with correctness measured