Search: “evaluation”

One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders

arXiv:2606.13610v2 Announce Type: replace-cross Abstract: Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM reco

25 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Evaluating Neural Cartographic Relief Shading for Urban Environments: A Downtown Calgary Study Using High-Resolution DEM and DSM Data

DGX agent

arXiv:2608.20149v1 Announce Type: new Abstract: This article explores the performance of analytical and neural-based hillshading methods in a dense urban environment using high-resolution digital elev

model-releasesarxiv-cs-lg

21 Aug 2026

A Framework for Using and Evaluating LLMs as Surrogate Experts in Security Surveys: Reliability, Bias, and Implications

arXiv:2608.16893v1 Announce Type: cross Abstract: Expert surveys are widely used in security research to study practitioner workows and decision-making, yet recruiting domain experts - especially in S

safetyarxiv-cs-ai

19 Aug 2026

Evaluating Multimodal LLMs across Text and Audio Modalities for Accessible Disaster Assistance

arXiv:2608.14651v1 Announce Type: new Abstract: Effective disaster risk communication is a foundational humanitarian challenge, yet current emergency infrastructure fails to meet the needs of individu

Evaluating the impact of adversarial traffic patterns on vanet communication using veins simulation

arXiv:2608.14583v1 Announce Type: cross Abstract: Vehicular Ad Hoc Networks (VANETs) are a key component of intelligent transportation systems, enabling real-time communication between vehicles. Howev

researcharxiv-cs-lg

MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios

arXiv:2505.21333v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved considerable accuracy in Optical Character Recognition (OCR) from static images. However, the

Predicting, Evaluating, and Explaining Top Misinformation Spreaders via Archetypal User Behavior

arXiv:2608.16323v1 Announce Type: cross Abstract: The spread of misinformation on social networks poses a significant challenge to online communities and society at large. Not all users contribute equ

researcharxiv-cs-lg

RigidBench: Evaluating Rigid-Body Physics in Video Generation Models

arXiv:2608.15555v1 Announce Type: new Abstract: Video models are increasingly used to predict what happens next in a scene, yet the metrics commonly used to compare their outputs say little about whet

PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessment

arXiv:2608.14284v1 Announce Type: cross Abstract: Fine-grained robotic evaluation matters for understanding embodied models, going beyond binary success rates and rule-based process scores. We present

17 Aug 2026

A Probe Direction Is a Property of Its Prompt

model-releasesarxiv-cs-lg

arXiv:2608.13329v1 Announce Type: new Abstract: A model that behaves differently when it senses it is being tested would undermine the evaluations we rely on, so recent work has sought to read that se

14 Aug 2026

Evaluating AlphaEarth Foundations Embeddings for Wildfire Susceptibility Mapping

arXiv:2608.12663v1 Announce Type: cross Abstract: Wildfire susceptibility mapping typically relies on physical variables assembled from multiple remote-sensing, climate, and geospatial products. Alpha

applicationsarxiv-cs-lg

14 Aug 2026

NetlistBench: Evaluating LLM Reliability in SPICE Netlist Recognition and Manipulation

arXiv:2608.12197v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in circuit design workflows, yet their reliability on simulator-facing SPICE netlist recognition an

13 Aug 2026

Evaluating Semantic and Spatial Guidance for Foundation Model Segmentation of Small-Scale PV in Remote Sensing Imagery

arXiv:2608.10801v1 Announce Type: new Abstract: Spatio-temporal PV data are essential for understanding adoption processes in off-grid regions, yet such data remain largely unavailable. Automated segm

researcharxiv-cs-cv

12 Aug 2026

STEMMA: An Adversarial Multi-Agent Framework for Evaluating Self-Identity Consistency in LLMs

arXiv:2608.08164v1 Announce Type: cross Abstract: Knowledge Distillation is a widely adopted technique in the training and fine-tuning of large language models (LLMs) enabling transfer of structured i

11 Aug 2026

Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspective

arXiv:2509.21945v2 Announce Type: replace-cross Abstract: To efficiently tune configuration for better software system performance (e.g., latency) at the deployment and maintenance stage, many tuners

10 Aug 2026

SkillEval: Decomposing Agent Skill Quality into Interpretable Signals

arXiv:2608.06891v1 Announce Type: new Abstract: Agent skills provide reusable procedural knowledge that helps agents solve specialized tasks. As their use expands, evaluating skill quality becomes inc

10 Aug 2026

C^3PO: Evaluating Cross-Modal Composition and Counterfactual Performance in Omnimodal Models

arXiv:2608.05381v1 Announce Type: new Abstract: Current Multimodal Large Language Models (MLLMs) can process diverse sensory inputs, yet their reasoning remains heavily biased toward a dominant modali

OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Quality

arXiv:2608.05263v1 Announce Type: new Abstract: Multi-agent orchestration frameworks are moving from demos to production, yet benchmarks typically report task accuracy without diagnosing why a pipelin

TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMs

arXiv:2608.05616v1 Announce Type: new Abstract: Despite the remarkable progress of large vision language models (LVLMs), object hallucination remains a fundamental challenge that hinders their trustwo

researcharxiv-cs-cv

When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories

arXiv:2608.06057v1 Announce Type: new Abstract: Tool-calling agents infer task state from accumulated dialogue and tool traces. In persistent interactions, however, historical traces may remain struct

Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness

arXiv:2608.04519v1 Announce Type: new Abstract: Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Curre

6 Aug 2026

Searching for Sound-Meaning Collisions: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation at CLEF 2026 JOKER Task 2

arXiv:2608.04299v1 Announce Type: new Abstract: Fifteen years ago, Low proposed that pun translators should stop searching for equivalent words and instead search for new points of contact between sou

researcharxiv-cs-cl

6 Aug 2026

When Absence Is Evidence: Evaluating Completeness-Sensitive Negative Reasoning in Large Language Models

arXiv:2608.04591v1 Announce Type: cross Abstract: Large language models (LLMs) are often asked whether something is absent from a record, list, or retrieved context. Yet non-observation licenses a neg

6 Aug 2026

ChiEngMixBench: Evaluating Large Language Models on Expert-Style Chinese-English Terminology Mixing

arXiv:2601.16217v2 Announce Type: replace-cross Abstract: Large language models increasingly mediate multilingual professional communication, where useful generation requires adapting to community con

5 Aug 2026

Evaluating LLM Trade-offs for Enterprise Automation: Lessons from Workflow Generation in a Production Enterprise Platform

arXiv:2608.03311v1 Announce Type: cross Abstract: Enterprise compliance management requires rapid adaptation to evolving regulatory frameworks (e.g., DORA, AI RMF, FedRAMP) and tight remediation SLAs.

5 Aug 2026

CENTILE: A Telemetry Foundation Model Evaluated by the Decisions It Drives

arXiv:2608.01725v1 Announce Type: cross Abstract: Modern computing and networking infrastructure emits telemetry continuously, yet operators convert it into decisions with a separate predictor per tas

researcharxiv-cs-lg

Evaluating Forecasting Techniques for Hardware Errors on a Large-scale HPC System

arXiv:2608.01648v1 Announce Type: new Abstract: Hardware error logs in high-performance computing (HPC) systems provide early signals of abnormal behavior, yet there remain challenges in effectively f

applicationsarxiv-cs-lg

VLAGuard: A Framework for Evaluating and Mitigating Physical Attention Hijacking in Vision-Language-Action Robots within Wireless Sensor Networks

arXiv:2608.01028v1 Announce Type: new Abstract: Deploying Vision-Language-Action (VLA) robots as mobile edge nodes within wireless sensor networks (WSNs) requires robust protection against physical ad

safetyarxiv-cs-ro

When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification

arXiv:2608.01409v1 Announce Type: new Abstract: Biomedical fact-checking systems must do more than predict whether a claim is supported, contradicted, or unaddressed: they should also produce evidence

Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models

model-releasesarxiv-cs-lg

arXiv:2607.27386v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refi

31 Jul 2026

Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator wins

arXiv:2607.09306v3 Announce Type: replace Abstract: Behavioural auditing asks whether a language model behaves as it claims, but detection scores are reported without separating two targets: whether a

31 Jul 2026

Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations

arXiv:2601.16651v3 Announce Type: replace Abstract: Gradient-based methods for instance-based explanation for large language models (LLMs) are hindered by the immense dimensionality of model gradients

31 Jul 2026

CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition

arXiv:2607.25294v1 Announce Type: cross Abstract: Real-world tasks often require models to learn from task-specific context rather than relying only on pre-trained knowledge. While recent work has hig

Evaluating Communicative Belief Updates in Large Language Models via Implicature Recognition and Cancellation

arXiv:2607.25094v1 Announce Type: cross Abstract: Human language is driven by unspoken beliefs and belief updates, making these critical to model for successful communication between large language mo

Inspect India Evals: An Open Benchmarking Framework for Evaluating Large Language Models in the Indian Linguistic and Cultural Context

arXiv:2607.25375v1 Announce Type: new Abstract: India is a vast nation of over 1.4 billion people, varied by hundreds of diverse and locally specific traditions and cultures and 22 officially recogniz

What Gets Lost When Memory Becomes Media? Evaluating AI-Generated Oral History Visualization

arXiv:2607.24756v1 Announce Type: cross Abstract: What gets lost when memory becomes media? Diaspora oral-history interviews require a double transformation; first-person recollection to third-person

Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

arXiv:2607.24601v1 Announce Type: cross Abstract: Background: Large language models (LLMs) are increasingly used to automate code review, but the reasoning behind their decisions remains hard to under

applicationsarxiv-cs-ai

Evaluating the Impact of Reviewer Guideline Design on LLM-Based Automated Peer Review

arXiv:2607.22553v1 Announce Type: cross Abstract: Peer review is an essential process in scientific research, yet the growing workload has made its automation increasingly necessary. In this study, we

Music-Source-Separation-Training (MSST): A Unified Framework for Training and Evaluating Music Demixing Models

arXiv:2607.23395v1 Announce Type: cross Abstract: Music Source Separation (MSS), the task of recovering individual sound components (stems) from a polyphonic mixture, is central to applications rangin

applicationsarxiv-cs-lg

Plans Work in Mysterious Ways: Evaluating a Plan Mode for Spreadsheet Agents

arXiv:2607.23670v1 Announce Type: cross Abstract: Plan Modes have become standard features in agentic programming tools, allowing users to gain transparency and control by working with the agent to de

DBA-Bench: A Production-Fidelity Benchmark for LLM-Based Database Operations Agents

arXiv:2607.22165v1 Announce Type: cross Abstract: LLM-based database agents show promise, but differing task scopes, testbeds, and metrics hinder comparison. We identify four gaps between evaluation a

27 Jul 2026

Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content

arXiv:2607.20446v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly used by students to generate natural language responses and program code, there is growing interest in

researcharxiv-cs-cl

24 Jul 2026

Hardware

Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis

arXiv:2607.20527v1 Announce Type: new Abstract: Agentic LLM systems such as OpenScholar and PaperQA2 read the scientific literature and return cited answers, and both they and their benchmarks already

hardwarearxiv-cs-ai

24 Jul 2026

Effort-Based Criticality Metrics for Evaluating 3D Perception Errors in Autonomous Driving

arXiv:2603.28029v2 Announce Type: replace-cross Abstract: Criticality metrics such as time-to-collision (TTC) quantify collision urgency but do not distinguish the operational consequences of false-po

safetyarxiv-cs-ro

23 Jul 2026

Evaluating and Mitigating Gender Bias in Pre-trained Embeddings for ML-based Recruitment

arXiv:2607.20073v1 Announce Type: new Abstract: AI-based recruitment systems that rely on machine learning models trained on historical CV data, risk perpetuating and amplifying social biases. A key c

safetyarxiv-cs-lg

23 Jul 2026

Task Competence Is Not Instruction Following: Evaluating Instruction-Conflicting Behavior in Small Language Models

arXiv:2607.19608v1 Announce Type: new Abstract: Instruction tuning is meant to make language models follow user requests, yet it is unclear whether small models comply when an instruction conflicts wi

23 Jul 2026

Evaluating Vision Foundation Models for Pixel and Object Classification in Microscopy

arXiv:2603.19802v2 Announce Type: replace Abstract: Deep learning underlies most modern approaches and tools in computer vision, including biomedical imaging. However, for interactive semantic segment

16 Jul 2026

Evaluating Nonuniform Dependability Across Response Conditions: A Conditional Generalizability Framework Illustrated in Automated Essay Scoring

arXiv:2607.11981v1 Announce Type: cross Abstract: Aggregate reliability estimates can obscure heterogeneity in measurement-design burden across response conditions, so a single G- or D-study may misch