Search: “evaluation”

Beyond Random Splits: Evaluating Drug-Target Affinity Models Under Chemically and Biologically Motivated Distribution Shifts Copy

arXiv:2610.03456v1 Announce Type: new Abstract: Drug-target affinity (DTA) prediction is widely used to prioritize candidate compounds before costly experimental screening. DTA models are often compar

5 Oct 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents

DGX agent

arXiv:2610.03448v1 Announce Type: cross Abstract: LLM agents increasingly screen tool outputs with small prompt-injection detectors, and teams choose among detectors by their scores on public benchmar

model-releasesarxiv-cs-cl

5 Oct 2026

Applications

Evaluating LLM-Generated Preference Distributions

arXiv:2610.01000v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settin

applicationsarxiv-cs-ai

Evaluating the Robustness of Anti-UAV Detection under Controlled Fog Degradation: Fog-Aware Training and Clear-Sky Tradeoff

model-releasesarxiv-cs-cv

arXiv:2610.00141v1 Announce Type: new Abstract: Vision-based anti-UAV systems must function in poor visibility, yet most benchmarks use only clear-sky footage, and previous robustness studies treat ad

MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generation

arXiv:2505.17613v2 Announce Type: replace Abstract: Automatically evaluating multimodal generation presents a significant challenge, as automated metrics often struggle to align with human evaluation,

ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality

arXiv:2610.00710v1 Announce Type: new Abstract: As large language model (LLM) agents become widely adopted, they are increasingly deployed for tasks that require persistent monitoring or recurring act

Beyond the Shadows of Plato's Cave: Evaluating False Memory in Autonomous Agents via Counterfactual Reasoning

arXiv:2609.39473v1 Announce Type: new Abstract: Autonomous agents increasingly rely on memory to generalize beyond their training environments. However, agents are bounded by what they have seen and b

Evaluating Multi-Task Morphological Concept Learning for Pulmonary Nodule Malignancy Assessment in 3D CT

arXiv:2609.38271v1 Announce Type: new Abstract: Morphological characteristics such as spiculation and lobulation play an important role in assessing pulmonary nodules on computed tomography (CT), part

researcharxiv-cs-cv

Safety

Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems

arXiv:2609.38552v1 Announce Type: cross Abstract: Public institutions increasingly procure AI systems whose design they cannot inspect or change. In higher education, proprietary Early Warning Systems

safetyarxiv-cs-lg

MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models

arXiv:2609.38543v1 Announce Type: new Abstract: Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time,

Beyond Readability: Evaluating Task Information Recoverability

arXiv:2609.36957v1 Announce Type: new Abstract: Direct visual readability and task-information recoverability are different quantities. Failure to decode a target from a fixed observation need not eli

researcharxiv-cs-cv

Evaluating and Benchmarking the System One Model Jev

arXiv:2609.37647v1 Announce Type: cross Abstract: Jev is a commercial System One model from TypeSafe AI that does not generate text: given a state and typed questions, it returns a choice from fixed o

Evaluating the Effects of Prompt Perturbation on Bias and Hallucination in Large Language Models

arXiv:2609.35804v1 Announce Type: cross Abstract: Large language models (LLMs) have shown remarkable capabilities in various natural language processing tasks, leading to their widespread deployment a

Applications

Foresight at the Event Boundary: Evaluating Physical Prediction in Video World Models

arXiv:2609.36531v1 Announce Type: new Abstract: Video world models are largely regarded as predictive models of the physical world and are therefore expected to anticipate the consequences of observed

applicationsarxiv-cs-cv

UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training

arXiv:2609.38043v1 Announce Type: new Abstract: Interactive agent benchmarks and multi-turn reinforcement learning increasingly place a second language model in the role of the user. This simulated us

Beyond Correctness: Evaluating Semantic Knowledge in Cross-Table Transfer

arXiv:2609.34098v1 Announce Type: new Abstract: Semantic knowledge is increasingly used to bridge heterogeneous schemas in tabular learning, but how much does that knowledge actually improve predictio

29 Sept 2026

ScAn-Bench: Evaluating Scaling Analysis Methodology

arXiv:2609.35707v1 Announce Type: new Abstract: Recent progress in machine learning is driven by large-scale foundation models, where scaling laws and finding optimal scaling prescriptions for archite

29 Sept 2026

Coding Agents Aren't Enough! Evaluating an Enterprise Security Brain for Agentic Cloud Investigations

arXiv:2609.30345v1 Announce Type: cross Abstract: Cloud-security investigation is dominated by population tasks: which identities can read a data store, how many resources fail a control, which assets

Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomes

arXiv:2609.30798v1 Announce Type: new Abstract: Real-time voice agents have moved from research prototypes to production deployments, yet the literature describing them is fragmented across three comm

SciR: A Controllable Benchmark for Scientific Reasoning in LLMs

arXiv:2606.13020v3 Announce Type: replace Abstract: Three paradigmatic forms of inference recur across scientific reasoning: deduction, induction, and causal abduction. Reliably evaluating LLMs on the

The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge

arXiv:2609.30604v1 Announce Type: cross Abstract: Existing computer-use agent benchmarks do not fully evaluate agents acting as assistants. A useful assistant retrieves information across complex, mul

Syntheses

Wiki Lint Report — 2026-09-27

linthealth-checkautomated

Automated lint: 17 errors, 59669 warnings, 3 info

27 Sept 2026

Can Classical Semantic-Extractive Summarization Be Evaluated in Hindi? A Replication Study

arXiv:2609.29090v1 Announce Type: cross Abstract: We replicate the distributional-semantics extractive summarisation method of Mohd, Jan and Shah (2020) and adapt it to Hindi, substituting a Devanagar

25 Sept 2026

Evaluating Cross-region Generalization for Wavelet-Diffusion Precipitation Downscaling

arXiv:2609.28749v1 Announce Type: new Abstract: Diffusion models have shown strong potential for kilometer-scale precipitation downscaling, but their performance in geographically unseen regions and e

25 Sept 2026

Evaluating ADC-only deep learning pipelines for breast cancer detection and segmentation using standalone diffusion-weighted MRI

arXiv:2609.27815v1 Announce Type: cross Abstract: Dynamic contrast-enhanced (DCE) imaging is the gold standard technique for the detection and characterization of breast cancer using magnetic resonanc

FDE-Bench: Evaluating LLM Agents for Deployment Environment Configuration

arXiv:2609.27571v1 Announce Type: cross Abstract: Deployment requires an agent to turn application code into a running system whose services connect, become ready, and remain observable. FDE-Bench eva

MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design

arXiv:2609.27349v1 Announce Type: new Abstract: Real-world molecular design remains challenging for large language model (LLM)-based agents. It requires them to interpret design contexts, satisfy mult

The Path Matters: Evaluating Small Language Models Beyond Answer Accuracy in KGQA

arXiv:2609.27669v1 Announce Type: cross Abstract: Small language models (SLMs) are increasingly paired with knowledge graphs (KGs), yet end-to-end KG question answering conflates graph access, search,

ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry

arXiv:2511.17909v2 Announce Type: replace Abstract: Chemical reasoning inherently integrates visual, textual, and symbolic modalities, yet existing benchmarks rarely capture this complexity, often rel

Evaluating the Semantic-to-Geometric Gap in Adversarial Defenses Against Vision-Language Model-Based Plagiarism

arXiv:2609.26733v1 Announce Type: new Abstract: The rapidly advancing capabilities of vision-language models (VLMs) present a systemic challenge to academic integrity. VLMs now allow students to bypas

researcharxiv-cs-cv

SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning

arXiv:2609.22586v1 Announce Type: cross Abstract: Modern audio-language models are no longer judged only on what words they can transcribe, but on whether they can reason over what they hear: recoveri

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch

arXiv:2609.25081v1 Announce Type: new Abstract: We describe ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish language model pretrained from scratch on 13.5B tokens of openly li

Evaluating Decision Models for Text Annotation in Computational Social Science

arXiv:2609.24574v1 Announce Type: new Abstract: Computational social science increasingly relies on large language models for text annotation, and the validity of published findings now rests on the l

Safety

Evaluating Transformation Models for pCLE Mosaic Registration

arXiv:2609.24560v1 Announce Type: new Abstract: Confocal Laser Endomicroscopy (CLE) provides real-time, cellular-resolution optical biopsy but has a narrow field of view, which image mosaicing can ext

safetyarxiv-cs-cv

From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification

arXiv:2609.23966v1 Announce Type: new Abstract: LLMs can generate fluent descriptions from tables, but their outputs may remain logically unsupported by the structured data. We introduce STAT-TO-TEXT,

researcharxiv-cs-cl

Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences

arXiv:2609.24965v1 Announce Type: new Abstract: Scientific workflows often require choosing among known relations before a deterministic calculation can proceed. Whether observations share a culture,

researcharxiv-cs-cl

Team DArgk at the 2026 ELOQUENT lab for evaluating generative language model quality: Residuals of Humanity: AI Detection Evasion via GRPO Fine-Tuning

arXiv:2609.22221v1 Announce Type: new Abstract: Large language models (LLMs) can generate fluent and coherent text that is increasingly difficult to distinguish from human writing, motivating the deve

From Task Success to Productive Success: Evaluating Human-AI Collaboration by Quality and Cost

arXiv:2609.21117v1 Announce Type: cross Abstract: AI productivity is often measured by task completion time, economic value, or improvements in outcome quality. However, these measures usually treat c

21 Sept 2026

Tutorials

Reducing Barriers to Academic Support: Evaluating a Course-Specific RAG System for Addressing Help-Seeking Disparities in Higher Education

arXiv:2609.21600v1 Announce Type: new Abstract: Access to academic support is a key determinant of student success, yet students experience it unequally: some readily seek help from lecturers or tutor

tutorialsarxiv-cs-ai

21 Sept 2026

For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances

arXiv:2609.19504v1 Announce Type: cross Abstract: As model-generated content is increasingly consumed by other model instances in automated workflows, a practically important question arises: can a mo

18 Sept 2026

Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry

arXiv:2609.19154v1 Announce Type: new Abstract: While Large Language Models (LLMs) achieve high accuracy on established Classical Chinese Poetry benchmarks, it remains challenging to distinguish trans

18 Sept 2026

From Momentary Emotion Inference to Sustained Emotion Support: Evaluating a Companion Agent in a Longitudinal Study

arXiv:2609.16344v1 Announce Type: cross Abstract: Sustained emotional support is a long-horizon interaction task closely tied to human well-being. Recent research demonstrates generative agents' capac

16 Sept 2026

Safety

RobResilience: Implementing and Evaluating a Resilience Framework for Cyber-Physical Embodied Systems

arXiv:2609.17349v1 Announce Type: cross Abstract: In embodied cyber-physical systems, active cyberattacks pose an immediate threat not just to data, but to physical integrity and human safety. While e

safetyarxiv-cs-ro

16 Sept 2026

What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity

arXiv:2609.17515v1 Announce Type: new Abstract: Pruning can reduce the deployment cost of large language models (LLMs), but its impact on context-grounded tool calling remains poorly understood. We sy

researcharxiv-cs-cl

16 Sept 2026

Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context

arXiv:2609.15087v1 Announce Type: cross Abstract: Most time series forecasting benchmarks remain numerical-centric and provide limited support for evaluating contextual information that shapes real-wo

15 Sept 2026

Do LLMs Trust the Accuser or the Accusation? Measuring Belief Shifts in Werewolf

arXiv:2609.12446v1 Announce Type: cross Abstract: Social-deduction games such as Werewolf are increasingly used to evaluate LLM agents, but existing evaluations often rely on final game outcomes. We p

14 Sept 2026

PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognition

arXiv:2609.12917v1 Announce Type: cross Abstract: Handwritten mathematical expression recognition (HMER) is conventionally scored by exact-match rates and string-similarity metrics that are blind to w

14 Sept 2026

Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks

arXiv:2609.11018v1 Announce Type: new Abstract: The term agent in artificial intelligence lacks a standard definition, complicating the evaluation, comparison, and reproducibility of AI agent research