Search: “evaluation”

PetQA: Benchmarking Veterinary Knowledge and Clinical Reasoning

arXiv:2609.04598v1 Announce Type: cross Abstract: We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

DGX agent

arXiv:2609.05141v1 Announce Type: new Abstract: Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporti

model-releasesarxiv-cs-ai

7 Sept 2026

Single-Query Black-Box Calibration Auditing via Logit Bias

model-releasesarxiv-cs-lg

arXiv:2609.05125v1 Announce Type: new Abstract: Evaluating the calibration of Large Language Models (LLMs) is critical for their safe deployment as zero-shot classifiers. Yet, commercial API providers

TreeFI: Value-Aware Statistical Fault Injection for Deep Neural Networks

arXiv:2609.04912v1 Announce Type: cross Abstract: Reliability evaluation of deep neural networks under hardware faults commonly relies on fault injection, but exhaustive campaigns are intractable for

Applications

Wireless Foundation Models: State-of-the-Art and Open Challenges

arXiv:2609.04707v1 Announce Type: cross Abstract: Wireless foundation models (WFMs) have emerged as a promising approach for learning reusable representations from large-scale wireless data and adapti

applicationsarxiv-cs-ai

A comparative study on the accuracy & repeatability of mobile robotic platforms for the delivery of precision NDE measurement

arXiv:2609.03794v1 Announce Type: new Abstract: Mobile robotic platforms offer a flexible alternative to fixed manipulators for non-destructive evaluation (NDE) of large aerospace structures, but thei

safetyarxiv-cs-ro

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

arXiv:2609.03734v1 Announce Type: cross Abstract: BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language pro

Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognition

arXiv:2609.02901v1 Announce Type: new Abstract: Modern automatic speech recognition (ASR) scenarios require both spoken-form transcripts for faithful transcription and readable written-form transcript

researcharxiv-cs-cl

Environment Evolution for Terminal Agents

arXiv:2609.04128v1 Announce Type: new Abstract: Scaling interactive and verifiable environments is critical for training terminal agents. As frontier models become more capable, environments synthesiz

FailBench: How Reliable are VLMs at Judging Robot Task Success?

arXiv:2609.03611v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly used to evaluate robot manipulation outcomes, but existing benchmarks offer limited evidence of cross-d

Improving Clinical Target Volume Segmentation Accuracy using Anatomical Priors and Active Learning for the AGITG TOPGEAR Clinical Trial

arXiv:2609.03186v1 Announce Type: cross Abstract: Training deep learning-based medical image segmentation models is challenging with limited curated datasets. For AGITG TOPGEAR, a gastric cancer trial

researcharxiv-cs-cv

Interface-Induced Trajectory Censoring

arXiv:2609.03966v1 Announce Type: new Abstract: Agent evaluations report a tool-call rate read off the serving stack. That number can be zero while the model is emitting well-formed calls: the interfa

Applications

LLM4CKD: Large Language Models for Early Stage Chronic Kidney Disease Screening

arXiv:2609.04013v1 Announce Type: new Abstract: Early screening of chronic kidney disease (CKD) is critical for timely intervention, yet most machine learning (ML) and deep learning (DL) approaches re

applicationsarxiv-cs-ai

LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues

arXiv:2609.03507v1 Announce Type: cross Abstract: Tracking depression from multi-session counseling dialogues requires estimating both current symptom severity and how it changes across sessions. Yet

Not All LLM Workloads Are Equal: Benchmarking TPU Performance on Classification vs. Generation

model-releasesgoogle-cloud-ai

Moving Large Language Models (LLMs) from experimental prototypes into enterprise production exposes a critical truth: your infrastructure dictates both your performance ceilings and your unit economic

ProAct: Harnessing Streaming Motion Generation and Agentic Reasoning for Real-Time Embodied Social Interaction

model-releasesarxiv-cs-cv

arXiv:2602.14048v2 Announce Type: replace-cross Abstract: Real-time embodied social interaction places two equally demanding requirements on an agent: continuously generating fluent multimodal interac

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition

arXiv:2506.10226v3 Announce Type: replace-cross Abstract: Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on e

safetyarxiv-cs-ai

SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarking

arXiv:2609.03047v1 Announce Type: cross Abstract: Libraries and archives manage large collections with limited staff and computing budgets, yet common benchmarks do not systematically test their bibli

Tree species mapping in Denmark: A comparison of spectral-temporal features with geospatial foundation model embeddings

arXiv:2609.03480v1 Announce Type: cross Abstract: We map tree species across Denmark using National Forest Inventory plots and EO data, while evaluating the potential of foundation models for large-sc

User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios

arXiv:2510.20721v4 Announce Type: replace-cross Abstract: Large language models (LLMs) are rapidly being adopted for tasks like drafting emails, summarizing meetings, and answering health questions. I

safetyarxiv-cs-ai

When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

arXiv:2609.03467v1 Announce Type: cross Abstract: Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. Howev

Hardware

A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

arXiv:2609.02110v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) commonly evaluate the spatial derivatives appearing in partial differential equation residuals using automatic

hardwarearxiv-cs-lg

Auditory Illusion Benchmark for Large Audio Language Models

arXiv:2609.02277v1 Announce Type: cross Abstract: Perceptual illusions have long served as crucial probes into human cognition, revealing biases and limitations of perception. In the auditory domain,

Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization

arXiv:2609.01861v1 Announce Type: new Abstract: The performance of an LLM agent depends on the scaffold around a frozen model. A common way to improve that scaffold is to use a coding agent as an opti

Benchmarking Language Models for Statistical Problem Formulation

arXiv:2609.01982v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as assistants for statistical and data science work, yet existing evaluations largely assume the anal

DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation

arXiv:2609.02796v1 Announce Type: new Abstract: Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language c

researcharxiv-cs-cl

Disease Burden over Skin Tone: Decomposing the Dermatology-AI Generalization Gap

arXiv:2609.02111v1 Announce Type: cross Abstract: Dermatology artificial intelligence (AI) models are predominantly trained on light-skinned, cancer-focused image collections, yet they are increasingl

DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents

arXiv:2609.02059v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question a

Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods

model-releasesarxiv-cs-cv

arXiv:2609.01888v1 Announce Type: new Abstract: Recent inference-time hallucination mitigation methods for large vision-language models (LVLMs) report strong gains on hallucination benchmarks. However

Evidential Deep Learning for Multi-Modal Anti-UAV Detection

arXiv:2609.01742v1 Announce Type: new Abstract: Anti-UAV systems increasingly fuse multiple sensors, yet their detection heads provide no per-modality reliability signal. This study evaluates whether

researcharxiv-cs-cv

From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

arXiv:2609.02679v1 Announce Type: cross Abstract: When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited huma

LLM-as-a-Judge Is Not an Oracle: Why Self-Improving Agents Need Deterministic Guardrails

arXiv:2609.02246v1 Announce Type: new Abstract: Self-improving agent pipelines have a problem at their center. An optimizer rewrites prompts to score higher, and the score comes from a judge that is i

safetyarxiv-cs-ai

LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematization

arXiv:2607.10103v2 Announce Type: replace-cross Abstract: As large language models (LLMs) become widely deployed, their outputs can be copied, transformed, and redistributed at scale without reliable

Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting

arXiv:2609.02649v1 Announce Type: new Abstract: Aggregating noisy, conflicting textual hypotheses into a reliable consensus is a fundamental challenge when deploying NLP systems in real-world industri

MESSY STREETS: A Benchmark for Geocoding Real-World Addresses

arXiv:2609.01612v1 Announce Type: cross Abstract: We introduce MESSY STREETS, a benchmark for evaluating geocoders on verbatim web addresses, with existence verification and controlled measurement of

MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts

arXiv:2609.02379v1 Announce Type: cross Abstract: While existing work on LLM authorship attribution (AA) has made progress, available benchmarks remain limited, often focusing on English, controlled s

Post-Training Language Models for Gold-Medal Performance in Coding Competitions

arXiv:2609.02849v1 Announce Type: cross Abstract: Competitive programming has become a key test of large language model reasoning, with international competitions such as IOI and ICPC representing its

Agents

RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems

arXiv:2609.01622v1 Announce Type: cross Abstract: The rise of agentic AI has catalyzed a shift toward self-iterating systems, opening new frontiers for the autonomous optimization of production recomm

agentsarxiv-cs-ai

Local Ai

SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition

arXiv:2609.02036v1 Announce Type: new Abstract: Few-step diffusion models substantially compress temporal computation, making the spatial cost of each model evaluation an increasingly dominant source

local-aiarxiv-cs-cv

Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM

arXiv:2512.15662v4 Announce Type: replace Abstract: Human beings solve complex problems through critical thinking, where reasoning and evaluation are intertwined to converge toward correct solutions.

TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding

arXiv:2609.01810v1 Announce Type: new Abstract: Farsi, spoken by more than 120 million people, lacks a comprehensive benchmark for dialogue generation and understanding. We introduce TALKFA, a unified

VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation

model-releasesarxiv-cs-cv

arXiv:2609.02462v1 Announce Type: new Abstract: End-to-end autonomous driving in urban environments requires robust decision-making under partial observability and complex multi-agent interactions. Se

WinoQueer-NL: Assessing Bias in Dutch Language Models toward LGBTQ+ Identities

arXiv:2609.02651v1 Announce Type: new Abstract: While English language models have been widely examined for anti-queer bias, Dutch models remain understudied. To address this gap, we developed a cultu

WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling

arXiv:2609.01608v1 Announce Type: cross Abstract: Black-box optimization problems remain challenging because of large, weakly structured, and high-dimensional search spaces. Existing methods often suf

Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbooks

arXiv:2609.00763v1 Announce Type: new Abstract: Hierarchical Knowledge graph (KG)-based retrieval augmented generation (RAG) has emerged as a powerful approach for supporting large language models wit

Local Ai

Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling

arXiv:2609.00949v1 Announce Type: cross Abstract: Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now

local-aiarxiv-cs-ai

Can LLMs Design Video Coding Tools? A Case Study on Planar Mode

arXiv:2609.01535v1 Announce Type: cross Abstract: This paper explores whether large language models (LLMs) can design video coding tools, a highly challenging task due to the intricate algorithmic cou

Controllable Image Captioning with Prompt-Conditioned Scene Rewards

arXiv:2609.00709v1 Announce Type: cross Abstract: Large Vision-Language Models produce fluent image descriptions but offer limited semantic control: users cannot reliably specify whether captions shou