Search: “evaluation”

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs

model-releasesarxiv-cs-cl

arXiv:2609.09964v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable progress across natural language processing (NLP) tasks, yet their capabilities degrade sharply fo

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

CityPlanner: A Sandbox Agent for Executable Urban Planning

DGX agent

arXiv:2609.09578v1 Announce Type: cross Abstract: Urban planning is a real-world spatial optimization problem that requires selecting feasible actions from large candidate spaces under practical objec

model-releasesarxiv-cs-cl

10 Sept 2026

Development and Validation of a Physics-Guided Machine Learning Extrapolation Framework Using a Classical Transient Diffusion Benchmark

arXiv:2609.09912v1 Announce Type: new Abstract: Machine learning models used in engineering are typically trained within limited operating ranges, yet reliable predictions are often required beyond th

EmoMed: An Emotionally-Aware Agent for Multimodal Medical Support with Real-Time Information Retrieval

arXiv:2609.07194v1 Announce Type: new Abstract: We present EmoMed - a multimodal medical consultation agent that adapts its responses based on users' emotional states while maintaining clinical accura

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

arXiv:2602.20773v2 Announce Type: replace Abstract: Purpose: Developing generalizable medical image segmentation models is challenging because imaging data are distributed across institutions and diff

In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning

model-releasesarxiv-cs-cl

arXiv:2609.09243v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) grounds a language model in retrieved documents, which reduces hallucination but creates a new attack surface: if

Applications

It Is Not My Code Anymore

arXiv:2609.09022v1 Announce Type: cross Abstract: AI-assisted programming raises distinct questions about who produces code, who feels ownership of it, and who is responsible when it fails. This resea

applicationsarxiv-cs-ai

Measuring LLM Sycophancy under Sustained Multi-Turn Pressure

arXiv:2609.09090v1 Announce Type: cross Abstract: Large language models (LLMs) may abandon correct positions when users push back, exhibiting a failure mode known as sycophancy. Existing evaluations t

NOPE-HYPE: A Structured Simulation Workflow for Robust Speech-to-Text Across Diverse Acoustic Environments

arXiv:2609.10058v1 Announce Type: cross Abstract: Robust speech-to-text translation systems should perform reliably across diverse acoustic conditions, yet practical pipelines lack controllable tools

researcharxiv-cs-cl

Open Tabular Insight Extraction: Where Do We Stand, and Where Should We Go?

arXiv:2609.07629v1 Announce Type: cross Abstract: Democratizing access to the knowledge held in large corpora of tables such as data lakes is emerging as a central research challenge. Research in this

researcharxiv-cs-ai

Pretraining and Distillation Matter More Than Architecture Family for Label-Free Single-Cell Classification

arXiv:2609.09863v1 Announce Type: new Abstract: Choosing a deep learning architecture for label-free single-cell classification remains an open question, with microscopy benchmarks reporting conflicti

ProcArena: A Multi-Scenario Benchmark for LLMs on Direct and Interactive PL/SQL Development from Natural Language

arXiv:2609.06527v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong potential for translating natural-language (NL) requirements into PL/SQL programs, attracting increasin

SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure

arXiv:2609.06540v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-

The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?

arXiv:2609.08589v1 Announce Type: cross Abstract: Recent large language models can emit task-progress signals that agent frameworks use to decide whether a task should continue or stop, yet whether a

TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs

arXiv:2609.08226v1 Announce Type: new Abstract: Temporal graph learning models the evolution of dynamic systems, where both structural interactions and semantic states change over time. However, exist

What Makes Adversarial Examples Transfer Across Deepfake Detectors?

arXiv:2609.10002v1 Announce Type: new Abstract: Deepfake detectors remain vulnerable to transfer-based black-box attacks, in which adversarial examples are generated on a source surrogate model and tr

XAI-Refine: An Automated Explanation-Knowledge Loop for Brain-Age Prediction

arXiv:2609.09388v1 Announce Type: new Abstract: Brain-age prediction models are commonly evaluated by predictive accuracy, yet accurate predictions alone do not establish that a model relies on reprod

safetyarxiv-cs-lg

A Comparative Study of GAN-Based Deep Learning Models for Pneumonia Detection in Chest X-Rays

arXiv:2609.06276v1 Announce Type: new Abstract: This study evaluates pneumonia classification in chest X-rays using VGG19, MobileNetV2, ResNet50, and a custom CNN, and explores Generative Adversarial

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

arXiv:2609.09030v1 Announce Type: cross Abstract: Chain-of-thought reasoning provides a structured computation between a model's input and final answer. Yet it is often evaluated through endpoint accu

researcharxiv-cs-lg

CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

arXiv:2404.06349v3 Announce Type: replace Abstract: The ability to understand causality significantly impacts the competence of large language models (LLMs) in output explanation and counterfactual re

CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments

arXiv:2609.07685v1 Announce Type: new Abstract: In real-world applications, pedestrian trajectory prediction models rely on inputs from detection and tracking systems. Prior trajectory prediction benc

Diagnosing LLM Reranker Behavior Under Fixed Evidence Pools

arXiv:2602.18613v2 Announce Type: replace Abstract: Standard reranking evaluations study how a reranker orders candidates returned by an upstream retriever. This setup couples ranking behavior with re

safetyarxiv-cs-lg

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting

arXiv:2609.08117v1 Announce Type: new Abstract: Driver motion can provide cues to ongoing behavior, attention, and near-term driving intent. However, most existing driver-centric datasets focus on rec

Google is a Leader in the 2026 Gartner® Magic Quadrant™ for Enterprise AI Assistants

model-releasesgoogle-cloud-ai

We are excited to share that Gartner has named Google a Leader in its inaugural 2026 Magic Quadrant for Enterprise AI Assistants. In this comprehensive evaluation of top enterprise AI assistant vendor

GradeTrap: Authority Cues in Images Shift VLM Judgments Despite Explicit Instructions to Ignore Them

arXiv:2609.06058v1 Announce Type: new Abstract: As vision-language models (VLMs) become increasingly capable and are deployed in consequential real-world settings, they must evaluate evidence independ

HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care

arXiv:2609.06976v1 Announce Type: new Abstract: As medical wearables become integrated into daily chronic disease care, effectively interpreting longitudinal monitoring data is essential for patients

Is Single-View Mesh Reconstruction Ready for Robotics?

arXiv:2505.17966v3 Announce Type: replace-cross Abstract: This paper evaluates single-view mesh reconstruction models for their potential in enabling instant digital twin creation for real-time planni

MLIP Detective: Active Failure Mode Discovery Beyond Benchmark Scores for Machine-Learning Interatomic Potentials

arXiv:2609.08399v1 Announce Type: new Abstract: Universal machine-learning interatomic potentials (u-MLIPs) aim to generalize across diverse configurations. Benchmarks enable reproducible evaluation b

NutriBench-Kitchen: Benchmarking Embodied AI for Nutrition Management

arXiv:2609.07135v1 Announce Type: new Abstract: An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observ

Agents

Online Surrogate Repair: Decoupling High-Fidelity Feedback from Search Length in Closed-Loop Discovery

arXiv:2609.07655v1 Announce Type: new Abstract: Closed-loop AI scientists can generate candidate designs at low marginal computational cost, whereas reliable feedback may require wet-lab synthesis, ch

agentsarxiv-cs-lg

Hardware

Proper Dataset Valuation by Pointwise Mutual Information

arXiv:2405.18253v4 Announce Type: replace Abstract: Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance. T

hardwarearxiv-cs-lg

Agents

Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation

arXiv:2609.06078v1 Announce Type: new Abstract: This report summarizes the 8th Large-scale Video Object Segmentation (LSVOS) Challenge, held in conjunction with ECCV 2026. The challenge evaluates vide

agentsarxiv-cs-cv

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

arXiv:2609.07823v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains undere

Applications

Sparse Incident-Cluster Learning for 12-hour Port Flood Pre-warning in Digital-Twin Analytics

arXiv:2609.06109v1 Announce Type: new Abstract: Port flood digital twins require analytics that warn operators before disruption, but official warning incidents are often few and adjacent observations

applicationsarxiv-cs-lg

Sustained Performance and Energy Accounting for Nonlinear Forecasting Across Classical and Simulated Quantum Models

arXiv:2510.25183v2 Announce Type: replace-cross Abstract: Energy-efficient AI should be evaluated across the full application pipeline, not only by lowest error or shortest training time. We study thi

Tempora-Fusion: Time-Lock Puzzle with Efficient Verifiable Homomorphic Linear Combination

arXiv:2406.15070v3 Announce Type: replace-cross Abstract: We present Tempora-Fusion, the first homomorphic TLP scheme with efficient public verification of both individual puzzle solutions and homomor

A Deep Generative Model for Synthesizing Labeled Wireless Signals

arXiv:2609.05396v1 Announce Type: new Abstract: Wireless signals with position-related labels are pivotal for both performance evaluation and model training in the realm of wireless sensing. However,

Hardware

A Sim-to-Real Study of Surface-Code Decoder Benchmarking

arXiv:2609.04557v1 Announce Type: cross Abstract: Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under

hardwarearxiv-cs-lg

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

arXiv:2609.04407v1 Announce Type: new Abstract: Deep neural operators learn mappings between input functions and complete PDE solution fields, enabling forward evaluations of new problem instances ord

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

model-releasesarxiv-cs-cl

arXiv:2609.04280v1 Announce Type: cross Abstract: Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what they can do.

Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models

arXiv:2609.04720v1 Announce Type: cross Abstract: Vision-language models (VLMs) are expected to respond helpfully to appropriate requests while withholding compliance with requests that are incorrect,

Knowing When Not to Answer: Pseudo-Ensembles for Abstention in Music Audio-Language Models

arXiv:2609.04362v1 Announce Type: cross Abstract: Music audio-language models are evaluated almost entirely by accuracy on multiple-choice questions. This protocol forces the model to commit to an opt

researcharxiv-cs-cl

Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?

arXiv:2609.05088v1 Announce Type: new Abstract: AI oversight methods rely on ground truth for validation, but what constitutes appropriate AI behavior is contested. This leaves evaluation of moral rea

safetyarxiv-cs-ai

Measuring the Novelty of Biomedical Papers Using the Latent Distances between Knowledge Units

arXiv:2609.05175v1 Announce Type: cross Abstract: Measuring the novelty of scientific papers is a central concern in research evaluation and scientometrics. From a recombination perspective, prior stu

safetyarxiv-cs-cl

PetQA: Benchmarking Veterinary Knowledge and Clinical Reasoning

arXiv:2609.04598v1 Announce Type: cross Abstract: We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

arXiv:2609.05141v1 Announce Type: new Abstract: Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporti

Single-Query Black-Box Calibration Auditing via Logit Bias

arXiv:2609.05125v1 Announce Type: new Abstract: Evaluating the calibration of Large Language Models (LLMs) is critical for their safe deployment as zero-shot classifiers. Yet, commercial API providers

TreeFI: Value-Aware Statistical Fault Injection for Deep Neural Networks

arXiv:2609.04912v1 Announce Type: cross Abstract: Reliability evaluation of deep neural networks under hardware faults commonly relies on fault injection, but exhaustive campaigns are intractable for

researcharxiv-cs-ai