Search: “evaluation”

SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

arXiv:2603.17373v2 Announce Type: replace Abstract: Large language models are rapidly being deployed as AI tutors, yet current evaluation paradigms assess problem-solving accuracy and generic safety i

24 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Scribd, Inc. classifies more than 400 million documents with Gemini batch inference on Gemini Enterprise

DGX agent

Scribd, Inc. is home to one of the world's largest collections of human-created content. Scribd’s products leverage one of the world's largest collections of human-created content and intelligent tool

model-releasesgoogle-cloud-ai

24 Sept 2026

Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning

arXiv:2603.08930v2 Announce Type: replace-cross Abstract: This paper introduces a benchmark for evaluating whether vision-language models (VLMs) can generate plant simulation configurations from image

24 Sept 2026

VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark

arXiv:2508.13680v5 Announce Type: replace Abstract: We introduce VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark designed to evaluate how vision-language models (VLMs) in

24 Sept 2026

Auditing Proxy-Based Validation Across Text Spans

arXiv:2609.25808v1 Announce Type: cross Abstract: Evaluation scores are often validated by their agreement with inexpensive proxy labels. When the score and the proxy are computed from the same text s

Biased AI improves human performance but reduces perceived helpfulness

arXiv:2508.09297v4 Announce Type: replace-cross Abstract: Artificial intelligence (AI) increasingly shapes how people think, engage, and evaluate information. To minimize risk, most current systems ar

safetyarxiv-cs-ai

Detecting GPT-Assisted Writing Using Interpretable Stylometric Features

arXiv:2609.26687v1 Announce Type: new Abstract: Distinguishing GPT-assisted from independently authored student writing has become a critical challenge in academia. This paper evaluates the discrimina

researcharxiv-cs-cl

From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation

arXiv:2609.22664v1 Announce Type: cross Abstract: Research on large language model agents for penetration testing is evaluated almost entirely by capability: whether the agent captures a flag or repro

ISA-Bench: A Benchmark for Computational Reasoning Across Instruction Set Architectures

arXiv:2609.22878v1 Announce Type: new Abstract: Large language model code generation benchmarks primarily evaluate well-resourced languages like Python and Java, where models benefit from abundant tra

LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment

arXiv:2609.26205v1 Announce Type: new Abstract: Aligning with the human visual system~(HVS) in perceiving and evaluating the quality of visual signals is a central objective of machine-vision-based vi

researcharxiv-cs-cv

Mitigating Sequential Reappearance in Diffusion Data-Point Unlearning

arXiv:2609.25166v1 Announce Type: new Abstract: Diffusion data-point unlearning is typically evaluated immediately after each deletion, even though subsequent requests may repeatedly update the same m

researcharxiv-cs-lg

MLCommons Joins EU-Funded AIRIS Project to Build and Benchmark Next-Generation Biomedical AI

Standardizing independent evaluation, fairness, and interpretability for mechanism-informed generative AI in healthcare The post MLCommons Joins EU-Funded AIRIS Project to Build and Benchmark Next-Gen

model-releasesmlcommons

NavSafe-infty: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments

arXiv:2609.26618v1 Announce Type: new Abstract: End-to-end (E2E) driving policies have progressed rapidly on open-loop (OL) benchmarks, yet OL evaluation cannot reveal whether a policy withstands comp

PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AI

arXiv:2609.23695v1 Announce Type: new Abstract: Recent advances in Physical AI have accelerated the use of foundation models in autonomous systems such as unmanned aerial vehicles (UAVs), which must p

RoboTwin-Phys: Do WAMs and VLAs Understand the Physical World?

arXiv:2609.26292v1 Announce Type: new Abstract: Physical-condition diversity is largely missing from current benchmarks for robot manipulation. While large-scale simulation benchmarks increasingly inc

SAMI3D-DW: Interactive Segmentation of Any 3D Medical Images

arXiv:2609.25743v1 Announce Type: new Abstract: Interactive segmentation of 3D medical images supports quantitative analysis of anatomical structures and disease while allowing users to specify and re

TicTacBench: Benchmarking Timing Closure Capabilities of Coding Agents

arXiv:2609.23363v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have led to the emergence of coding agents capable of performing complex engineering tasks, including re

Toward Auditable and Calibrated AI for Dementia-Related Crash Severity Prediction: A Selective Deferral Framework to Support Human Review

arXiv:2609.22694v1 Announce Type: new Abstract: Public crash databases increasingly support automated safety analysis, but crash severity prediction remains difficult to translate into public-sector d

TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models

arXiv:2609.26314v1 Announce Type: new Abstract: Embodied world models predict the outcomes of robot actions to support learning and planning. For robots equipped with head and wrist cameras, this requ

What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

arXiv:2609.12708v2 Announce Type: replace-cross Abstract: AI coding assistants are becoming co-authors of production software, yet their evaluation centers on functional correctness, leaving open whet

ALPINE: Adaptive Localization for Parameter- and Sample-Efficient Few-Shot Learning

arXiv:2609.22323v1 Announce Type: new Abstract: Few-shot learning research is predominantly evaluated on accuracy alone, with limited attention to the parameter and training-sample budgets required to

Tutorials

Authorship identification under domain shift: a survey of stylistic measures and learned author representations

arXiv:2310.00436v2 Announce Type: replace Abstract: Authorship identification uses patterns in writing to infer who wrote a text, but those patterns also reflect topic, genre, and register. This surve

tutorialsarxiv-cs-cl

CCTU: A Benchmark for Tool Use under Complex Constraints

arXiv:2603.15309v2 Announce Type: replace Abstract: Solving problems through tool use under explicit constraints constitutes a highly challenging yet unavoidable scenario for large language models (LL

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

arXiv:2609.24657v1 Announce Type: cross Abstract: Language models can be adapted by changing the computations applied to individual tokens. Quantum circuits offer one such approach, but evaluating wid

Decomposing Error and Style in Automated Clinical Coding

arXiv:2609.24877v1 Announce Type: new Abstract: In automated clinical coding, where the label space spans tens of thousands of diagnosis and procedure codes, models are currently evaluated against a s

safetyarxiv-cs-cl

Do LiDAR Language Models Really Understand Spatio-temporal Relationships?

arXiv:2609.24452v1 Announce Type: new Abstract: Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same

DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models

arXiv:2609.22762v1 Announce Type: new Abstract: Generative world-action models (WAMs) jointly generate future video and vehicle actions, while their action branches remain primarily optimized by exper

safetyarxiv-cs-cv

Financial Language Models as Applied Artificial Intelligence Systems for News-Based Trading under Market Frictions

arXiv:2609.23703v1 Announce Type: new Abstract: Financial language models can transform unstructured firm-specific news into structured decision signals, but financial AI research lacks an integrated

researcharxiv-cs-cl

GRACE: Grounded Adversarial Reasoning over Canadian Law

arXiv:2609.23726v1 Announce Type: new Abstract: Large language models have shown strong performance across a range of legal tasks, but existing benchmarks rarely evaluate the ability to take and defen

safetyarxiv-cs-cl

Learning Physics from an Imperfect Ancestor

arXiv:2609.24947v1 Announce Type: new Abstract: Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed ne

Leveraging Inference-Time Compute for Diffusion Models via Global Scheduling of Denoising Trajectories

arXiv:2609.22867v1 Announce Type: new Abstract: Diffusion models generate a sample by traversing a denoising trajectory, a sequence of stochastic noise-reduction steps that transforms pure noise into

OSCAR: Order-aware Scoring and Calibration for AI Rankings

arXiv:2609.24128v1 Announce Type: cross Abstract: Judge-specific sensitivity is useful for aggregating pairwise LLM evaluations, but its interpretation depends on which systematic presentation effects

Predictors and Orchestrators: Parsimonious Machine Learning within an Agentic AI Harness for Multi-Horizon Karst Aquifer Forecasting

arXiv:2609.22251v1 Announce Type: new Abstract: Forecasting karst aquifer dynamics is difficult because recharge responses are nonlinear, event-driven, and governed by strongly heterogeneous flow path

safetyarxiv-cs-lg

Reinforcement learning with an expectile-based objective

arXiv:2602.09300v2 Announce Type: replace Abstract: We consider the policy evaluation and control in a finite horizon reinforcement learning (RL) setting under an expectile-based objective. First, we

safetyarxiv-cs-lg

Rule-Constrained Assignment for Cue-Ball Identification in Broadcast Snooker

arXiv:2609.23450v1 Announce Type: new Abstract: Accurate cue-ball identification is essential for metric analysis of broadcast snooker. Existing systems evaluate each candidate independently against a

researcharxiv-cs-cv

SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs

arXiv:2609.22153v1 Announce Type: new Abstract: Methods for addressing safety drift in fine-tuned Large Language Models (LLMs) are scattered across incompatible implementations, lifecycle stages, and

safetyarxiv-cs-lg

The Operational Value of Spatial Dependence in Renewable Forecast Scenarios for Single-Period Economic Dispatch: A Controlled Ablation Study

arXiv:2609.23988v1 Announce Type: cross Abstract: Renewable forecasts are evaluated by statistical skill (e.g., CRPS), but grid operators pay for realized dispatch cost. We diagnose what drives dispat

researcharxiv-cs-lg

The Situated Identity Test: Distinguishing Persistent Cognitive Identity from Persona Imitation

arXiv:2609.22195v1 Announce Type: new Abstract: Large language models can convincingly adopt personas, recall past dialogues, and weave rich autobiographies. Yet this conversational eloquence conceals

Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models

arXiv:2609.22097v1 Announce Type: new Abstract: The evaluation of large language models (LLMs) on coding tasks has primarily focused on performance metrics such as pass@k. As LLMs continue to advance,

researcharxiv-cs-cl

Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI

arXiv:2609.24815v1 Announce Type: new Abstract: Scalable simulation is essential for robot data generation, policy training, evaluation, and safe iteration, yet real-world interaction is costly and co

safetyarxiv-cs-ro

When Residualization Helps an Audit: Format Effects, Slice Gains, and Their Limits

arXiv:2609.24194v1 Announce Type: new Abstract: Evaluation scores used around LLM systems -- including reward models, rerankers, and LLM judges -- can track surface form instead of the quality they cl

safetyarxiv-cs-cl

X-Beat: An Explainable Framework for ECG Image Classification

arXiv:2609.22631v1 Announce Type: new Abstract: Accurate automated interpretation of electrocardio- grams (ECGs) is essential for early detection of cardiac condi- tions such as myocardial infarction

Co-Evolving Zero-Day Jamming: Adaptive Attack Synthesis and Graph Attention-Based Online Detection

arXiv:2609.21334v1 Announce Type: cross Abstract: Effective evaluation of zero-day jamming detectors requires robust adversarial models. However, existing attack models often assume prior knowledge of

Hardware

How Much of a Real Workload Can LLM-Generated GPU Kernels Actually Reach?

arXiv:2609.21058v1 Announce Type: cross Abstract: Language models can now write GPU kernels that outperform PyTorch. We evaluate five model configurations on KernelBench level 1 and find that a fronti

hardwarearxiv-cs-ai

Implicit Rule Induction with Test-Time Task Embeddings in ARC-like Tasks

arXiv:2609.21181v1 Announce Type: new Abstract: The Abstraction and Reasoning Corpus and related benchmarks evaluate whether AI models can solve novel reasoning tasks, but often leave unclear whether

researcharxiv-cs-ai

PopNavShift: Stress-Testing Social Navigation under Behavioral Population Shift

arXiv:2609.21838v1 Announce Type: new Abstract: Social-navigation algorithms are often evaluated under a fixed pedestrian-behavior distribution, despite substantial variation in pedestrian responses t

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

arXiv:2609.21521v1 Announce Type: cross Abstract: While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understandi

When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

arXiv:2609.20942v1 Announce Type: new Abstract: Large language models (LLMs) increasingly participate in scientific evaluation, both as automated reviewers and as assistants to human reviewers. As mod