Search: “evaluation”

arXiv:2605.01548v1 Announce Type: cross Abstract: Electrocardiogram (ECG) biometrics have emerged as a promising modality for continuous, liveness-aware authentication in wearable systems. However, ma

applicationsarxiv-cs-cv

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Foundation Models to Unlock Real-World Evidence from Nationwide Medical Claims

DGX agent

arXiv:2605.02740v1 Announce Type: cross Abstract: Evidence derived from large-scale real-world data (RWD) is increasingly informing regulatory evaluation and healthcare decision-making. Administrative

safetyarxiv-cs-cl

5 May 2026

Hardware

Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Models

arXiv:2605.01870v1 Announce Type: new Abstract: Large Language Models (LLMs) have substantially advanced the field of Natural Language Processing (NLP), achieving state-of-the-art performance across a

hardwarearxiv-cs-cl

Orthographic Constraint Satisfaction and Human Difficulty Alignment in Large Language Models

arXiv:2511.21086v2 Announce Type: replace Abstract: Large language models must satisfy hard orthographic constraints during controlled text generation, yet systematic cross-family evaluation remains l

Tutorials

Rethinking Electro-Optical Vision Foundation Models for Remote Sensing Retrieval: A Controlled Comparison with Generalist VFM

arXiv:2605.02283v1 Announce Type: new Abstract: Vision foundation models have attracted significant attention for their ability to leverage large-scale unlabeled visual data. This advantage is particu

tutorialsarxiv-cs-cv

SurgCheck: Do Vision-Language Models Really Look at Images in Surgical VQA?

model-releasesarxiv-cs-cv

arXiv:2605.01911v1 Announce Type: new Abstract: Purpose: Vision-language models (VLMs) have shown promising performance in surgical visual question answering (VQA). However, existing surgical VQA data

TRAP: Tail-aware Ranking Attack for World-Model Planning

arXiv:2605.01950v1 Announce Type: new Abstract: World models enable long-horizon planning by internally generating and evaluating imagined trajectories, making them a promising foundation for generali

safetyarxiv-cs-lg

Concolic Testing on Individual Fairness of Neural Network Models

model-releasesarxiv-cs-lg

arXiv:2509.06864v2 Announce Type: replace Abstract: This paper introduces PyFair, a formal framework for evaluating and verifying individual fairness of Deep Neural Networks (DNNs). By adapting the co

4 May 2026

InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information

arXiv:2508.07630v2 Announce Type: replace Abstract: We introduce InterChart, a diagnostic benchmark that evaluates how well vision-language models (VLMs) reason across multiple related charts, a task

4 May 2026

Local Ai

TimesNet-Gen: Deep Learning-based Site Specific Strong Motion Generation

arXiv:2512.04694v3 Announce Type: replace Abstract: Effective earthquake risk reduction relies on accurate site-specific evaluations, which require models capable of representing the influence of loca

local-aiarxiv-cs-lg

4 May 2026

Agents

Agentic Compilation: Mitigating the LLM Rerun Crisis for Minimized-Inference-Cost Web Automation

arXiv:2604.09718v2 Announce Type: cross Abstract: LLM-driven web agents operating through continuous inference loops -- repeatedly querying a model to evaluate browser state and select actions -- exhi

agentsarxiv-cs-ai

CBEN -- A Multimodal Machine Learning Dataset for Cloud Robust Remote Sensing Image Understanding

arXiv:2602.12652v2 Announce Type: replace Abstract: Clouds are a common phenomenon that distorts optical satellite imagery, which poses a challenge for remote sensing. However, in the literature cloud

researcharxiv-cs-cv

CoAX: Cognitive-Oriented Attribution eXplanation User Model of Human Understanding of AI Explanations

arXiv:2604.27354v1 Announce Type: new Abstract: Explainable AI (XAI) aims to improve user understanding and decisions when using AI models. However, despite innovations in XAI, recent user evaluations

safetyarxiv-cs-ai

D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery

arXiv:2604.27977v1 Announce Type: new Abstract: Despite recent progress in language models and agents for scientific data-driven discovery, further advancing their capabilities is held back by the abs

safetyarxiv-cs-ai

From Test-taking to Cognitive Scaffolding: A Pedagogical Diagnostic Benchmark for LLMs on English Standardized Tests

arXiv:2505.17056v2 Announce Type: replace-cross Abstract: As large language models (LLMs) are increasingly integrated into educational tools, current evaluations on standardized tests predominantly fo

Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations

arXiv:2604.27093v1 Announce Type: cross Abstract: Current LLM safety alignment techniques improve model robustness against adversarial attacks, but overlook whether and how LLMs can recover helpfulnes

Applications

When Your LLM Reaches End-of-Life: A Framework for Confident Model Migration in Production Systems

arXiv:2604.27082v1 Announce Type: new Abstract: We present a framework for migrating production Large Language Model (LLM) based systems when the underlying model reaches end-of-life or requires repla

applicationsarxiv-cs-ai

Reasoning Gets Harder for LLMs Inside A Dialogue

arXiv:2603.20133v2 Announce Type: replace Abstract: Large Language Models (LLMs) achieve strong performance on many reasoning benchmarks, yet these evaluations typically focus on isolated tasks that d

30 Apr 2026

Risk Reporting for Developers' Internal AI Model Use

arXiv:2604.24966v1 Announce Type: cross Abstract: Frontier AI companies first deploy their most advanced models internally, for weeks or months of safety testing, evaluation, and iteration, before a p

safetyarxiv-cs-ai

30 Apr 2026

Safety Is Not Universal: The Selective Safety Trap in LLM Alignment

arXiv:2601.04389v2 Announce Type: replace-cross Abstract: Current safety evaluations of large language models (LLMs) create a dangerous illusion of universal protection by aggregating harms under gene

30 Apr 2026

Tutorials

Assessment of the quantitative impact of occlusal positioning splints on temporomandibular joint conditions

arXiv:2604.25322v1 Announce Type: new Abstract: A computational method for quantitative analysis of temporomandibular joint (TMJ) configuration using occlusal positioning splints is proposed and demon

tutorialsarxiv-cs-cv

BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks

arXiv:2604.24955v1 Announce Type: new Abstract: As benchmarks grow in complexity, many apparent agent failures are not failures of the agent at all - they are failures of the benchmark itself: broken

Feasible-First Exploration for Constrained ML Deployment Optimization in Crash-Prone Hierarchical Search Spaces

model-releasesarxiv-cs-lg

arXiv:2604.25073v1 Announce Type: new Abstract: Deploying machine learning models under production constraints requires joint optimization over model family, quantization scheme, runtime backend, and

Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

model-releasesarxiv-cs-ro

arXiv:2601.02078v2 Announce Type: replace Abstract: The development of robust and generalizable robot learning models is critically contingent upon the availability of large-scale, diverse training da

Limited Linguistic Diversity in Embodied AI Datasets

arXiv:2601.03136v2 Announce Type: replace Abstract: Language plays a critical role in Vision-Language-Action (VLA) models, yet the linguistic characteristics of the datasets used to train and evaluate

researcharxiv-cs-cl

OMHBench: Benchmarking Balanced and Grounded Omni-Modal Multi-Hop Reasoning

arXiv:2508.16198v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have increasingly supported omni-modal processing across text, vision, and speech. However, existing evalua

Agents

TEACar: An Open-Source Autonomous Driving Platform

arXiv:2604.24934v1 Announce Type: new Abstract: Intelligent Transportation Systems (ITS) increasingly rely on vision-based perception and learning-based control, necessitating experimental platforms t

agentsarxiv-cs-ro

The Forensic Cost of Watermark Removal

model-releasesarxiv-cs-cv

arXiv:2604.25491v1 Announce Type: new Abstract: Current watermark removal methods are evaluated on two axes: attack success rate and perceptual quality. We show this is insufficient. While state-of-th

Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation

arXiv:2511.21517v2 Announce Type: replace Abstract: Unlike text, speech conveys information about the speaker, such as gender, through acoustic cues like pitch. This gives rise to modality-specific bi

safetyarxiv-cs-cl

Agents

A Systematic Approach for Large Language Models Debugging

arXiv:2604.23027v1 Announce Type: new Abstract: Large language models (LLMs) have become central to modern AI workflows, powering applications from open-ended text generation to complex agent-based re

agentsarxiv-cs-ai

GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems

arXiv:2604.24477v1 Announce Type: cross Abstract: The rapid integration of Large Language Models (LLMs) into Multi-Agent Systems (MAS) has significantly enhanced their collaborative problem-solving ca

HalalBench: A Multilingual OCR Benchmark for Food Packaging Ingredient Extraction

arXiv:2604.22754v1 Announce Type: cross Abstract: No standardized benchmark exists for evaluating OCR on food packaging, despite its critical role in automated halal food verification. Existing benchm

In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions

arXiv:2604.22817v1 Announce Type: cross Abstract: Recent advances in speech-aware language models have coupled strong acoustic encoders with large language models, enabling systems that move beyond tr

safetyarxiv-cs-cl

JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems

arXiv:2604.23478v1 Announce Type: new Abstract: Large language models are increasingly deployed as automated judges for evaluating other models, yet the stability of their verdicts under semantically

Large Language Model based Interactive Decision-Making for Autonomous Driving

arXiv:2604.23513v1 Announce Type: new Abstract: In high-conflict mixed-traffic scenarios involving human-driven and autonomous vehicles, most existing autonomous driving systems default to overly cons

safetyarxiv-cs-ro

LLM4SCREENLIT: Recommendations on Assessing the Performance of Large Language Models for Screening Literature in Systematic Reviews

arXiv:2511.12635v2 Announce Type: replace-cross Abstract: Context: Large language models (LLMs) are increasingly used to screen literature for systematic reviews (SRs), but the standard confusion-matr

Point Cloud Registration for Fusion between SPECT MPI and CTA Images

arXiv:2604.24524v1 Announce Type: new Abstract: Clinical fusion of Single Photon Emission Computed Tomography Myocardial Perfusion Imaging (SPECT MPI) and Computed Tomography Angiography (CTA) remains

researcharxiv-cs-cv

Probing CLIP's Comprehension of 360-Degree Textual and Visual Semantics

arXiv:2604.24642v1 Announce Type: new Abstract: The dream of instantly creating rich 360-degree panoramic worlds from text is rapidly becoming a reality, yet a crucial gap exists in our ability to rel

safetyarxiv-cs-cv

Self-Supervised Learning for Android Malware Detection on a Time-Stamped Dataset

arXiv:2604.23025v1 Announce Type: cross Abstract: Android malware detectors built with machine learning often suffer from temporal bias: models are trained and evaluated without respecting apps' actua

safetyarxiv-cs-lg

SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents

arXiv:2512.07538v3 Announce Type: replace Abstract: Recognizing semantic differences across documents is crucial for text generation evaluation and content alignment, especially in cross-lingual setti

The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers

arXiv:2604.24155v1 Announce Type: cross Abstract: The quest to align machine behavior with human values raises fundamental questions about the moral frameworks that should govern AI decision-making. M

The Collapse of Heterogeneity in Silicon Philosophers

arXiv:2604.23575v1 Announce Type: cross Abstract: Silicon samples are increasingly used as a low-cost substitute for human panels and have been shown to reproduce aggregate human opinion with high fid

safetyarxiv-cs-cl

The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applications

arXiv:2604.24668v1 Announce Type: new Abstract: Given the increased use of LLMs in financial systems today, it becomes important to evaluate the safety and robustness of such systems. One failure mode

Zero-shot Large Language Models for Automatic Readability Assessment

arXiv:2604.24470v1 Announce Type: new Abstract: Unsupervised automatic readability assessment (ARA) methods have important practical and research applications (e.g., ensuring medical or educational ma

researcharxiv-cs-cl

A Kinematic Analysis of Palm Degrees of Freedom for Enhancing Thumb Opposability in Robotic Hands

arXiv:2604.22283v1 Announce Type: new Abstract: This study investigates the kinematic role of palm degrees of freedom (DoF) in enhancing thumb opposability in a five-finger robotic hand. A hand model

researcharxiv-cs-ro

An Artifact-based Agent Framework for Adaptive and Reproducible Medical Image Processing

arXiv:2604.21936v1 Announce Type: new Abstract: Medical imaging research is increasingly shifting from controlled benchmark evaluation toward real-world clinical deployment. In such settings, applying

Assessing the impact of dimensionality reduction on clustering performance -- a systematic study

arXiv:2604.22099v1 Announce Type: new Abstract: Dimensionality reduction is a critical preprocessing step for clustering high-dimensional data, yet comprehensive evaluation of its impact across divers

researcharxiv-cs-lg

Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?

arXiv:2508.03963v4 Announce Type: replace Abstract: Uncovering hidden symbolic laws from time series data, as an aspiration dating back to Kepler's discovery of planetary motion, remains a core challe