Search: “evaluation”

Retrieval-Augmented Long-Context Translation for Cultural Image Captioning: Gators submission for AmericasNLP 2026 shared task

model-releasesarxiv-cs-cl

arXiv:2605.20626v1 Announce Type: new Abstract: We present the University of Florida Gators submission to the AmericasNLP 2026 shared task on cultural image captioning for Indigenous languages. Our tw

21 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Under Pressure: Emotional Framing Induces Measurable Behavioral Shifts and Structured Internal Geometry in Small Language Models

DGX agent

arXiv:2605.20202v1 Announce Type: new Abstract: I study whether emotionally framed evaluation follow-ups change both the behavior and the calm-relative internal representations of small, locally deplo

model-releasesarxiv-cs-cl

21 May 2026

Local Ai

You Don't Need Attention: Gated Convolutional Modeling for Watch-Based Fall Detection

arXiv:2605.20275v1 Announce Type: new Abstract: Existing deep learning approaches for wearable fall detection systems rely on self-attention mechanisms that impose quadratic computational overhead, di

local-aiarxiv-cs-cv

21 May 2026

A Systematic Failure Analysis of Vision Foundation Models for Open Set Iris Presentation Attack Detection

model-releasesarxiv-cs-cv

arXiv:2605.19020v1 Announce Type: new Abstract: Vision foundation models have demonstrated strong transferability across diverse visual recognition tasks and are increasingly considered for biometric

An LLM-Based System for Argument Mining

model-releasesarxiv-cs-cl

arXiv:2605.13793v2 Announce Type: replace Abstract: Arguments are a fundamental aspect of human reasoning, in which claims are supported, challenged, and weighed against one another. We present an end

Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries

arXiv:2605.18891v1 Announce Type: cross Abstract: Evaluations of unlearning on reasoning models sometimes show a bypass pattern. The answer side looks unlearned, but the model's own thinking trace kee

Applications

Automated Big Data Quality Assessment using Knowledge Graph Embeddings

arXiv:2605.18833v1 Announce Type: cross Abstract: Automated data quality assessment is crucial for managing big data, but existing solutions face challenges in achieving accurate context-aware assessm

applicationsarxiv-cs-ai

Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs

arXiv:2605.19043v1 Announce Type: cross Abstract: Automated grading systems have enabled scalable assessment for many response types, but handwritten mathematics remains a barrier due to the complexit

Automatically Improving Simulation Physics for Articulated Objects

arXiv:2605.19136v1 Announce Type: new Abstract: Simulation is a central tool for scalable robot learning, but its effectiveness depends on the quality of object assets. While modern 3D datasets provid

safetyarxiv-cs-ro

Benchmarking Commercial ASR Systems on Code-Switching Speech: Arabic, Persian, and German

arXiv:2605.19069v1 Announce Type: cross Abstract: Code-switching -- the natural alternation between two languages within a single utterance -- represents one of the most challenging and under-studied

Applications

Beyond Leakage and Complexity: Towards Realistic and Efficient Information Cascade Prediction

arXiv:2510.25348v2 Announce Type: replace Abstract: Information cascade popularity prediction is a key problem in analyzing content diffusion in social networks. However, current related works suffer

applicationsarxiv-cs-lg

CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning

arXiv:2507.15698v2 Announce Type: replace-cross Abstract: Process Reward Models (PRMs) play a central role in evaluating and guiding multi-step reasoning in large language models (LLMs), especially fo

CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing

arXiv:2605.19484v1 Announce Type: cross Abstract: While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workfl

Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

arXiv:2605.20145v1 Announce Type: cross Abstract: Bayesian optimization (BO) selects evaluation points for expensive black-box objectives using Gaussian process (GP) predictive distributions. Kernel c

researcharxiv-cs-lg

Guiding Neuro-Symbolic Scenario Generation with Spatio-Temporal Logic

arXiv:2605.19038v1 Announce Type: cross Abstract: The rapid advancement of autonomous driving (AD) technologies has outpaced the development of robust safety evaluation methods. Conventional testing r

safetyarxiv-cs-lg

How do LLMs Compute Verbal Confidence

arXiv:2603.17839v3 Announce Type: replace-cross Abstract: Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from

Knowing When Not to Predict: Self Supervised Learning and Abstention for Safer DR Screening

arXiv:2605.19133v1 Announce Type: cross Abstract: Self-supervised learning (SSL) is now a standard way to pretrain medical image models, but performance is still mostly judged by downstream accuracy.

Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanations

arXiv:2605.19274v1 Announce Type: new Abstract: LLMs deployed multilingually are often audited via English explanations for non-English inputs. We evaluate extractive explanations ''where the model id

researcharxiv-cs-cl

MANGO: Meta-Adaptive Network Gradient Optimization for Online Continual Learning

arXiv:2605.19080v1 Announce Type: cross Abstract: In Online Continual Learning (OCL), a neural network sequentially learns from a non-stationary data stream in a single-pass with access only to a limi

Markov Chain Decoders Overcome the Heavy-Tail Limitations of Lipschitz Generative Models

arXiv:2605.18931v1 Announce Type: cross Abstract: Heavy-tailed distributions are prevalent in performance evaluation, network traffic, and risk modeling. This behavior poses a fundamental challenge fo

OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments

arXiv:2605.18758v1 Announce Type: cross Abstract: Current benchmarks for graphical user interface (GUI) agents predominantly rely on static screenshots. However, real-world smartphone interaction rout

Agents

PAVE: A Cognitive Architecture for Legitimate Violation in Generative Agent Societies

arXiv:2605.19351v1 Announce Type: cross Abstract: Generative agents based on large language models reproduce believable human behavior in cooperative settings, but how they should reason in situations

agentsarxiv-cs-ai

Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering

arXiv:2605.19220v1 Announce Type: cross Abstract: Uncertainty Quantification (UQ) is widely regarded as the primary safeguard for deploying Large Language Models (LLMs) in high-stakes domains. However

RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents

arXiv:2605.18805v1 Announce Type: cross Abstract: LLM recommendation agents increasingly produce structured recommendation reports: sets of items accompanied by natural-language justifications. Yet ex

The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility

model-releasesarxiv-cs-lg

arXiv:2605.19537v1 Announce Type: new Abstract: Progress in LLMs is increasingly measured through standardized benchmarks, where state-of-the-art improvements are often separated by fractions of a per

What Do Evolutionary Coding Agents Evolve?

arXiv:2605.20086v1 Announce Type: cross Abstract: Recent work pairs LLMs with evolutionary search to iteratively generate, modify, and select code using task-specific feedback. These systems have prod

A New Perspective on Precision and Recall for Generative Models

arXiv:2511.02414v2 Announce Type: replace Abstract: With the recent success of generative models in image and text, the question of their evaluation has recently gained a lot of attention. While most

Adversarial Fragility and Language Vulnerability in Clinical AI: A Systematic Audit of Diagnostic Collapse Under Imperceptible Perturbations and Cross-Lingual Drift in Low-Resource Healthcare Settings

arXiv:2605.16993v1 Announce Type: cross Abstract: Current clinical artificial intelligence (AI) systems are evaluated almost exclusively on clean, standardised, English-language inputs, conditions tha

Agent Bazaar: Enabling Economic Alignment in Multi-Agent Marketplaces

arXiv:2605.17698v1 Announce Type: new Abstract: The deployment of Large Language Models (LLMs) as autonomous economic agents introduces systemic risks that extend beyond individual capability failures

safetyarxiv-cs-lg

ANVIL: Analogies and Videos for Lecturers

arXiv:2605.16295v1 Announce Type: cross Abstract: We present ANVIL, a multimodal generative system that automates the production of analogy-based instructional animations for computer science topics.

tutorialsarxiv-cs-ai

Artificial Intolerance: Stigmatizing Language in Clinical Documentation Skews Large Language Model Decision-Making

arXiv:2605.17228v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as clinical decision support and medical documentation. However, the

safetyarxiv-cs-cl

ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents

arXiv:2605.17324v1 Announce Type: cross Abstract: Clarification-seeking behavior is widely regarded as a desirable property of LLM agents, enabling them to resolve ambiguity before acting on underspec

Benchmarking inference at scale: coding agents

model-releasestogether-ai-blog

This article presents benchmarking results for AI coding agents evaluated at scale, likely comparing performance metrics such as code generation accuracy, execution success rates, and inference effici

Beyond Accuracy: Robustness, Interpretability and Expressiveness of EEG Foundation Models

arXiv:2605.17562v1 Announce Type: cross Abstract: EEG foundation models (EEG-FMs) have been evaluated predominantly on clean, in-distribution accuracy, leaving their robustness, interpretability and r

Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs

arXiv:2507.01533v2 Announce Type: replace-cross Abstract: We prove consistency of a recently proposed scheme that evaluates expected values by composing a learned transport map with Clenshaw--Curtis s

tutorialsarxiv-cs-lg

Agents

CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery

arXiv:2604.01658v2 Announce Type: replace Abstract: Large language model (LLM)-based evolution is a promising approach for open-ended discovery, where progress requires sustained search and knowledge

agentsarxiv-cs-ai

Could Large Language Models work as Post-hoc Explainability Tools in Credit Risk Models?

model-releasesarxiv-cs-lg

arXiv:2602.18895v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown promise in translating model-based explanations into human-readable narratives. This study evaluates w

Local Ai

Cross-Source Supervision for Bone Infection Segmentation in Dual-Modality PET-CT

arXiv:2605.16373v1 Announce Type: cross Abstract: Early and accurate diagnosis and lesion localization of bone infections are crucial for clinical treatment. PET-CT integrates anatomical information f

local-aiarxiv-cs-ai

CrossView Suite: Harnessing Cross-view Spatial Intelligence of MLLMs with Dataset, Model and Benchmark

arXiv:2605.18621v1 Announce Type: cross Abstract: Spatial intelligence requires multimodal large language models (MLLMs) to move beyond single-view perception and reason consistently about objects, vi

CT-DegradBench: A Physics-Informed Benchmark for CT Degradation Detection and Severity Estimation

model-releasesarxiv-cs-cv

arXiv:2605.16431v1 Announce Type: new Abstract: Computed tomography (CT) images are frequently degraded by acquisition artifacts, including noise, blur, streaking, aliasing, and metal artifacts. Yet C

Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence?

arXiv:2605.16292v1 Announce Type: cross Abstract: Perceptions of intelligence shape how learners evaluate and rely on artificial intelligence (AI) systems. Despite rapid advances in AI capabilities, t

tutorialsarxiv-cs-ai

Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

arXiv:2511.20857v2 Announce Type: replace-cross Abstract: Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving. This makes memory a critica

Exploring Trust Calibration in XAI - The Impact of Exposing Model Limitations to Lay Users

arXiv:2605.18036v1 Announce Type: cross Abstract: Trust calibration -- aligning user trust judgment with model capability -- is crucial for safe deployment of explainable AI (XAI), yet is often evalua

Flow-Direct: Feedback-Efficient and Reusable Guidance for Flow Models via Non-Parametric Guidance Field

arXiv:2605.16348v1 Announce Type: cross Abstract: Training-free guidance enables pre-trained diffusion and flow models to optimize application-specific objectives using feedback from external black-bo

Applications

From Documents to Segments: A Contextual Reformulation for Topic Assignment

arXiv:2605.17714v1 Announce Type: new Abstract: Traditional topic modeling assigns a single topic to each document. In practice, however, many real-world documents, such as product reviews or open-end

applicationsarxiv-cs-cl

From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction

arXiv:2605.16927v1 Announce Type: new Abstract: Clinical decision-making is a feedback system where risk estimates influence treatment, which in turn changes disease trajectories, and both shape clini

HalluScore: Large Language Model Hallucination Question Answering Benchmark

model-releasesarxiv-cs-cl

arXiv:2605.17007v1 Announce Type: new Abstract: Large language models (LLMs) have achieved remarkable progress in natural language generation, but remain susceptible to hallucination. In response to g

How to run evals for the model router

tutorialsmicrosoft-foundry

Walk through running quality, cost, and latency evaluations for the Foundry model router using an open-source GitHub repo designed for router-aware eval pipelines. The post How to run evals for the mo