Search: “evaluation”

Epistemic Constitutionalism Or: how to avoid coherence bias

arXiv:2601.14295v3 Announce Type: replace Abstract: Large language models increasingly function as artificial reasoners: they evaluate arguments, assign credibility, and express confidence. Yet their

safetyarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Evian: Towards Explainable Visual Instruction-tuning Data Auditing

DGX agent

arXiv:2604.20544v1 Announce Type: cross Abstract: The efficacy of Large Vision-Language Models (LVLMs) is critically dependent on the quality of their training data, requiring a precise balance betwee

model-releasesarxiv-cs-ai

23 Apr 2026

Exploiting LLM-as-a-Judge Disposition on Free Text Legal QA via Prompt Optimization

arXiv:2604.20726v1 Announce Type: cross Abstract: This work explores the role of prompt design and judge selection in LLM-as-a-Judge evaluations of free text legal question answering. We examine wheth

Fairness Testing of Large Language Models in Role-Playing

arXiv:2411.00585v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A cr

Generalization and Membership Inference Attack a Practical Perspective

arXiv:2604.19936v1 Announce Type: cross Abstract: With the emergence of new evaluation metrics and attack methodologies for Membership Inference Attacks (MIA), it becomes essential to reevaluate previ

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

arXiv:2604.20382v1 Announce Type: new Abstract: Rising demand for mental health support has increased interest in using Large Language Models (LLMs) for counseling. However, adapting LLMs to this high

safetyarxiv-cs-cl

Human-like Content Analysis for Generative AI with Language-Grounded Sparse Encoders

arXiv:2508.18236v4 Announce Type: replace Abstract: The rapid development of generative AI has transformed content creation, communication, and human development. However, this technology raises profo

applicationsarxiv-cs-cv

If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems

arXiv:2604.19844v1 Announce Type: cross Abstract: Recent advances in embodied Vision-Language Agentic Systems (VLAS), powered by large vision-language models (LVLMs), enable AI systems to perceive and

agentsarxiv-cs-ai

Inference Headroom Ratio: A Diagnostic and Control Framework for Inference Stability Under Constraint

arXiv:2604.19760v1 Announce Type: new Abstract: We present a simulation-based evaluation of the Inference Headroom Ratio (IHR), a dimensionless diagnostic quantity for characterizing inference stabili

Interpretability from the Ground Up: Stakeholder-Centric Design of Automated Scoring in Educational Assessments

arXiv:2511.17069v3 Announce Type: replace Abstract: AI-driven automated scoring systems offer scalable and efficient means of evaluating complex student-generated responses. Yet, despite increasing de

applicationsarxiv-cs-cl

Intersectional Fairness in Large Language Models

arXiv:2604.20677v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in socially sensitive settings, raising concerns about fairness and biases, particularly across i

KD-Judge: A Knowledge-Driven Automated Judge Framework for Functional Fitness Movements on Edge Devices

arXiv:2604.19834v1 Announce Type: new Abstract: Functional fitness movements are widely used in training, competition, and health-oriented exercise programs, yet consistently enforcing repetition (rep

researcharxiv-cs-cv

Lever: Inference-Time Policy Reuse under Support Constraints

arXiv:2604.20174v1 Announce Type: new Abstract: Reinforcement learning (RL) policies are typically trained for fixed objectives, making reuse difficult when task requirements change. We study inferenc

safetyarxiv-cs-lg

MedSkillAudit: A Domain-Specific Audit Framework for Medical Research Agent Skills

arXiv:2604.20441v1 Announce Type: new Abstract: Background: Agent skills are increasingly deployed as modular, reusable capability units in AI agent systems. Medical research agent skills require safe

safetyarxiv-cs-ai

Relative Entropy Estimation in Function Space: Theory and Applications to Trajectory Inference

model-releasesarxiv-cs-lg

arXiv:2604.20775v1 Announce Type: new Abstract: Trajectory Inference (TI) seeks to recover latent dynamical processes from snapshot data, where only independent samples from time-indexed marginals are

Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem

arXiv:2604.20805v1 Announce Type: cross Abstract: The value alignment problem for artificial intelligence (AI) is often framed as a purely technical or normative challenge, sometimes focused on hypoth

safetyarxiv-cs-ai

RExBench: Can coding agents autonomously implement AI research extensions?

arXiv:2506.22598v3 Announce Type: replace Abstract: Agents based on Large Language Models (LLMs) have shown promise for performing sophisticated software engineering tasks autonomously. In addition, t

Robustness of Spatio-temporal Graph Neural Networks for Fault Location in Partially Observable Distribution Grids

arXiv:2604.20403v1 Announce Type: new Abstract: Fault location in distribution grids is critical for reliability and minimizing outage durations. Yet, it remains challenging due to partial observabili

researcharxiv-cs-lg

SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark

arXiv:2604.20319v1 Announce Type: new Abstract: Fine-grained spatiotemporal reasoning on surgical videos is critical, yet the capabilities of Multi-modal Large Language Models (MLLMs) in this domain r

What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization

arXiv:2601.17609v2 Announce Type: replace Abstract: In domains like medicine and finance, large-scale labeled data is costly and often unavailable, leading to models trained on small datasets that str

applicationsarxiv-cs-cl

X-Cache: Cross-Chunk Block Caching for Few-Step Autoregressive World Models Inference

arXiv:2604.20289v1 Announce Type: new Abstract: Real-time world simulation is becoming a key infrastructure for scalable evaluation and online reinforcement learning of autonomous driving systems. Rec

agentsarxiv-cs-cv

A Dual Perspective on Synthetic Trajectory Generators: Utility Framework and Privacy Vulnerabilities

arXiv:2604.19653v1 Announce Type: new Abstract: Human mobility data are used in numerous applications, ranging from public health to urban planning. Human mobility is inherently sensitive, as it can c

Are Large Language Models Economically Viable for Industry Deployment?

arXiv:2604.19342v1 Announce Type: new Abstract: Generative AI-powered by Large Language Models (LLMs)-is increasingly deployed in industry across healthcare decision support, financial analytics, ente

BALTIC: A Benchmark and Cross-Domain Strategy for 3D Reconstruction Across Air and Underwater Domains Under Varying Illumination

arXiv:2604.19133v1 Announce Type: new Abstract: Robust 3D reconstruction across varying environmental conditions remains a critical challenge for robotic perception, particularly when transitioning be

Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Models

arXiv:2604.18751v1 Announce Type: cross Abstract: Nonlinear machine-learning models are increasingly used to discover causal relationships in time-series data, yet the interpretation of their outputs

applicationsarxiv-cs-ai

Beyond Linear Probes: Dynamic Safety Monitoring for Language Models

arXiv:2509.26238v4 Announce Type: replace Abstract: Monitoring large language models' (LLMs) activations is an effective way to detect harmful requests before they lead to unsafe outputs. However, tra

safetyarxiv-cs-lg

Comparing energy consumption and accuracy in text classification inference

arXiv:2508.14170v2 Announce Type: replace Abstract: The increasing deployment of large language models (LLMs) in natural language processing (NLP) tasks raises concerns about energy efficiency and sus

researcharxiv-cs-cl

Concept Inconsistency in Dermoscopic Concept Bottleneck Models: A Rough-Set Analysis of the Derm7pt Dataset

arXiv:2604.19323v1 Announce Type: cross Abstract: Concept Bottleneck Models (CBMs) route predictions exclusively through a clinically grounded concept layer, binding interpretability to concept-label

DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning

arXiv:2604.18964v1 Announce Type: new Abstract: This paper introduces DW-Bench, a new benchmark that evaluates large language models (LLMs) on graph-topology reasoning over data warehouse schemas, exp

Energy-Weighted Flow Matching: Unlocking Continuous Normalizing Flows for Efficient and Scalable Boltzmann Sampling

model-releasesarxiv-cs-lg

arXiv:2509.03726v2 Announce Type: replace-cross Abstract: Sampling from unnormalized target distributions, e.g. Boltzmann distributions mu_{ext{target}}(x) propto exp(-E(x)/T), is fundamental to many

Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model

arXiv:2604.18831v1 Announce Type: new Abstract: Frame-wise semantic segmentation of indoor lidar scans is a fundamental step toward higher-level 3D scene understanding and mapping applications. Howeve

agentsarxiv-cs-cv

Has Automated Essay Scoring Reached Sufficient Accuracy? Deriving Achievable QWK Ceilings from Classical Test Theory

arXiv:2604.19131v1 Announce Type: new Abstract: Automated essay scoring (AES) is commonly evaluated on public benchmarks using quadratic weighted kappa (QWK). However, because benchmark labels are ass

How Far Are Video Models from True Multimodal Reasoning?

arXiv:2604.19193v1 Announce Type: new Abstract: Despite remarkable progress toward general-purpose video models, a critical question remains unanswered: how far are these models from achieving true mu

LSTM-MAS: A Long Short-Term Memory Inspired Multi-Agent System for Long-Context Understanding

arXiv:2601.11913v2 Announce Type: replace-cross Abstract: Effectively processing long contexts remains a fundamental yet unsolved challenge for large language models (LLMs). Existing single-LLM-based

Mesh Memory Protocol: Semantic Infrastructure for Multi-Agent LLM Systems

arXiv:2604.19540v1 Announce Type: cross Abstract: Teams of LLM agents increasingly collaborate on tasks spanning days or weeks: multi-day data-generation sprints where generator, reviewer, and auditor

agentsarxiv-cs-ai

Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes

arXiv:2604.19318v1 Announce Type: new Abstract: Multi-view crowd tracking estimates each person's tracking trajectories on the ground of the scene. Recent research works mainly rely on CNNs-based mult

applicationsarxiv-cs-cv

Local Ai

Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agents

arXiv:2604.00555v2 Announce Type: replace Abstract: Enterprise adoption of Large Language Models (LLMs) is constrained by hallucination, domain drift, and the inability to enforce regulatory complianc

local-aiarxiv-cs-ai

Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications

arXiv:2411.06837v2 Announce Type: replace Abstract: The rapid rise of Large Language Models (LLMs) has created new disruptive possibilities for persuasive communication, enabling fully-automated, pers

safetyarxiv-cs-cl

RESFL: An Uncertainty-Aware Framework for Responsible Federated Learning by Balancing Privacy, Fairness and Utility

arXiv:2503.16251v2 Announce Type: replace-cross Abstract: Federated Learning (FL) has gained prominence in machine learning applications across critical domains by enabling collaborative model trainin

safetyarxiv-cs-cv

Seeing Candidates at Scale: Multimodal LLMs for Visual Political Communication on Instagram

arXiv:2604.19489v1 Announce Type: new Abstract: This paper presents a computational case study that evaluates the capabilities of specialized machine learning models and emerging multimodal large lang

applicationsarxiv-cs-cv

Sentipolis: Emotion-Aware Agents for Social Simulations

arXiv:2601.18027v2 Announce Type: replace Abstract: LLM agents are increasingly used for social simulation, yet emotion is often treated as a transient cue, causing emotional amnesia and weak long-hor

Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation

arXiv:2604.18758v1 Announce Type: new Abstract: Low-resource machine translation requires methods that differ from those used for high-resource languages. This paper proposes a novel in-context learni

researcharxiv-cs-cl

The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models

arXiv:2604.19139v1 Announce Type: cross Abstract: As Large Language Models (LLMs) continue to evolve through alignment techniques such as Reinforcement Learning from Human Feedback (RLHF) and Constitu

Toward Clinically Acceptable Chest X-ray Report Generation: A Qualitative Retrospective Pilot Study of CXRMate-2

arXiv:2604.18967v1 Announce Type: new Abstract: Chest X-ray (CXR) radiology report generation (RRG) models have shown rapid progress, yet their clinical utility remains uncertain due to limited evalua

safetyarxiv-cs-cv

Towards Generalization of Graph Neural Networks for AC Optimal Power Flow

arXiv:2510.06860v2 Announce Type: replace-cross Abstract: AC Optimal Power Flow (ACOPF) is computationally intensive for large-scale grids, often requiring prohibitive solution times with conventional

VISTA: Verification In Sequential Turn-based Assessment

arXiv:2510.27052v5 Announce Type: replace Abstract: Hallucination--defined here as generating statements unsupported or contradicted by available evidence or conversational context--remains a major ob

researcharxiv-cs-cl

Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India

arXiv:2604.19151v1 Announce Type: new Abstract: Existing Indic ASR benchmarks often use scripted, clean speech and leaderboard driven evaluation that encourages dataset specific overfitting. In additi

Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety

arXiv:2604.18487v1 Announce Type: new Abstract: The Adversarial Humanities Benchmark (AHB) evaluates whether model safety refusals survive a shift away from familiar harmful prompt forms. Starting fro