Search: “evaluation”

MedSkillAudit: A Domain-Specific Audit Framework for Medical Research Agent Skills

arXiv:2604.20441v1 Announce Type: new Abstract: Background: Agent skills are increasingly deployed as modular, reusable capability units in AI agent systems. Medical research agent skills require safe

safetyarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Relative Entropy Estimation in Function Space: Theory and Applications to Trajectory Inference

DGX agent

arXiv:2604.20775v1 Announce Type: new Abstract: Trajectory Inference (TI) seeks to recover latent dynamical processes from snapshot data, where only independent samples from time-indexed marginals are

model-releasesarxiv-cs-lg

23 Apr 2026

Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem

arXiv:2604.20805v1 Announce Type: cross Abstract: The value alignment problem for artificial intelligence (AI) is often framed as a purely technical or normative challenge, sometimes focused on hypoth

safetyarxiv-cs-ai

RExBench: Can coding agents autonomously implement AI research extensions?

arXiv:2506.22598v3 Announce Type: replace Abstract: Agents based on Large Language Models (LLMs) have shown promise for performing sophisticated software engineering tasks autonomously. In addition, t

Robustness of Spatio-temporal Graph Neural Networks for Fault Location in Partially Observable Distribution Grids

arXiv:2604.20403v1 Announce Type: new Abstract: Fault location in distribution grids is critical for reliability and minimizing outage durations. Yet, it remains challenging due to partial observabili

researcharxiv-cs-lg

SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark

arXiv:2604.20319v1 Announce Type: new Abstract: Fine-grained spatiotemporal reasoning on surgical videos is critical, yet the capabilities of Multi-modal Large Language Models (MLLMs) in this domain r

What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization

arXiv:2601.17609v2 Announce Type: replace Abstract: In domains like medicine and finance, large-scale labeled data is costly and often unavailable, leading to models trained on small datasets that str

applicationsarxiv-cs-cl

X-Cache: Cross-Chunk Block Caching for Few-Step Autoregressive World Models Inference

arXiv:2604.20289v1 Announce Type: new Abstract: Real-time world simulation is becoming a key infrastructure for scalable evaluation and online reinforcement learning of autonomous driving systems. Rec

agentsarxiv-cs-cv

A Dual Perspective on Synthetic Trajectory Generators: Utility Framework and Privacy Vulnerabilities

arXiv:2604.19653v1 Announce Type: new Abstract: Human mobility data are used in numerous applications, ranging from public health to urban planning. Human mobility is inherently sensitive, as it can c

researcharxiv-cs-ai

Are Large Language Models Economically Viable for Industry Deployment?

arXiv:2604.19342v1 Announce Type: new Abstract: Generative AI-powered by Large Language Models (LLMs)-is increasingly deployed in industry across healthcare decision support, financial analytics, ente

Industry

Are LLM agents good at join order optimization?

This Databricks article examines whether Large Language Model agents can effectively optimize SQL join order, a critical query optimization problem in database systems. The analysis likely evaluates L

industrydatabricks

BALTIC: A Benchmark and Cross-Domain Strategy for 3D Reconstruction Across Air and Underwater Domains Under Varying Illumination

arXiv:2604.19133v1 Announce Type: new Abstract: Robust 3D reconstruction across varying environmental conditions remains a critical challenge for robotic perception, particularly when transitioning be

Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Models

arXiv:2604.18751v1 Announce Type: cross Abstract: Nonlinear machine-learning models are increasingly used to discover causal relationships in time-series data, yet the interpretation of their outputs

applicationsarxiv-cs-ai

Beyond Linear Probes: Dynamic Safety Monitoring for Language Models

arXiv:2509.26238v4 Announce Type: replace Abstract: Monitoring large language models' (LLMs) activations is an effective way to detect harmful requests before they lead to unsafe outputs. However, tra

safetyarxiv-cs-lg

Comparing energy consumption and accuracy in text classification inference

arXiv:2508.14170v2 Announce Type: replace Abstract: The increasing deployment of large language models (LLMs) in natural language processing (NLP) tasks raises concerns about energy efficiency and sus

researcharxiv-cs-cl

Concept Inconsistency in Dermoscopic Concept Bottleneck Models: A Rough-Set Analysis of the Derm7pt Dataset

arXiv:2604.19323v1 Announce Type: cross Abstract: Concept Bottleneck Models (CBMs) route predictions exclusively through a clinically grounded concept layer, binding interpretability to concept-label

DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning

arXiv:2604.18964v1 Announce Type: new Abstract: This paper introduces DW-Bench, a new benchmark that evaluates large language models (LLMs) on graph-topology reasoning over data warehouse schemas, exp

Enabling the agentic enterprise: business and industry agents arrive in Gemini Enterprise

model-releasesgoogle-cloud-ai

We are officially moving past the era of one-size-fits-all AI. Enterprises today require highly specialized, role-specific tools to drive real productivity — but they cannot afford to sacrifice securi

Energy-Weighted Flow Matching: Unlocking Continuous Normalizing Flows for Efficient and Scalable Boltzmann Sampling

model-releasesarxiv-cs-lg

arXiv:2509.03726v2 Announce Type: replace-cross Abstract: Sampling from unnormalized target distributions, e.g. Boltzmann distributions mu_{ext{target}}(x) propto exp(-E(x)/T), is fundamental to many

Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model

arXiv:2604.18831v1 Announce Type: new Abstract: Frame-wise semantic segmentation of indoor lidar scans is a fundamental step toward higher-level 3D scene understanding and mapping applications. Howeve

agentsarxiv-cs-cv

Has Automated Essay Scoring Reached Sufficient Accuracy? Deriving Achievable QWK Ceilings from Classical Test Theory

arXiv:2604.19131v1 Announce Type: new Abstract: Automated essay scoring (AES) is commonly evaluated on public benchmarks using quadratic weighted kappa (QWK). However, because benchmark labels are ass

How Far Are Video Models from True Multimodal Reasoning?

arXiv:2604.19193v1 Announce Type: new Abstract: Despite remarkable progress toward general-purpose video models, a critical question remains unanswered: how far are these models from achieving true mu

LSTM-MAS: A Long Short-Term Memory Inspired Multi-Agent System for Long-Context Understanding

arXiv:2601.11913v2 Announce Type: replace-cross Abstract: Effectively processing long contexts remains a fundamental yet unsolved challenge for large language models (LLMs). Existing single-LLM-based

Mesh Memory Protocol: Semantic Infrastructure for Multi-Agent LLM Systems

arXiv:2604.19540v1 Announce Type: cross Abstract: Teams of LLM agents increasingly collaborate on tasks spanning days or weeks: multi-day data-generation sprints where generator, reviewer, and auditor

agentsarxiv-cs-ai

Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes

arXiv:2604.19318v1 Announce Type: new Abstract: Multi-view crowd tracking estimates each person's tracking trajectories on the ground of the scene. Recent research works mainly rely on CNNs-based mult

applicationsarxiv-cs-cv

Local Ai

Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agents

arXiv:2604.00555v2 Announce Type: replace Abstract: Enterprise adoption of Large Language Models (LLMs) is constrained by hallucination, domain drift, and the inability to enforce regulatory complianc

local-aiarxiv-cs-ai

Tools

Our reward design combines correctness, preference, and efficiency. Preference only counts when the answer is correct. This keeps the model …

Perplexity's reward design system prioritizes correctness as a foundational requirement, then evaluates user preference and efficiency only when answers meet correctness standards. This hierarchical a

toolsperplexity--x

Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications

arXiv:2411.06837v2 Announce Type: replace Abstract: The rapid rise of Large Language Models (LLMs) has created new disruptive possibilities for persuasive communication, enabling fully-automated, pers

safetyarxiv-cs-cl

RESFL: An Uncertainty-Aware Framework for Responsible Federated Learning by Balancing Privacy, Fairness and Utility

arXiv:2503.16251v2 Announce Type: replace-cross Abstract: Federated Learning (FL) has gained prominence in machine learning applications across critical domains by enabling collaborative model trainin

safetyarxiv-cs-cv

Seeing Candidates at Scale: Multimodal LLMs for Visual Political Communication on Instagram

arXiv:2604.19489v1 Announce Type: new Abstract: This paper presents a computational case study that evaluates the capabilities of specialized machine learning models and emerging multimodal large lang

applicationsarxiv-cs-cv

Sentipolis: Emotion-Aware Agents for Social Simulations

arXiv:2601.18027v2 Announce Type: replace Abstract: LLM agents are increasingly used for social simulation, yet emotion is often treated as a transient cue, causing emotional amnesia and weak long-hor

researcharxiv-cs-ai

Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation

arXiv:2604.18758v1 Announce Type: new Abstract: Low-resource machine translation requires methods that differ from those used for high-resource languages. This paper proposes a novel in-context learni

researcharxiv-cs-cl

The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models

arXiv:2604.19139v1 Announce Type: cross Abstract: As Large Language Models (LLMs) continue to evolve through alignment techniques such as Reinforcement Learning from Human Feedback (RLHF) and Constitu

Today we’re introducing two big steps for health at OpenAI: - ChatGPT for Clinicians, a free version of ChatGPT designed for clinical work -…

model-releasessam-altman--x

Today we’re introducing two big steps for health at OpenAI: - ChatGPT for Clinicians, a free version of ChatGPT designed for clinical work - HealthBench Professional, a new benchmark to evaluate real

Toward Clinically Acceptable Chest X-ray Report Generation: A Qualitative Retrospective Pilot Study of CXRMate-2

arXiv:2604.18967v1 Announce Type: new Abstract: Chest X-ray (CXR) radiology report generation (RRG) models have shown rapid progress, yet their clinical utility remains uncertain due to limited evalua

safetyarxiv-cs-cv

Towards Generalization of Graph Neural Networks for AC Optimal Power Flow

arXiv:2510.06860v2 Announce Type: replace-cross Abstract: AC Optimal Power Flow (ACOPF) is computationally intensive for large-scale grids, often requiring prohibitive solution times with conventional

researcharxiv-cs-ai

VISTA: Verification In Sequential Turn-based Assessment

arXiv:2510.27052v5 Announce Type: replace Abstract: Hallucination--defined here as generating statements unsupported or contradicted by available evidence or conversational context--remains a major ob

researcharxiv-cs-cl

Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India

arXiv:2604.19151v1 Announce Type: new Abstract: Existing Indic ASR benchmarks often use scripted, clean speech and leaderboard driven evaluation that encourages dataset specific overfitting. In additi

We are excited to launch VideoGameBench on Antim Labs, created by @a1zhang, Thomas L. Griffiths (@cocosci_lab), @karthik_r_n, and @OfirPress…

VideoGameBench is a new benchmark launched on Antim Labs, created by a1zhang, Thomas L. Griffiths, Karthik R. N, and Ofir Press. The benchmark likely evaluates AI model performance on video game-relat

agentsyohei-nakajima--x

Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety

arXiv:2604.18487v1 Announce Type: new Abstract: The Adversarial Humanities Benchmark (AHB) evaluates whether model safety refusals survive a shift away from familiar harmful prompt forms. Starting fro

Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs

arXiv:2601.13099v2 Announce Type: replace Abstract: Arabic is a highly diglossic language where most daily communication occurs in regional dialects rather than Modern Standard Arabic (MSA). Despite t

Tutorials

Argument Reconstruction as Supervision for Critical Thinking in LLMs

arXiv:2603.17432v2 Announce Type: replace Abstract: To think critically about arguments, human learners are trained to identify, reconstruct, and evaluate arguments. Argument reconstruction is especia

tutorialsarxiv-cs-cl

BASIL: Bayesian Assessment of Sycophancy in LLMs

arXiv:2508.16846v5 Announce Type: replace-cross Abstract: Sycophancy (overly agreeable or flattering behavior) poses a fundamental challenge for human-AI collaboration, particularly in high-stakes dec

applicationsarxiv-cs-cl

Benchmarking System Dynamics AI Assistants: Cloud Versus Local LLMs on CLD Extraction and Discussion

model-releasesarxiv-cs-lg

arXiv:2604.18566v1 Announce Type: cross Abstract: We present a systematic evaluation of large language model families -- spanning both proprietary cloud APIs and locally-hosted open-source models -- o

Beyond Reproduction: A Paired-Task Framework for Assessing LLM Comprehension and Creativity in Literary Translation

arXiv:2604.18169v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for creative tasks such as literary translation. Yet translational creativity remains underexplored a

BridgeEQA: Virtual Embodied Agents for Real Bridge Inspections

arXiv:2511.12676v2 Announce Type: replace Abstract: Deploying embodied agents that can answer questions about their surroundings in realistic real-world settings remains difficult, partly due to the s

Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA

arXiv:2604.17316v1 Announce Type: new Abstract: Safe clinical deployment of Large Language Models (LLMs) requires not only high accuracy but also robust uncertainty calibration to ensure models defer

applicationsarxiv-cs-cl

Concurrent Criterion Validation of a Validity Screen for LLM Confidence Signals via Selective Prediction

arXiv:2604.17716v1 Announce Type: new Abstract: The validity screen (Cacioli, 2026d, 2026e) classifies LLM confidence signals as Valid, Indeterminate, or Invalid. We test whether these classifications