Search: “evaluation”

CLARITY: A Framework and Benchmark for Conversational Language Ambiguity and Unanswerability in Interactive NL2SQL Systems

model-releasesarxiv-cs-cl

arXiv:2604.22313v1 Announce Type: new Abstract: NL2SQL systems deployed in industry settings often encounter ambiguous or unanswerable queries, particularly in interactive scenarios with incomplete us

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Eidolon: A Post-Quantum Signature Scheme Based on k-Colorability in the Age of Graph Neural Networks

DGX agent

arXiv:2602.02689v2 Announce Type: replace-cross Abstract: We propose Eidolon, a post-quantum signature scheme grounded on the NP-complete k-colorability problem. Our construction generalizes the Goldr

researcharxiv-cs-ai

27 Apr 2026

Estimating Tail Risks in Language Model Output Distributions

arXiv:2604.22167v1 Announce Type: cross Abstract: Language models are increasingly capable and are being rapidly deployed on a population-level scale. As a result, the safety of these models is increa

Foundation models for discovering robust biomarkers of neurological disorders from dynamic functional connectivity

arXiv:2604.22018v1 Announce Type: cross Abstract: Several brain foundation models (FM) have recently been proposed to predict brain disorders by modelling dynamic functional connectivity (FC). While t

Agents

HACHIMI: Scalable and Controllable Student Persona Generation via Orchestrated Agents

arXiv:2603.04855v3 Announce Type: replace Abstract: Student Personas (SPs) are emerging as infrastructure for educational LLMs, yet prior work often relies on ad-hoc prompting or hand-crafted profiles

agentsarxiv-cs-cl

How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

model-releasesarxiv-cs-cl

arXiv:2604.22750v1 Announce Type: new Abstract: The wide adoption of AI agents in complex human workflows is driving rapid growth in LLM token consumption. When agents are deployed on tasks that requi

How Hard is it to Decide if a Fact is Relevant to a Query?

arXiv:2604.22422v1 Announce Type: cross Abstract: We consider the following fundamental problem: given a database D, Boolean conjunctive query (CQ) q, and fact f in D, decide whether f is relevant to

Learning Coverage- and Power-Optimal Transmitter Placement from Building Maps: A Comparative Study of Direct and Indirect Neural Approaches

arXiv:2604.22056v1 Announce Type: new Abstract: Optimal wireless transmitter placement is a central task in radio-network planning, yet exhaustive search becomes prohibitively expensive at scale. This

Railway Artificial Intelligence Learning Benchmark (RAIL-BENCH): A Benchmark Suite for Perception in the Railway Domain

model-releasesarxiv-cs-cv

arXiv:2604.22507v1 Announce Type: new Abstract: Automated train operation on existing railway infrastructure requires robust camera-based perception, yet the railway domain lacks public benchmark suit

Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review

model-releasesarxiv-cs-cv

arXiv:2506.14831v3 Announce Type: replace Abstract: With the emergence of powerful data-driven methods in human trajectory prediction (HTP), gaining a finer understanding of multi-agent interactions l

Applications

Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities

arXiv:2604.22749v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for text generation tasks from everyday use to high-stakes enterprise and government applications, in

applicationsarxiv-cs-cl

System-Mediated Attention Imbalances Make Vision-Language Models Say Yes

arXiv:2601.12430v2 Announce Type: replace Abstract: Vision-language model (VLM) hallucination is commonly linked to imbalanced allocation of attention across input modalities: system, image and text.

safetyarxiv-cs-cl

Adaptive Defense Orchestration for RAG: A Sentinel-Strategist Architecture against Multi-Vector Attacks

arXiv:2604.20932v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) systems are increasingly deployed in sensitive domains such as healthcare and law, where they rely on private, do

AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security

arXiv:2601.18491v2 Announce Type: replace Abstract: The rise of AI agents introduces complex safety and security challenges arising from autonomous tool use and environmental interactions. Current gua

Align Generative Artificial Intelligence with Human Preferences: A Novel Large Language Model Fine-Tuning Method for Online Review Management

arXiv:2604.21209v1 Announce Type: new Abstract: Online reviews have played a pivotal role in consumers' decision-making processes. Existing research has highlighted the significant impact of manageria

Hardware

Association Is Not Similarity: Learning Corpus-Specific Associations for Multi-Hop Retrieval

arXiv:2604.20850v1 Announce Type: cross Abstract: Dense retrieval systems rank passages by embedding similarity to a query, but multi-hop questions require passages that are associatively related thro

hardwarearxiv-cs-ai

Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts

arXiv:2604.21344v1 Announce Type: cross Abstract: Charts are widely used to present complex information. Deriving meaningful insights in real-world contexts often requires interpreting multiple relate

CaST-POI: Candidate-Conditioned Spatiotemporal Modeling for Next POI Recommendation

arXiv:2604.20845v1 Announce Type: cross Abstract: Next Point-of-Interest (POI) recommendation plays a crucial role in location-based services by predicting users' future mobility patterns. Existing me

Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination

arXiv:2506.21546v4 Announce Type: replace-cross Abstract: Segmentation Vision-Language Models (VLMs) have significantly advanced grounded visual understanding, yet they remain prone to pixel-grounding

Applications

Cross-Domain Data Selection and Augmentation for Automatic Compliance Detection

arXiv:2604.21469v1 Announce Type: new Abstract: Automating the detection of regulatory compliance remains a challenging task due to the complexity and variability of legal texts. Models trained on one

applicationsarxiv-cs-cl

Do LLM Decoders Listen Fairly? Benchmarking How Language Model Priors Shape Bias in Speech Recognition

arXiv:2604.21276v1 Announce Type: cross Abstract: As pretrained large language models replace task-specific decoders in speech recognition, a critical question arises: do their text-derived priors mak

Fine-Tuning Regimes Define Distinct Continual Learning Problems

arXiv:2604.21927v1 Announce Type: new Abstract: Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge. Despite substantial progress in benc

Identifying Bias in Machine-generated Text Detection

arXiv:2512.09292v2 Announce Type: replace-cross Abstract: The meteoric rise in text generation capability has been accompanied by parallel growth in interest in machine-generated text detection: the c

Ideological Bias in LLMs' Economic Causal Reasoning

arXiv:2604.21334v1 Announce Type: new Abstract: Do large language models (LLMs) exhibit systematic ideological bias when reasoning about economic causal effects? As LLMs are increasingly used in polic

KompeteAI: Accelerated Autonomous Multi-Agent System for End-to-End Pipeline Generation for Machine Learning Problems

arXiv:2508.10177v3 Announce Type: replace Abstract: Recent Large Language Model (LLM)-based AutoML systems demonstrate impressive capabilities but face significant limitations such as constrained expl

Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery

arXiv:2604.21102v1 Announce Type: cross Abstract: We present a novel framework for automatically evaluating building conditions nationwide in the United States by leveraging large language models (LLM

M-CARE: Standardized Clinical Case Reporting for AI Model Behavioral Disorders, with a 20-Case Atlas and Experimental Validation

arXiv:2604.20871v1 Announce Type: cross Abstract: We introduce M-CARE (Model Clinical Assessment and Reporting for Evaluation), a clinical case report framework for AI model behavioral disorders adapt

Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review

arXiv:2603.18740v2 Announce Type: replace-cross Abstract: Automated Code Review (ACR) systems integrating Large Language Models (LLMs) are increasingly adopted in software development workflows, rangi

Reinforcing privacy reasoning in LLMs via normative simulacra from fiction

arXiv:2604.20904v1 Announce Type: cross Abstract: Information handling practices of LLM agents are broadly misaligned with the contextual privacy expectations of their users. Contextual Integrity (CI)

Replay-buffer engineering for noise-robust quantum circuit optimization

arXiv:2604.21863v1 Announce Type: cross Abstract: Deep reinforcement learning (RL) for quantum circuit optimization faces three fundamental bottlenecks: replay buffers that ignore the reliability of t

Revealing Geography-Driven Signals in Zone-Level Claim Frequency Models: An Empirical Study using Environmental and Visual Predictors

arXiv:2604.21893v1 Announce Type: cross Abstract: Geographic context is often consider relevant to motor insurance risk, yet public actuarial datasets provide limited location identifiers, constrainin

SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning

arXiv:2604.21782v1 Announce Type: new Abstract: We present the shared task on narrative similarity and narrative representation learning - NSNRL (pronounced 'nass-na-rel'). The task operationalizes na

researcharxiv-cs-cl

Agents

The AI Criminal Mastermind

arXiv:2604.20868v1 Announce Type: cross Abstract: In this paper, I evaluate the risks of an AI criminal mastermind, an AI agent capable of planning, coordinating, and committing a crime through the on

agentsarxiv-cs-ai

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

arXiv:2604.21696v1 Announce Type: new Abstract: Tabular foundation models aim to learn universal representations of tabular data that transfer across tasks and domains, enabling applications such as t

Tuning for TraceTarnish: Techniques, Trends, and Testing Tangible Traits

arXiv:2512.03465v3 Announce Type: replace-cross Abstract: In this study, we more rigorously evaluated our attack script extit{TraceTarnish}, which leverages adversarial stylometry principles to anonym

researcharxiv-cs-cl

Validating a Deep Learning Algorithm to Identify Patients with Glaucoma using Systemic Electronic Health Records

arXiv:2604.20921v1 Announce Type: new Abstract: We evaluated whether a glaucoma risk assessment (GRA) model trained on All of Us national data can identify patients at high probability of glaucoma usi

researcharxiv-cs-lg

VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought

arXiv:2604.21396v1 Announce Type: cross Abstract: The advancement of Large Vision-Language Models (LVLMs) requires precise local region-based reasoning that faithfully grounds the model's logic in act

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

model-releasesarxiv-cs-cl

arXiv:2604.21255v1 Announce Type: new Abstract: Model distillation is a primary driver behind the rapid progress of LLM agents, yet it often leads to behavioral homogenization. Many emerging agents sh

Why Do Language Model Agents Whistleblow?

arXiv:2511.17085v3 Announce Type: replace-cross Abstract: The deployment of Large Language Models (LLMs) as tool-using agents causes their alignment training to manifest in new ways. Recent work finds

Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs

arXiv:2604.08016v2 Announce Type: replace Abstract: Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an

Accumulated Aggregated D-Optimal Designs for Estimating Main Effects in Black-Box Models

arXiv:2510.08465v2 Announce Type: replace-cross Abstract: Estimating how individual input variables affect the output of a black-box model is a central task in explainable machine learning. However, e

researcharxiv-cs-lg

Aligning Human-AI-Interaction Trust for Mental Health Support: Survey and Position for Multi-Stakeholders

arXiv:2604.20166v1 Announce Type: new Abstract: Building trustworthy AI systems for mental health support is a shared priority across stakeholders from multiple disciplines. However, 'trustworthy' rem

safetyarxiv-cs-cl

Assessing the Robustness of Climate Foundation Models under No-Analog Distribution Shifts

arXiv:2603.23043v2 Announce Type: replace-cross Abstract: The accelerating pace of climate change introduces profound non-stationarities that challenge the ability of Machine Learning based climate em

Caught in the Web of Words: Do LLMs Fall for Spin in Medical Literature?

arXiv:2502.07963v4 Announce Type: replace-cross Abstract: Medical research faces well-documented challenges in translating novel treatments into clinical practice. Publishing incentives encourage rese

Applications

Cortex 2.0: Grounding World Models in Real-World Industrial Deployment

arXiv:2604.20246v1 Announce Type: cross Abstract: Industrial robotic manipulation demands reliable long-horizon execution across embodiments, tasks, and changing object distributions. While Vision-Lan

applicationsarxiv-cs-ai

Diagnosing CFG Interpretation in LLMs

arXiv:2604.20811v1 Announce Type: new Abstract: As LLMs are increasingly integrated into agentic systems, they must adhere to dynamically defined, machine-interpretable interfaces. We evaluate LLMs as

DistortBench: Benchmarking Vision Language Models on Image Distortion Identification

arXiv:2604.19966v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used in settings where sensitivity to low-level image degradations matters, including content moderatio

Effects of Cross-lingual Evidence in Multilingual Medical Question Answering

arXiv:2604.20531v1 Announce Type: new Abstract: This paper investigates Multilingual Medical Question Answering across high-resource (English, Spanish, French, Italian) and low-resource (Basque, Kazak

researcharxiv-cs-cl