Search: “evaluation”

Ideological Bias in LLMs' Economic Causal Reasoning

arXiv:2604.21334v1 Announce Type: new Abstract: Do large language models (LLMs) exhibit systematic ideological bias when reasoning about economic causal effects? As LLMs are increasingly used in polic

Industry

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Introducing our fourth annual AI Festival jurors. Submissions close Monday, April 27. Enter your project today at the link below.

DGX agent

Runway ML announced the fourth annual AI Festival and introduced its panel of jurors for evaluating submissions. The submission deadline was set for Monday, April 27, with creators encouraged to enter

industrycristobal-valenzuela--x

24 Apr 2026

KompeteAI: Accelerated Autonomous Multi-Agent System for End-to-End Pipeline Generation for Machine Learning Problems

arXiv:2508.10177v3 Announce Type: replace Abstract: Recent Large Language Model (LLM)-based AutoML systems demonstrate impressive capabilities but face significant limitations such as constrained expl

Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery

arXiv:2604.21102v1 Announce Type: cross Abstract: We present a novel framework for automatically evaluating building conditions nationwide in the United States by leveraging large language models (LLM

M-CARE: Standardized Clinical Case Reporting for AI Model Behavioral Disorders, with a 20-Case Atlas and Experimental Validation

arXiv:2604.20871v1 Announce Type: cross Abstract: We introduce M-CARE (Model Clinical Assessment and Reporting for Evaluation), a clinical case report framework for AI model behavioral disorders adapt

Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review

arXiv:2603.18740v2 Announce Type: replace-cross Abstract: Automated Code Review (ACR) systems integrating Large Language Models (LLMs) are increasingly adopted in software development workflows, rangi

Tools

More on CursorBench: http://cursor.com/blog/cursorbench

CursorBench is a benchmarking tool or evaluation framework developed by Cursor, likely designed to measure and compare the performance of AI code completion or development features. The post provides

toolscursor--x

Reinforcing privacy reasoning in LLMs via normative simulacra from fiction

arXiv:2604.20904v1 Announce Type: cross Abstract: Information handling practices of LLM agents are broadly misaligned with the contextual privacy expectations of their users. Contextual Integrity (CI)

Replay-buffer engineering for noise-robust quantum circuit optimization

arXiv:2604.21863v1 Announce Type: cross Abstract: Deep reinforcement learning (RL) for quantum circuit optimization faces three fundamental bottlenecks: replay buffers that ignore the reliability of t

Revealing Geography-Driven Signals in Zone-Level Claim Frequency Models: An Empirical Study using Environmental and Visual Predictors

model-releasesarxiv-cs-lg

arXiv:2604.21893v1 Announce Type: cross Abstract: Geographic context is often consider relevant to motor insurance risk, yet public actuarial datasets provide limited location identifiers, constrainin

SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning

arXiv:2604.21782v1 Announce Type: new Abstract: We present the shared task on narrative similarity and narrative representation learning - NSNRL (pronounced 'nass-na-rel'). The task operationalizes na

researcharxiv-cs-cl

Agents

The AI Criminal Mastermind

arXiv:2604.20868v1 Announce Type: cross Abstract: In this paper, I evaluate the risks of an AI criminal mastermind, an AI agent capable of planning, coordinating, and committing a crime through the on

agentsarxiv-cs-ai

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

model-releasesarxiv-cs-lg

arXiv:2604.21696v1 Announce Type: new Abstract: Tabular foundation models aim to learn universal representations of tabular data that transfer across tasks and domains, enabling applications such as t

Tuning for TraceTarnish: Techniques, Trends, and Testing Tangible Traits

arXiv:2512.03465v3 Announce Type: replace-cross Abstract: In this study, we more rigorously evaluated our attack script extit{TraceTarnish}, which leverages adversarial stylometry principles to anonym

researcharxiv-cs-cl

Validating a Deep Learning Algorithm to Identify Patients with Glaucoma using Systemic Electronic Health Records

arXiv:2604.20921v1 Announce Type: new Abstract: We evaluated whether a glaucoma risk assessment (GRA) model trained on All of Us national data can identify patients at high probability of glaucoma usi

researcharxiv-cs-lg

VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought

arXiv:2604.21396v1 Announce Type: cross Abstract: The advancement of Large Vision-Language Models (LVLMs) requires precise local region-based reasoning that faithfully grounds the model's logic in act

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

model-releasesarxiv-cs-cl

arXiv:2604.21255v1 Announce Type: new Abstract: Model distillation is a primary driver behind the rapid progress of LLM agents, yet it often leads to behavioral homogenization. Many emerging agents sh

Why Do Language Model Agents Whistleblow?

arXiv:2511.17085v3 Announce Type: replace-cross Abstract: The deployment of Large Language Models (LLMs) as tool-using agents causes their alignment training to manifest in new ways. Recent work finds

Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs

arXiv:2604.08016v2 Announce Type: replace Abstract: Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an

Industry

3,000 people are testing the ml-intern!

Hugging Face's ml-intern project has reached a testing milestone with 3,000 people actively participating in its evaluation. This suggests significant community interest in what is likely an open-sour

industryclem-delangue--x

Accumulated Aggregated D-Optimal Designs for Estimating Main Effects in Black-Box Models

arXiv:2510.08465v2 Announce Type: replace-cross Abstract: Estimating how individual input variables affect the output of a black-box model is a central task in explainable machine learning. However, e

researcharxiv-cs-lg

Aligning Human-AI-Interaction Trust for Mental Health Support: Survey and Position for Multi-Stakeholders

arXiv:2604.20166v1 Announce Type: new Abstract: Building trustworthy AI systems for mental health support is a shared priority across stakeholders from multiple disciplines. However, 'trustworthy' rem

safetyarxiv-cs-cl

Assessing the Robustness of Climate Foundation Models under No-Analog Distribution Shifts

arXiv:2603.23043v2 Announce Type: replace-cross Abstract: The accelerating pace of climate change introduces profound non-stationarities that challenge the ability of Machine Learning based climate em

Caught in the Web of Words: Do LLMs Fall for Spin in Medical Literature?

arXiv:2502.07963v4 Announce Type: replace-cross Abstract: Medical research faces well-documented challenges in translating novel treatments into clinical practice. Publishing incentives encourage rese

Applications

Cortex 2.0: Grounding World Models in Real-World Industrial Deployment

arXiv:2604.20246v1 Announce Type: cross Abstract: Industrial robotic manipulation demands reliable long-horizon execution across embodiments, tasks, and changing object distributions. While Vision-Lan

applicationsarxiv-cs-ai

Diagnosing CFG Interpretation in LLMs

arXiv:2604.20811v1 Announce Type: new Abstract: As LLMs are increasingly integrated into agentic systems, they must adhere to dynamically defined, machine-interpretable interfaces. We evaluate LLMs as

DistortBench: Benchmarking Vision Language Models on Image Distortion Identification

arXiv:2604.19966v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used in settings where sensitivity to low-level image degradations matters, including content moderatio

Effects of Cross-lingual Evidence in Multilingual Medical Question Answering

arXiv:2604.20531v1 Announce Type: new Abstract: This paper investigates Multilingual Medical Question Answering across high-resource (English, Spanish, French, Italian) and low-resource (Basque, Kazak

researcharxiv-cs-cl

Epistemic Constitutionalism Or: how to avoid coherence bias

arXiv:2601.14295v3 Announce Type: replace Abstract: Large language models increasingly function as artificial reasoners: they evaluate arguments, assign credibility, and express confidence. Yet their

Evian: Towards Explainable Visual Instruction-tuning Data Auditing

arXiv:2604.20544v1 Announce Type: cross Abstract: The efficacy of Large Vision-Language Models (LVLMs) is critically dependent on the quality of their training data, requiring a precise balance betwee

Exploiting LLM-as-a-Judge Disposition on Free Text Legal QA via Prompt Optimization

arXiv:2604.20726v1 Announce Type: cross Abstract: This work explores the role of prompt design and judge selection in LLM-as-a-Judge evaluations of free text legal question answering. We examine wheth

Fairness Testing of Large Language Models in Role-Playing

arXiv:2411.00585v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A cr

Generalization and Membership Inference Attack a Practical Perspective

arXiv:2604.19936v1 Announce Type: cross Abstract: With the emergence of new evaluation metrics and attack methodologies for Membership Inference Attacks (MIA), it becomes essential to reevaluate previ

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

arXiv:2604.20382v1 Announce Type: new Abstract: Rising demand for mental health support has increased interest in using Large Language Models (LLMs) for counseling. However, adapting LLMs to this high

safetyarxiv-cs-cl

Applications

Human-like Content Analysis for Generative AI with Language-Grounded Sparse Encoders

arXiv:2508.18236v4 Announce Type: replace Abstract: The rapid development of generative AI has transformed content creation, communication, and human development. However, this technology raises profo

applicationsarxiv-cs-cv

Agents

If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems

arXiv:2604.19844v1 Announce Type: cross Abstract: Recent advances in embodied Vision-Language Agentic Systems (VLAS), powered by large vision-language models (LVLMs), enable AI systems to perceive and

agentsarxiv-cs-ai

Inference Headroom Ratio: A Diagnostic and Control Framework for Inference Stability Under Constraint

arXiv:2604.19760v1 Announce Type: new Abstract: We present a simulation-based evaluation of the Inference Headroom Ratio (IHR), a dimensionless diagnostic quantity for characterizing inference stabili

Applications

Interpretability from the Ground Up: Stakeholder-Centric Design of Automated Scoring in Educational Assessments

arXiv:2511.17069v3 Announce Type: replace Abstract: AI-driven automated scoring systems offer scalable and efficient means of evaluating complex student-generated responses. Yet, despite increasing de

applicationsarxiv-cs-cl

Intersectional Fairness in Large Language Models

model-releasesarxiv-cs-cl

arXiv:2604.20677v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in socially sensitive settings, raising concerns about fairness and biases, particularly across i

KD-Judge: A Knowledge-Driven Automated Judge Framework for Functional Fitness Movements on Edge Devices

arXiv:2604.19834v1 Announce Type: new Abstract: Functional fitness movements are widely used in training, competition, and health-oriented exercise programs, yet consistently enforcing repetition (rep

researcharxiv-cs-cv

Kimi K2.6 becomes the #1 open model on MathArena!

model-releaseskimi-moonshot--x

Kimi K2.6 achieved the top ranking on MathArena, a benchmark for evaluating mathematical problem-solving capabilities in open-source language models. This announcement highlights the model's superior

Lever: Inference-Time Policy Reuse under Support Constraints

arXiv:2604.20174v1 Announce Type: new Abstract: Reinforcement learning (RL) policies are typically trained for fixed objectives, making reuse difficult when task requirements change. We study inferenc

safetyarxiv-cs-lg

MedSkillAudit: A Domain-Specific Audit Framework for Medical Research Agent Skills

arXiv:2604.20441v1 Announce Type: new Abstract: Background: Agent skills are increasingly deployed as modular, reusable capability units in AI agent systems. Medical research agent skills require safe

Relative Entropy Estimation in Function Space: Theory and Applications to Trajectory Inference

model-releasesarxiv-cs-lg

arXiv:2604.20775v1 Announce Type: new Abstract: Trajectory Inference (TI) seeks to recover latent dynamical processes from snapshot data, where only independent samples from time-indexed marginals are

Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem

arXiv:2604.20805v1 Announce Type: cross Abstract: The value alignment problem for artificial intelligence (AI) is often framed as a purely technical or normative challenge, sometimes focused on hypoth

RExBench: Can coding agents autonomously implement AI research extensions?

model-releasesarxiv-cs-cl

arXiv:2506.22598v3 Announce Type: replace Abstract: Agents based on Large Language Models (LLMs) have shown promise for performing sophisticated software engineering tasks autonomously. In addition, t

Robustness of Spatio-temporal Graph Neural Networks for Fault Location in Partially Observable Distribution Grids

arXiv:2604.20403v1 Announce Type: new Abstract: Fault location in distribution grids is critical for reliability and minimizing outage durations. Yet, it remains challenging due to partial observabili

researcharxiv-cs-lg

SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark

model-releasesarxiv-cs-cv

arXiv:2604.20319v1 Announce Type: new Abstract: Fine-grained spatiotemporal reasoning on surgical videos is critical, yet the capabilities of Multi-modal Large Language Models (MLLMs) in this domain r