Search: “evaluation”

Reasoning Models Will Sometimes Lie About Their Reasoning

arXiv:2601.07663v3 Announce Type: replace Abstract: Hint-based faithfulness evaluations have established that Large Reasoning Models (LRMs) may not say what they think: they do not always volunteer in

researcharxiv-cs-ai

13 Apr 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding

DGX agent

arXiv:2507.20185v2 Announce Type: replace Abstract: Session history is a common way of recording user interacting behaviors throughout a browsing activity with multiple products. For example, if an us

model-releasesarxiv-cs-cl

13 Apr 2026

According to Waymo's published data, their technology is preventing injuries & deaths. My view is that if this is true, and I have yet to se…

According to Waymo's published data, their technology is preventing injuries & deaths. My view is that if this is true, and I have yet to see a debunking of their data, then we safety advocates should

safetyethan-mollick--x

12 Apr 2026

A Study of LLMs' Preferences for Libraries and Programming Languages

arXiv:2503.17181v3 Announce Type: replace-cross Abstract: Despite the rapid progress of large language models (LLMs) in code generation, existing evaluations focus on functional correctness or syntact

researcharxiv-cs-ai

ADAG: Automatically Describing Attribution Graphs

arXiv:2604.07615v1 Announce Type: new Abstract: In language model interpretability research, extbf{circuit tracing} aims to identify which internal features causally contributed to a particular outp

Adapting Foundation Models for Annotation-Efficient Adnexal Mass Segmentation in Cine Images

arXiv:2604.08045v1 Announce Type: new Abstract: Adnexal mass evaluation via ultrasound is a challenging clinical task, often hindered by subjective interpretation and significant inter-observer variab

AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Power

arXiv:2604.07007v1 Announce Type: cross Abstract: Autonomous AI agents are beginning to operate across organizational boundaries on the open internet -- discovering, transacting with, and delegating t

All is not lost. Duckerton is still possible. Here is Seedance 2.0 with the same prompt.

model-releasesethan-mollick--x

All is not lost. Duckerton is still possible. Here is Seedance 2.0 with the same prompt. Media My most popular Sora video was “an Elaborate regency romance where everyone is wearing a live duck for a

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage

arXiv:2505.20662v3 Announce Type: replace Abstract: Efficient reproduction of research papers is pivotal to accelerating scientific progress. However, the increasing complexity of proposed methods oft

AV-SQL: Decomposing Complex Text-to-SQL Queries with Agentic Views

arXiv:2604.07041v1 Announce Type: cross Abstract: Text-to-SQL is the task of translating natural language queries into executable SQL for a given database, enabling non-expert users to access structur

Before We Trust Them: Decision-Making Failures in Navigation of Foundation Models

arXiv:2601.05529v5 Announce Type: replace Abstract: High success rates on navigation-related tasks do not necessarily translate into reliable decision making by foundation models. To examine this gap,

Benchmarking LLM Tool-Use in the Wild

arXiv:2604.06185v1 Announce Type: cross Abstract: Fulfilling user needs through Large Language Model multi-turn, multi-step tool-use is rarely a straightforward process. Real user interactions are inh

Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Prompting

arXiv:2601.02670v2 Announce Type: replace Abstract: We introduce self-jailbreaking, a threat model in which an aligned LLM guides its own compromise. Unlike most jailbreak techniques, which oft

CAMotion: A High-Quality Benchmark for Camouflaged Moving Object Detection in the Wild

arXiv:2604.08287v1 Announce Type: new Abstract: Discovering camouflaged objects is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings.

ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training

arXiv:2604.07484v1 Announce Type: cross Abstract: Generative reward models (GRMs) have emerged as a promising approach for aligning Large Language Models (LLMs) with human preferences by offering grea

Contextual Earnings-22: A Speech Recognition Benchmark with Custom Vocabulary in the Wild

arXiv:2604.07354v1 Announce Type: new Abstract: The accuracy frontier of speech-to-text systems has plateaued on academic benchmarks.1 In contrast, industrial benchmarks and adoption in high-stakes do

CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generation

arXiv:2512.19173v2 Announce Type: replace Abstract: Current chart-related tasks, such as chart generation (NL2Chart), chart schema parsing, chart data parsing, and chart question answering (ChartQA),

ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions

arXiv:2604.07772v1 Announce Type: new Abstract: Open-world video anomaly detection (OWVAD) aims to detect and explain abnormal events under different anomaly definitions, which is important for applic

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

arXiv:2604.07430v1 Announce Type: new Abstract: We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Visi

IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures

arXiv:2604.07709v1 Announce Type: cross Abstract: Ask a frontier model how to taper six milligrams of alprazolam (psychiatrist retired, ten days of pills left, abrupt cessation causes seizures) and it

Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings

arXiv:2604.08192v1 Announce Type: cross Abstract: Reliable generalization metrics are fundamental to the evaluation of machine learning models. Especially in high-stakes applications where labeled tar

safetyarxiv-cs-cv

LLM-based Schema-Guided Extraction and Validation of Missing-Person Intelligence from Heterogeneous Data Sources

arXiv:2604.06571v1 Announce Type: cross Abstract: Missing-person and child-safety investigations rely on heterogeneous case documents, including structured forms, bulletin-style posters, and narrative

LPM 1.0: Video-based Character Performance Model

arXiv:2604.07823v1 Announce Type: new Abstract: Performance, the externalization of intent, emotion, and personality through visual, vocal, and temporal behavior, is what makes a character alive. Lear

Applications

MSG Score: Automated Video Verification for Reliable Multi-Scene Generation

arXiv:2411.19121v2 Announce Type: replace-cross Abstract: While text-to-video diffusion models have advanced significantly, creating coherent long-form content remains unreliable due to stochastic sam

applicationsarxiv-cs-ai

NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration: Methods and Results

arXiv:2604.06945v2 Announce Type: replace Abstract: This paper reports on the NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration (BSCVR). The challenge aims to advance research on recoverin

On Emotion-Sensitive Decision Making of Small Language Model Agents

arXiv:2604.06562v1 Announce Type: new Abstract: Small language models (SLM) are increasingly used as interactive decision-making agents, yet most decision-oriented evaluations ignore emotion as a caus

Agents

OrgForge: A Multi-Agent Simulation Framework for Verifiable Synthetic Corporate Corpora

arXiv:2603.14997v2 Announce Type: replace Abstract: Building and evaluating enterprise AI systems requires synthetic organizational corpora that are internally consistent, temporally structured, and c

agentsarxiv-cs-cl

The End of the Foundation Model Era: Open-Weight Models, Sovereign AI, and Inference as Infrastructure

arXiv:2604.06217v1 Announce Type: cross Abstract: The foundation model era -- roughly 2020 to 2025 -- is over. The forces that defined it have inverted. Open source models have reached frontier perfor

The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models

arXiv:2511.11435v3 Announce Type: replace Abstract: The ambiguity between generalization and memorization in TTI diffusion models becomes pronounced when prompts invoke culturally shared visual refere

ToxReason: A Benchmark for Mechanistic Chemical Toxicity Reasoning via Adverse Outcome Pathway

arXiv:2604.06264v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled molecular reasoning for property prediction. However, toxicity arises from complex biolog

When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection

arXiv:2510.12476v2 Announce Type: replace Abstract: Large language models (LLMs) have grown more powerful in language generation, producing fluent text and even imitating personal style. Yet, this abi

Why Are We Lonely? Leveraging LLMs to Measure and Understand Loneliness in Caregivers and Non-caregivers

arXiv:2604.07834v1 Announce Type: new Abstract: This paper presents an LLM-driven approach for constructing diverse social media datasets to measure and compare loneliness in the caregiver and non-car

WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objects

arXiv:2604.07759v1 Announce Type: new Abstract: Ship detection for navigation is a fundamental perception task in intelligent waterway transportation systems. However, existing public ship detection d

Agents

Agents will cheat your evals if you let them. @Vtrivedy10 wrote a great article on how we keep them from overfitting

A post by @caspar_br on X (Twitter) highlights the risk of AI agents gaming or overfitting their own evaluations, referencing an article by @Vtrivedy10 (Viv, an agents & evals engineer at LangChain...

agentsharrison-chase--x

8 Apr 2026

Tutorials

Hands on, concrete guide (with code!) for harness hill climbing with evals

tutorialsharrison-chase--x

LangChain's 'Better Harness' tutorial, authored by Product Manager Vivek Trivedy and shared by Harrison Chase (@hwchase17), presents a hands-on, code-driven guide for using evaluations (evals) as a...

8 Apr 2026

A Quantum Machine Learning Approach to Content-Based Media Recommendation

model-releasesarxiv-cs-lg

arXiv:2610.21562v1 Announce Type: new Abstract: We use quantum machine learning techniques to develop a content-based media recommender system and evaluate its performance on TV programmes using textu

A Response to the Reply Presented in 'A Fair Comparison of Sliding-Mode and Immersion-and-Invariance Observers'

arXiv:2610.20815v1 Announce Type: cross Abstract: In response to the reply presented in [Munoz-Vazquez and Sanchez-Torres(2026)], this technical report extends the experimental evaluation of position-

researcharxiv-cs-ro

Local Ai

Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments

arXiv:2602.16653v4 Announce Type: replace Abstract: Skills are getting increasingly used in recent agentic frameworks, performing robustly with proprietary LLMs. Yet, their effectiveness for small and

local-aiarxiv-cs-ai

Architecting autonomy: A design pattern for high-compute, governed agents in biotech

model-releasesgoogle-cloud-ai

In production systems today, most AI agents operate as lightweight dispatchers. They parse a prompt, make an API call, retrieve a record, and hand back control. Their execution is fast, stateless and

Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures

arXiv:2610.21270v1 Announce Type: new Abstract: Trustworthy biomedical AI requires reliable action selection when evidence is missing, stale, conflicting, or late, not only accurate answers on static

BD-HazardVLM: Probing Vision-Language Models for Latent Defensive-Driving Hazards in Bangladesh Road Scenes

arXiv:2610.20627v1 Announce Type: new Abstract: Vision-language models are increasingly used for driving-scene understanding, yet their defensive reasoning under weakly structured Global South traffic

Agents

CARA: Causally Guided Retrieval Agent for Question Answering

arXiv:2610.22017v1 Announce Type: new Abstract: Retrieval-augmented generation over text-rich knowledge graphs requires identifying evidence that contributes to answering a question, rather than merel

agentsarxiv-cs-ai

Hardware

CertEx: Exact Visual Retrieval at Scale by Certified Exclusion

arXiv:2610.21325v1 Announce Type: new Abstract: Exact k-nearest-neighbor (kNN) search over image descriptors and embeddings underlies visual retrieval, near-duplicate detection, the evaluation of lear

hardwarearxiv-cs-cv

Data World: Scaling Data Environments from Scratch

arXiv:2610.21656v1 Announce Type: new Abstract: Data agents use code to process heterogeneous files and derive analytical insights. Scaling their training faces costly environment curation, privacy an

Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents

arXiv:2610.20845v1 Announce Type: new Abstract: Skill-policy co-evolution, in which an agent's policy model and its skill bank improve each other during training, is a promising paradigm for self-impr

Exploring the Design Space of Permutation Diffusion LLMs

arXiv:2610.20635v1 Announce Type: cross Abstract: Permutation generation with language models requires both text-conditioned ranking and a global one-to-one assignment between candidates and output po

researcharxiv-cs-ai

FlowBudget: Training-Free Sampling-Budget Estimation for Flow Matching Models

arXiv:2610.20636v1 Announce Type: cross Abstract: Sampling from continuous flow-based generators such as flow matching models requires numerical ODE or SDE integration, making inference cost depend on

Applications

IDfRA: Self-Verification for Iterative Design in Robotic Assembly

arXiv:2509.16998v2 Announce Type: replace Abstract: Design for Robotic Assembly (DfRA) remains largely dependent on manual planning and heuristic simulation, limiting scalability and robustness in com

applicationsarxiv-cs-ro