Search: “claude”

CalBrief: A Pilot Diagnostic Benchmark for Evidence-Calibrated Scientific Briefing with Large Language Models

arXiv:2606.27383v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as research assistants, yet it remains unclear whether they can calibrate research takeaways to the

29 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context

DGX agent

arXiv:2601.17642v2 Announce Type: replace Abstract: Safety alignment in Large Language Models is critical for healthcare; however, reliance on binary refusal boundaries often results in over-refusal o

model-releasesarxiv-cs-ai

29 Jun 2026

LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation

model-releasesarxiv-cs-ro

arXiv:2606.27871v1 Announce Type: new Abstract: Vision Language Models (VLMs) have emerged in the robotic domain as a powerful tool that enables environmental perception with language context, serving

29 Jun 2026

NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning

arXiv:2606.27826v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly deployed as embodied planners in egocentric environments, where task success requires not only

29 Jun 2026

Empirical Software Engineering TerraProbe: A Layered-Oracle Framework for Detecting Deceptive Fixes in LLM-Assisted Terraform

arXiv:2606.26590v1 Announce Type: new Abstract: Security misconfigurations in Terraform Infrastructure-as-Code are a growing risk in cloud deployments, and large language models are increasingly used

Instruction Bleed: Cross-Module Interference in Prompt-Composed Agentic Systems

arXiv:2606.26356v1 Announce Type: new Abstract: Practitioners of prompt-composed agentic systems report a recurring failure mode: editing one prompt module silently shifts the behavior of others despi

Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations

arXiv:2606.26185v1 Announce Type: new Abstract: LLM-as-judge ('grader') components are now standard in evaluation harnesses, including safety evaluations where a pass/fail verdict may gate downstream

RSPC: A Benchmark for Modeling Stress and Psychiatric Conditions in Digitally Mediated Relationships using Psychiatrist Annotations

arXiv:2606.27247v1 Announce Type: new Abstract: In NLP, mental health conditions are often modeled as isolated phenomena, without interpersonal context. We use Reddit posts about long-distance relatio

Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation

model-releasesarxiv-cs-cl

arXiv:2606.25782v1 Announce Type: new Abstract: With the widespread adoption of large language models (LLMs) in chatbots and everyday applications, companies increasingly need guardrails that are effe

25 Jun 2026

InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy

arXiv:2606.25984v1 Announce Type: cross Abstract: Large language models are increasingly deployed as investment research assistants, yet no benchmark tests whether they can accurately reconstruct and

25 Jun 2026

MINIF2F-DAFNY: LLM-Guided Mathematical Theorem Proving via Auto-Active Verification

arXiv:2512.10187v3 Announce Type: replace Abstract: LLMs excel at reasoning, but validating their steps remains challenging. Formal verification offers a solution through mechanically checkable proofs

25 Jun 2026

AD-Bench: A Real-World, Trajectory-Aware Advertising Analytics Benchmark for LLM Agents

arXiv:2602.14257v2 Announce Type: replace-cross Abstract: While Large Language Model (LLM) agents have made remarkable progress on complex reasoning, evaluating them in real-world environments remains

AI Agents Can Already Autonomously Perform Experimental High Energy Physics

arXiv:2603.20179v3 Announce Type: replace-cross Abstract: Large language model-based AI agents are now able to autonomously execute substantial portions of a high energy physics (HEP) analysis pipelin

BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems

arXiv:2606.20668v1 Announce Type: cross Abstract: LLM supervision systems, namely input/output moderation filters and jailbreak detectors, are the primary safeguard against misuse in deployed AI appli

Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

arXiv:2606.15127v2 Announce Type: replace Abstract: Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students inte

Catching Lies Without Sending the Video: Privacy-Preserving Multimodal Deception Detection

model-releasesarxiv-cs-cv

arXiv:2606.22699v1 Announce Type: new Abstract: Frontier multimodal models can guess whether a person is lying from a testimony video. To do so, they stream that raw face and voice to a third-party mo

Fursee: Hybrid YOLO-DINOv3 Framework for Fursuit Identity Retrieval and Clustering

model-releasesarxiv-cs-cv

arXiv:2606.22872v1 Announce Type: new Abstract: Global furry conventions produce massive fursuit photographs, while manual sorting brings heavy labor costs and calls for automatic identity retrieval a

Rebuttals Move Peer-Review Scores, but Initial-Review Structure Bounds the Movement

arXiv:2606.22166v1 Announce Type: cross Abstract: Author rebuttals are the main post-submission window in peer review, but their effect on reviewer scores remains hard to measure because score updates

Reinforcement learning to improve large language model-based automated code compliance systems

arXiv:2606.22402v1 Announce Type: cross Abstract: Large language model (LLM)-based approaches for automated code compliance (ACC) of building regulations are prone to generating incorrect and hallucin

RLM-Cascade: Response-Level Speculative Decoding for Cost-Efficient LLM API Serving

arXiv:2606.22840v1 Announce Type: new Abstract: We present RLM-Cascade, a proxy-layer system that applies speculative decoding at the response level to reduce LLM API costs without requiring model arc

Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction

arXiv:2606.22606v1 Announce Type: cross Abstract: Large language models (LLMs) achieve strong relation extraction (RE), but their computational demands and reliance on proprietary APIs limit deploymen

When Robots Rate Their Own Interactions: Engagement Validity and the Strangeness Failure

model-releasesarxiv-cs-ro

arXiv:2606.23339v1 Announce Type: new Abstract: Human-robot interaction (HRI) evaluation relies almost exclusively on human-completed questionnaires, leaving the robot's perspective unexamined. We pro

Forecasting Future Behavior as a Learning Task

arXiv:2606.11445v1 Announce Type: new Abstract: Trust in an AI system is often anchored by explanations of how it works, which one then uses to forecast its behavior on new inputs. For large reasoning

11 Jun 2026

NightFeats @ MMU-RAGent NeurIPS 2025: A Context-Optimized Multi-Agent RAG System for the Text-to-Text Track

arXiv:2606.11199v1 Announce Type: cross Abstract: We present NightFeats, a structured multi-agent retrieval-augmented generation (RAG) system submitted to the MMU-RAGent competition at NeurIPS 2025, w

11 Jun 2026

Toward Generalist Autonomous Research via Hypothesis-Tree Refinement

arXiv:2606.11926v1 Announce Type: cross Abstract: Scientific progress depends on a repeated loop of exploration, experimentation, and abstraction. Researchers test candidate directions, interpret the

11 Jun 2026

au-Rec: A Verifiable Benchmark for Agentic Recommender Systems

arXiv:2606.10156v1 Announce Type: cross Abstract: As recommender systems transition toward agentic, multi-turn conversational interfaces, evaluation paradigms have struggled to keep pace. Current benc

Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis

arXiv:2606.09854v1 Announce Type: cross Abstract: Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation bias: models tend to protect pee

CodeAlchemy: Synthetic Code Rewriting at Scale

model-releasesarxiv-cs-cl

arXiv:2606.10087v1 Announce Type: new Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats. While synthetic data has proven transformative f

CollabSkill: Evaluating Human-Agent Collaboration On Real-World Tasks

arXiv:2606.09833v1 Announce Type: cross Abstract: AI agents are reshaping the workspace, leading to drastic change of how humans work. Despite the considerable potential of human-agent collaboration b

Frontier Coding Agents Use Metaprogramming to Adapt to Unfamiliar Programming Languages

arXiv:2606.10933v1 Announce Type: new Abstract: LLM-based coding agents are usually evaluated in familiar software settings: mainstream languages, common libraries, and public repositories. These benc

How can we assess human-agent interactions? Case studies in software agent design

arXiv:2510.09801v3 Announce Type: replace Abstract: While benchmarks measure the accuracy of LLM-powered agents, they mostly assume full automation, failing to represent the collaborative nature of re

Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents

arXiv:2606.10209v1 Announce Type: new Abstract: Large language models deployed as autonomous agents for enterprise workflows face a key challenge: verbose tool responses from enterprise systems can ca

Quality Is Not a Safety Proxy Under Quantization

arXiv:2606.10154v1 Announce Type: new Abstract: Quantized checkpoints are often screened first with quality metrics and only later, if at all, with direct safety tests. This paper audits that shortcut

Who Brought Easter Eggs to Eid? Auditing Cultural Translation of Math Word Problems Across Diverse Languages and Regions

model-releasesarxiv-cs-cl

arXiv:2606.11009v1 Announce Type: new Abstract: Large language models are increasingly used to adapt math word problems for personalized learning at scale, but it remains an open question whether thos

Closing the Sim-to-Real Gap: An Evaluation Framework for Autonomous Cyber Defense Configuration of Commercial EDR

arXiv:2606.08168v1 Announce Type: cross Abstract: Leading commercial endpoint detection and response (EDR) products have shifted from operator-configured rule sets to multi-component systems where aut

Emergence World: A Platform for Evaluating Long-Horizon Multi-Agent Autonomy

arXiv:2606.08367v1 Announce Type: cross Abstract: Most evaluations of LLM agents look like exams: a discrete task, a clean environment, a score in minutes or hours. We argue that this approach is mism

From `May' to `Is': Certainty Distortion in Language Model Rewriting

arXiv:2606.07951v1 Announce Type: cross Abstract: Humans increasingly turn to Language Models (LMs) in ways that shape beliefs and drive decisions, including discussing, rewriting, and summarizing inf

FunctionEvolve: Structure-Guided Symbolic Regression with LLMs

arXiv:2606.07704v1 Announce Type: cross Abstract: Symbolic regression aims to uncover explicit scientific laws from data. Recent methods use LLMs to guide mutation from background text, which is more

Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops

arXiv:2606.08960v1 Announce Type: cross Abstract: Agent benchmarks score submissions with outcome verifiers that are typically hand-written and brittle, leaving them open to reward hacking. We audit 1

Implementing Grassroots Logic Programs with Multiagent Transition Systems and AI (Full Version)

arXiv:2602.06934v4 Announce Type: replace-cross Abstract: Grassroots Logic Programs (GLP) is a concurrent logic programming language in which logic variables are partitioned into paired readers and wr

MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance

arXiv:2605.22664v2 Announce Type: replace Abstract: LLM agents are increasingly expected to carry out end-to-end workflows, producing complete artifacts from high-level user instructions. To meet ente

NutriMLLM: Multimodal Large Language Models for Dietary Micronutrient Analysis

arXiv:2606.08948v1 Announce Type: cross Abstract: Comprehensive estimation of dietary micronutrients from food images could improve clinical nutrition care, but training such models requires large mul

PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

arXiv:2510.17947v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are improving at an exceptional rate. With the advent of agentic workflows, multi-turn dialogue has become the de

Rosetta Memory: Adaptive Memory for Cross-LLM Agents

arXiv:2606.07711v1 Announce Type: cross Abstract: Memory is the key component for transforming a stateless LLM into a persistent, evolving agent through experience accumulation, long-horizon planning,

Scaffold Effects on GAIA: A Controlled Comparison

arXiv:2606.08529v1 Announce Type: new Abstract: Published agent capability scores conflate what a model can do with what its scaffold lets it do, and the magnitude of this elicitation gap is not well

Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories

arXiv:2606.07889v1 Announce Type: cross Abstract: LLM-based coding agents sometimes acknowledge a problem in their own reasoning and then proceed anyway. We call this pattern strained coherence: a saf

The AI Epistemic Deference Index: A Continuous Measure of Sycophancy

arXiv:2606.07897v1 Announce Type: new Abstract: Current AI models frequently exhibit epistemic sycophancy, endorsing claims to agree with a user. Existing evaluations typically measure this either by

Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

arXiv:2606.07462v1 Announce Type: new Abstract: As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-