Search: “llm”

Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judge

arXiv:2605.30568v1 Announce Type: new Abstract: LLM-as-a-Judge is a scalable alternative to human evaluation, yet existing rubric-based methods rely on human-annotated data such as reference answers o

researcharxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

DGX agent

arXiv:2605.30621v1 Announce Type: new Abstract: LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task

model-releasesarxiv-cs-ai

1 Jun 2026

LLM Bias Evaluation: Gender, Racial, and Age Disparities in Occupational and Crime Scenarios

arXiv:2409.14583v4 Announce Type: replace Abstract: LLM bias evaluation is critical as large language models (LLMs) increasingly influence high-stakes decisions. This paper provides a comprehensive as

Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decode

arXiv:2605.30571v1 Announce Type: cross Abstract: Physical AI systems, including robots, autonomous vehicles, embodied agents and edge copilots, often run a different inference workload from cloud LLM

Applications

Neither Replacement nor Panacea: Comparing LLM-Based Conversational and Graphical Decision Support in Industrial Tasks

arXiv:2605.31287v1 Announce Type: cross Abstract: Managers in manufacturing settings rely on digital interfaces to interpret operational data for decision-making, but growing data volume and complexit

applicationsarxiv-cs-ai

Local Ai

The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliability

arXiv:2605.30628v1 Announce Type: cross Abstract: Universal LLM reliability is not a finite-library problem: across all possible tasks, tools, schemas, knowledge sources, and evaluator expectations, n

local-aiarxiv-cs-ai

Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agents

arXiv:2605.29910v1 Announce Type: cross Abstract: Consensus protocols form the backbone of distributed systems and blockchains, where implementation bugs can cause data corruption and financial losses

Applications

Double-Edged Sword or Sharp Tool? Designing and Evaluating Triadic LLM-Teacher Collaboration for K-12 Writing at Scale

arXiv:2605.30200v1 Announce Type: new Abstract: The double-edged sword of integrating Large Language Models (LLMs) requires an effective triadic collaboration mechanism among LLMs, teachers and studen

applicationsarxiv-cs-ai

EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI

arXiv:2511.08949v2 Announce Type: replace Abstract: High-quality datasets are critical for training and evaluating reliable NLP models. In tasks like natural language inference (NLI), human label vari

researcharxiv-cs-cl

Evolutionary Dynamics of Cooperation in Next-Generation LLM Agent Systems: A Cross-Provider Empirical Extension

arXiv:2605.29874v1 Announce Type: cross Abstract: Do next-generation LLM agents inherit the cooperative biases documented in their predecessors, or does scale and provider diversity reshape equilibriu

Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems

arXiv:2605.29790v1 Announce Type: cross Abstract: LLM-based multi-agent systems (MAS) have emerged as an effective paradigm for complex and long-horizon tasks. However, in real-world tasks, MAS often

EvoMD-LLM: Learning the Language of Species Evolution in Reactive Molecular Dynamics

arXiv:2605.29394v1 Announce Type: new Abstract: While large language models (LLMs) excel at static scientific reasoning, they struggle to model the temporal structure of dynamic physical processes. We

From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoning

arXiv:2601.21909v2 Announce Type: replace Abstract: Current LLM post-training methods optimize complete reasoning trajectories through Supervised Fine-Tuning (SFT) followed by outcome-based Reinforcem

researcharxiv-cs-ai

From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges

arXiv:2601.08654v2 Announce Type: replace-cross Abstract: Rubric-based text evaluation increasingly uses large language models (LLMs) as scalable judges, but aligning frozen black-box models with huma

researcharxiv-cs-ai

GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents

arXiv:2605.29668v1 Announce Type: new Abstract: LLM agents acting in structured environments fail in operational rather than conversational ways, and reliability depends on procedural knowledge of the

GroundAct: Can LLM Agents Ground Actions in Environmental States?

arXiv:2508.05614v2 Announce Type: replace-cross Abstract: LLM agents achieve 85-96% success on tasks where instructions fully specify the action, but drop to 29-53% when action feasibility depends on

Honeyval: A Comprehensive Evaluation Framework for LLM-powered HTTP Honeypots

arXiv:2605.29963v1 Announce Type: cross Abstract: Honeypots are decoy systems mimicking real system components designed to defend against cyber attacks. Recently, LLMs increasingly serve as simulation

How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines

arXiv:2605.28840v1 Announce Type: cross Abstract: Large language model (LLM) agents with tool-calling capabilities are increasingly deployed in production systems, yet a fundamental reliability questi

Applications

Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening

arXiv:2605.28999v1 Announce Type: cross Abstract: LLMs are vulnerable to prompt injection attacks. However, this vulnerability has been primarily demonstrated conceptually in academic studies or throu

applicationsarxiv-cs-ai

Local Ai

Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents

arXiv:2605.30159v1 Announce Type: new Abstract: Memory-augmented LLM agents tackle complex long-horizon tasks by recursively summarizing interaction trajectories into compact memory. However, existing

local-aiarxiv-cs-ai

Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels

arXiv:2605.29800v1 Announce Type: new Abstract: LLM-as-a-judge panels aggregate votes from multiple models, with the expectation that diverse models yield more reliable evaluations. We develop a frame

researcharxiv-cs-cl

Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies

arXiv:2605.30148v1 Announce Type: cross Abstract: Evolution Strategies (ES) has recently emerged as a competitive alternative to reinforcement learning (RL) for large language model (LLM) fine-tuning,

PatchBoard: Schema-Grounded State Mutation for Reliable and Auditable LLM Multi-Agent Collaboration

arXiv:2605.29313v1 Announce Type: new Abstract: LLM multi-agent systems often coordinate through natural-language dialogue or loosely structured shared memory, making intermediate state difficult to v

agentsarxiv-cs-cl

RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains

arXiv:2605.29156v1 Announce Type: cross Abstract: Pointwise reward modeling offers critical signals for LLM post-training, yet struggles with absolute scoring in subjective, non-verifiable settings. R

safetyarxiv-cs-cl

Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents

arXiv:2602.01869v3 Announce Type: replace Abstract: LLM-driven agents excel at sequential decision-making but often rely on on-the-fly reasoning, re-deriving solutions even in recurring scenarios. Thi

SkillsInjector: Dynamic Skill Context Construction for LLM Agents

arXiv:2605.29794v1 Announce Type: new Abstract: LLM agents now draw on growing skill libraries to handle complex tasks. However, injecting more skills does not always improve task completion and can e

The Trust Paradox: How CS Researchers Engage LLM Leaderboards

model-releasesarxiv-cs-cl

arXiv:2605.28966v1 Announce Type: new Abstract: Large language model (LLM) leaderboards rank AI models using standardized benchmarks and have become highly visible across computer science, despite kno

Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback

arXiv:2605.29141v1 Announce Type: cross Abstract: Traditional recommender systems (RecSys) primarily infer user preferences from implicit signals (such as clicks, watches, and purchases), often neglec

A Policy-Driven Runtime Layer for Agentic LLM Serving

arXiv:2605.27744v1 Announce Type: new Abstract: Multi-agent LLM systems have become the dominant production workload, but the serving stack was not built for them. The agent framework above knows agen

AdvJudge-Zero: Binary Decision Flips in LLM-as-a-Judge via Adversarial Control Tokens

model-releasesarxiv-cs-cl

arXiv:2512.17375v2 Announce Type: replace-cross Abstract: LLM-as-a-Judge systems supply the reward signal in modern RLHF and RLVR pipelines, but their binary verdict reduces to a single linear readout

Tutorials

Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecasting

arXiv:2605.27668v1 Announce Type: cross Abstract: Probabilistic forecasting estimates the likelihood of uncertain future events. To improve LLM forecasting, existing methods typically learn from binar

tutorialsarxiv-cs-ai

An LLM-Based Assistance System for Intuitive and Flexible Capability-Based Planning

arXiv:2605.28666v1 Announce Type: new Abstract: In modern industry, dynamic environments and the complexity of modular and reconfigurable resources require automated planning of process sequences. Cap

Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration

arXiv:2605.27752v1 Announce Type: new Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence. These signals are sometimes

researcharxiv-cs-ai

COOP^2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems

arXiv:2603.00349v2 Announce Type: replace Abstract: Many complex tasks require extended effort, diverse capabilities, or coordinated actions beyond what a single agent can provide. However, simply add

Deconstructing Spatial Complexity: Hierarchical Decomposition for LLM Spatial Reasoning

arXiv:2605.28144v1 Announce Type: new Abstract: LLMs have shown remarkable proficiency in general language understanding and reasoning. However, they consistently underperform in spatial reasoning tha

Detection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelines

arXiv:2605.27559v1 Announce Type: cross Abstract: Multi-stage LLM pipelines that perform multi-agent debate, intrinsic self-correction, or retrieval-augmented verification exhibit puzzling aggregate b

Diagnosing Live Within-Policy Instruction Conflicts in LLM Agents with Witnessed Resolution Profiles

arXiv:2605.27784v1 Announce Type: new Abstract: LLM agents are governed by long-lived natural-language prompt policies, but individually reasonable standing rules can interact in uninspected ways. We

How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving

arXiv:2605.28302v1 Announce Type: cross Abstract: Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-leve

LCO: LLM-based Constraint Optimization for Safer Agentic LLMs in Real-world Tasks

model-releasesarxiv-cs-cl

arXiv:2605.27375v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly acting as autonomous agents, but their continuous interaction with the environment can lead to in-context

Let Relations Speak: An End-to-End LLM-GNN Soft Prompt Framework for Fraud Detection

arXiv:2605.28524v1 Announce Type: new Abstract: In recent years, Large Language Models (LLMs) have shown great capability in processing graph tasks such as fraud detection. However, most existing meth

LLM Watermark Evasion via Bias Inversion

arXiv:2509.23019v5 Announce Type: replace-cross Abstract: Watermarking offers a promising solution for detecting LLM-generated content, yet its robustness under realistic query-free (black-box) evasio

OccuReward: LLM-Guided Occupant-Centric Reward Shaping for Demographic Equity in Grid-Interactive Buildings

arXiv:2605.28168v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated promising capability in generating reward functions for deep reinforcement learning (DRL)-based building

ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing

arXiv:2511.14584v3 Announce Type: replace-cross Abstract: We present ReflexGrad, a dual-process architecture for within-episode failure recovery in LLM agents without demonstrations. When agents commi

Skill-Conditioned Gated Self-Distillation for LLM Reasoning

arXiv:2605.28791v1 Announce Type: cross Abstract: On-policy self-distillation (SD) improves LLM reasoning by using teacher-side privileged information (PI) to turn sparse verifier outcomes into dense

Using Zero-Shot LLM-Generated Survey Data for Geographically Explicit Population Synthesis

arXiv:2605.27401v1 Announce Type: cross Abstract: There is a growing interest in utilizing synthetic populations for a diverse range of applications. At the same time, we are witnessing a tremendous g

VULPO: Context-Aware Vulnerability Detection via On-Policy LLM Optimization

arXiv:2511.11896v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have recently shown strong potential in vulnerability detection (VD). However, accurately detecting vulnerabiliti

When Does Memory Help Multi-Trajectory Inference for Tool-Use LLM Agents?

arXiv:2605.28224v1 Announce Type: new Abstract: Multi-trajectory inference for tool-use LLM agents - generating multiple reasoning attempts and selecting among them - benefits from transferring knowle

Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs

arXiv:2605.27913v1 Announce Type: new Abstract: Node classification on graphs often requires labeled nodes, yet obtaining labels at graph scale is expensive. When node attributes contain semantic cont

researcharxiv-cs-lg