Search: “llm”

Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops

arXiv:2606.08960v1 Announce Type: cross Abstract: Agent benchmarks score submissions with outcome verifiers that are typically hand-written and brittle, leaving them open to reward hacking. We audit 1

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Harmonia: End-to-End RAG Serving Optimization

DGX agent

arXiv:2505.07833v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) improves the reliability of large language models by integrating external knowledge, but serving RAG pipe

researcharxiv-cs-ai

9 Jun 2026

Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation

arXiv:2606.07936v1 Announce Type: cross Abstract: Human evaluation plays a critical role in assessing the quality of generated text. However, the reliability and reproducibility of these evaluations d

Introducing Gemma 4 12B: a unified, encoder-free multimodal model

model-releasesgoogle-deepmind

Gemma 4 12B is a unified, encoder-free multimodal model designed to bring high-performance intelligence to laptops and released under an Apache 2.0 license. It eliminates separate encoders by projecti

Tutorials

June 9th Researcher Reciprocity License 'if you train on it, you let us generate - reverse terms of use void' Status quo 1. We teach frontie…

tutorialsjeremy-howard--x

June 9th Researcher Reciprocity License 'if you train on it, you let us generate - reverse terms of use void' Status quo 1. We teach frontier devs with ICLR/NeurIPS papers, OSS Github contributions 2.

Language-based Trial and Error Falls Behind in the Era of Experience

arXiv:2601.21754v3 Announce Type: replace Abstract: While Large Language Models (LLMs) excel in language-based agentic tasks, their applicability to unseen, nonlinguistic environments (e.g., symbolic

LCAM: A Framework for Diagnosing Interactional Alignment Failures in Con-versational AI

arXiv:2606.08131v1 Announce Type: cross Abstract: Conversational AI is increasingly used for advice, interpretation, reassurance, and decision support in contexts where users may be vulnerable, uncert

MatMind: A Structure-Activity Knowledge-Driven Generative Foundation Model for Materials Science

arXiv:2606.07712v1 Announce Type: cross Abstract: Progress in AI-driven crystal materials science has so far been carried by narrow architectures purpose-built for individual tasks -- graph neural net

MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science

arXiv:2510.12171v2 Announce Type: replace Abstract: Large Language Models have shown strong scientific reasoning ability, but their performance on materials science problems remains less studied. To f

MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Models

arXiv:2606.07996v1 Announce Type: cross Abstract: Pretraining is fundamental to the development of Large Language Models (LLMs), yet the opacity of pretraining data complicates model analysis and rais

Observability for Delegated Execution in Agentic AI Systems

arXiv:2606.09692v1 Announce Type: cross Abstract: Delegation-scoped execution is not identifiable from standard observables: audit logs and execution traces can be identical under multiple incompatibl

OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics

arXiv:2606.09826v1 Announce Type: cross Abstract: Vision-language model (VLM) agents are increasingly deployed in interactive game environments. Yet game benchmarks for VLM agents typically report a s

Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models

arXiv:2602.16061v2 Announce Type: replace-cross Abstract: Estimating population quantities such as mean outcomes from user feedback is fundamental to platform evaluation and social science, yet feedba

researcharxiv-cs-lg

Local Ai

PhysAgent: Automating Physics-Based 4D Synthesis via Trajectory-Grounded Multi-Agent Feedback

arXiv:2606.08688v1 Announce Type: cross Abstract: Achieving fully automated, physically plausible 3D motion synthesis is a core objective in graphics and generative AI. However, configuring complex en

local-aiarxiv-cs-cv

PIPE-Cypher: Automatic Enterprise Benchmark Generation for Text-to-Cypher Systems

arXiv:2606.08481v1 Announce Type: cross Abstract: Enterprise property graphs vary widely in schema structure, internal terminology, domain assumptions, governance constraints, and user interaction pat

PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

arXiv:2510.17947v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are improving at an exceptional rate. With the advent of agentic workflows, multi-turn dialogue has become the de

Tutorials

Post-training is (Massive) Supervised Learning

arXiv:2606.07527v1 Announce Type: cross Abstract: The prevailing paradigm for training LLMs has evolved to rely on a massive post-training phase consisting of SFT and RL. In this position paper, we ar

tutorialsarxiv-cs-ai

RadOT-Eval: Auditable Structured-Evidence Transport for Radiology Report Evaluation

arXiv:2606.08769v1 Announce Type: cross Abstract: Automatic evaluation is critical for high-stakes text generation, where errors often involve omitted findings, hallucinated content, polarity reversal

RAILS: Verification-Native Clearing For Agentic Commerce

arXiv:2606.08790v1 Announce Type: new Abstract: Autonomous agents negotiate, purchase, deploy code, and move funds, but no neutral mechanism determines whether they met their delegated obligation, who

Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation

arXiv:2606.08011v1 Announce Type: cross Abstract: Although directly prompting off-the-shelf Large Language Models (LLMs) to generate meaning-preserving source rewrites can effectively enhance Machine

SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Models

arXiv:2606.07705v1 Announce Type: cross Abstract: Although multi-objective reinforcement learning (MORL) is central to aligning large language models with complex human preferences, the prevailing pra

Setting a custom price for a model in AgentsView

model-releasessimon-willison

TIL: Setting a custom price for a model in AgentsView I've been really enjoying AgentsView by Wes McKinney as a tool for exploring my token usage across different coding agents running on my laptop. C

Structuring agentic AI for HPC code modernization

arXiv:2606.08710v1 Announce Type: cross Abstract: Modernization of legacy scientific codes is often necessary to keep up with the ever-evolving changes in the compute resource ecosystem. Parallelizati

Industry

this is the biggest wake-up call to protect and nourish open source AI if you don't build out sovereign and independent models+infra closed …

this is the biggest wake-up call to protect and nourish open source AI if you don't build out sovereign and independent models+infra closed labs will patronize you to an insulting degree mythos will b

industryclem-delangue--x

TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles

arXiv:2606.07520v1 Announce Type: cross Abstract: Instruction Following (IF) is a core capability of LLMs, requiring strict adherence to diverse constraints, ranging from verifiable ones (e.g., output

safetyarxiv-cs-lg

Towards Long-Horizon Vessel Trajectory and Destination Forecasting with Reasoning Large Language Models

arXiv:2606.08633v1 Announce Type: new Abstract: Long-horizon maritime trajectory prediction is important for shipping management, logistics planning, and maritime risk analysis, yet month-level foreca

TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation

arXiv:2606.07688v1 Announce Type: cross Abstract: Generative recommendation formulates next-item prediction as autoregressive generation over semantic ID (SID) sequences derived from users' historical

UnWeaving the knots of GraphRAG -- turns out VectorRAG is almost enough

arXiv:2603.29875v3 Announce Type: replace-cross Abstract: One of the key problems in Retrieval-augmented generation (RAG) systems is that chunk-based retrieval pipelines represent the source chunks as

Tutorials

Vision-Language Guided Hyperspectral Object Tracking via Semantics Fusion and Contextual Template Updating

arXiv:2606.09167v1 Announce Type: new Abstract: Hyperspectral object tracking (HOT) leverages the rich spectral information provided by hyperspectral videos (HSVs), offering substantial potential for

tutorialsarxiv-cs-cv

what constitutes reasoning in AI is a critical debate. i hope that @dwarkesh_sp will respond.

what constitutes reasoning in AI is a critical debate. i hope that @dwarkesh_sp will respond. what does this even mean, @dwarkesh_sp, “the real deal”? is it even a falsifiable conjecture? what’s the e

safetygary-marcus--x

Applications

Zero and Few Shot Load Forecasting with Large Language Models

arXiv:2411.11350v2 Announce Type: replace Abstract: Deep learning models have shown strong performance in load forecasting, but they generally require large amounts of data for model training before b

applicationsarxiv-cs-lg

ZIPP:Zero-shot Image Personalization from Personas

arXiv:2606.08841v1 Announce Type: new Abstract: Text-to-image diffusion models are increasingly deployed in open-ended creative contexts, yet their outputs remain impersonal, optimized for aggregate a

AdMem: Advanced Memory for Task-solving Agents

arXiv:2606.06787v1 Announce Type: new Abstract: Large Language Models (LLMs) show promise as tool-using agents but remain limited in long-horizon tasks that require remembering, organizing, and reusin

Agentic Large Language Models for Automated Structural Analysis of 3D Frame Systems

arXiv:2606.06525v1 Announce Type: cross Abstract: Large language models (LLMs) have emerged as powerful foundation models with strong reasoning capabilities across domains. Beyond reactive text genera

An Analysis Focused on Womens Safety: Can VAD Models Be Enhanced by a Multi-modal Dataset?

model-releasesarxiv-cs-cv

arXiv:2605.25806v2 Announce Type: replace Abstract: Women's safety and security are paramount for a modern society. Crimes against women occur in daylight as well as in low-light conditions. Often, su

Autonomous heterogeneous catalyst discovery with a self-evolving multi-agent digital twin

arXiv:2606.05050v1 Announce Type: cross Abstract: Theoretical heterogeneous catalysis promises rapid catalyst discovery, yet computational and machine-learning predictions often deviate from experimen

Local Ai

b9564

local-aillama-cpp-releases

The search results did not provide specific information about the b9564 release. Based on the context of llama.cpp releases and the pattern observed with nearby releases (b9542, b9543, b9544, etc.), I

Beyond Rubrics: Exploration-Guided Evaluation Skills for Reward Modeling

model-releasesarxiv-cs-cl

arXiv:2606.07040v1 Announce Type: new Abstract: Open-ended reward modeling requires judges that can follow subtle, domain-specific preferences when verifiable answers are unavailable. Existing rubric-

CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks

arXiv:2509.14380v3 Announce Type: replace Abstract: Multi-Agent Reinforcement Learning (MARL) provides a powerful framework for learning coordination in multi-agent systems. However, applying MARL to

agentsarxiv-cs-ro

DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving

arXiv:2606.07001v1 Announce Type: cross Abstract: High-quality training data is essential to large language models (LLMs) and typically requires extensive and costly manual curation. Existing automati

Evaluate your Amazon Nova Sonic voice agent at scale, no microphone required

In this post, we walk you through the Nova Sonic Test Harness, an open source framework that we built to solve both problems. It serves as a rapid iteration tool for tuning system prompts and tool con

agentsaws-ml-blog

Evaluating AI-based Scientific Knowledge Synthesis with Epidemiological Systematic Reviews

arXiv:2603.22327v2 Announce Type: replace-cross Abstract: Systematic literature reviews (SLRs) are a demanding and high-stakes form of scientific knowledge synthesis that remains underspecified as an

Hearing the Unspoken: Language Model Priors for Acoustic Adversarial Attacks

arXiv:2606.06833v1 Announce Type: cross Abstract: Automatic Speech Recognition (ASR) systems operating in real-time settings must process acoustic input under strict temporal constraints, where transc

HKVM-RAG: Key-Value-Separated Hypergraph Evidence Organization for Multi-Hop RAG

arXiv:2606.07218v1 Announce Type: cross Abstract: Multi-hop RAG poses a data-engineering problem beyond passage matching: under fixed retrieval budgets, a system must organize retrieved text into evid

researcharxiv-cs-cl

How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures

arXiv:2606.06635v1 Announce Type: cross Abstract: Failures in language model reasoning emerge through distinct processes that leave identifiable signatures in the reasoning trace. We characterize thes

Local Ai

Learning Explicit Behavioral Models with Adaptive Questions and World-Model Probes

arXiv:2606.07127v1 Announce Type: new Abstract: Interactive agents trained only against task return can achieve high scores while failing to represent the mechanisms that make their actions succeed. T

local-aiarxiv-cs-lg

MACD: Model-Aware Contrastive Decoding via Counterfactual Data

arXiv:2602.01740v3 Announce Type: replace Abstract: Video language models (Video-LLMs) are prone to hallucinations, generating plausible but ungrounded content when visual evidence is weak, ambiguous,

Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA

arXiv:2603.24481v2 Announce Type: replace Abstract: Miscalibrated confidence scores are a practical obstacle to deploying AI in clinical settings. A model that is always overconfident offers no useful