safety

Here’s what AI leaders are saying about Trump’s new safety plan

After hosting a meal with Big Tech leaders on Tuesday, President Donald Trump responded to journalist questions about his artificial intelligence announcements in typical fashion. He said his previous

safetythe-verge-ai

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Here’s how tech leaders will self-police AI safety under Trump’s deal

DGX agent

We now have the full details of the 'morally binding' AI safety deal announced by President Trump yesterday, in which top executives agreed to self-regulate their artificial intelligence technology. T

safetythe-verge-ai

30 Sept 2026

High-Dimensional Partial Least Squares: Spectral Analysis and Fundamental Limitations

arXiv:2512.15684v2 Announce Type: replace-cross Abstract: Partial Least Squares (PLS) is a widely used method for data integration, designed to extract latent components shared across paired high-dime

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and (L_0,L_1)-Smoothness

arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees over

HiRAE: Hierarchical Representation Autoencoding with Residual Budgets

arXiv:2609.37775v1 Announce Type: cross Abstract: Pretrained visual representations support image generation, but may not fully preserve the fine-grained details needed for faithful reconstruction. Me

How Local Mixing Encodes Relative Position in Global NoPE Attention

arXiv:2609.38109v1 Announce Type: cross Abstract: The attention operation is naively position invariant. However, positional information is fundamental to natural language, and therefore a variety of

How to Run Statistics over LLM Judges and Trust the Results: Calibrated Inference for Small-Sample AI Evaluation with evalstats

arXiv:2609.35815v1 Announce Type: cross Abstract: Researchers across academia increasingly base significance claims on LLM judge scores and small-sample AI evaluations. Yet without well-calibrated con

Improving scalable oversight with co-trained monitors

arXiv:2609.36049v1 Announce Type: cross Abstract: Worker-monitor setups are a promising approach to AI oversight, but training workers against fixed monitors can incentivize monitor evasion. We study

In light of the latest Anthropic's latest not-particularly-oblique attempt to build a case against open weights, I thought I would write dow…

In light of the latest Anthropic's latest not-particularly-oblique attempt to build a case against open weights, I thought I would write down how I currently think about open weight AI models. (1) Ope

safetyyann-lecun--x

Interacting particle guidance for sampling reward-tilted generative priors

arXiv:2609.37227v1 Announce Type: new Abstract: Inference-time steering adapts pretrained diffusion and flow-based models to new tasks, e.g., to generate samples from a conditional distribution or sam

Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation

arXiv:2609.37170v1 Announce Type: cross Abstract: Off-policy and on-policy distillation have traditionally been formulated as separate paradigms, each favoring a different property of distillation tra

Introducing the CZAR Loss: A Tailored Objective Function for Financial Log-Return Predictions

arXiv:2609.36061v1 Announce Type: new Abstract: In quantitative finance, standard regression losses are misaligned with the economics of return prediction. As the conditional mean of financial log-ret

Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation

arXiv:2609.36695v1 Announce Type: new Abstract: Self-distillation turns knowledge distillation into a closed learning loop and offers a path toward recursive self-improvement. Without an external teac

KPI: A Promptable Kernel for Physical Interaction on Humanoids

arXiv:2609.36151v1 Announce Type: new Abstract: Humanoids now walk, balance and reach with remarkable generality: one whole-body tracking policy follows references from a human, or from an end-to-end

safetyarxiv-cs-ro

KUPAS MASTER: Distilling the Tacit Expertise of Master Practitioners into Agent-Ready Experience Corpora

arXiv:2609.37673v1 Announce Type: new Abstract: Experienced professionals know more than just facts and conclusions. They know which cues matter, why a judgment is reasonable, and which action to take

Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

arXiv:2609.37024v1 Announce Type: new Abstract: Integrating transcriptomic and electrophysiological data is essential for building multimodal foundation models for neuroscience. Patch-seq provides pai

Large-scale factor analysis shows machine intelligence is only partially interpretable

arXiv:2609.36515v1 Announce Type: cross Abstract: A common assumption in language model development is that cognitive abilities are organized around a general, domain-free intelligence factor, like fl

LatentSift: Policy-State Filtering for Token-Efficient Verification of Software Engineering Agents

arXiv:2609.36371v1 Announce Type: cross Abstract: Test-time scaling improves software engineering agents by generating multiple candidate trajectories and selecting the best one. Verifying and selecti

Learn from the Gap: Differential-Aware Advantage Pruning with Adaptive Rollout Sampling for GRPO

arXiv:2609.36932v1 Announce Type: new Abstract: Recently, Group Relative Policy Optimization (GRPO) and its variants have been developed for policy optimization and demonstrated notable performance ga

Learned Reporting Preferences in RLVR Can Conflict with the Current Request

arXiv:2609.36587v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a prominent approach for improving language-model performance on reasoning tasks using

Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR

arXiv:2609.37868v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) methods such as GRPO rely on successful self-generated trajectories, but finite rollout budgets

Learning Expressive and Compositional Motion Representation via Spectral Skills

arXiv:2609.37677v1 Announce Type: new Abstract: Robotic foundation models offer a promising path toward general-purpose humanoid robot control, often through hierarchical architectures. However, their

safetyarxiv-cs-ro

Learning from Think-Mode Advantage via On-Policy Distillation

arXiv:2609.37044v1 Announce Type: new Abstract: Explicit intermediate reasoning gives large language models (LLMs) a stronger problem-solving mode. We study learning from this think-mode advantage via

safetyarxiv-cs-cl

Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents

arXiv:2609.37111v1 Announce Type: new Abstract: Long-horizon LLM agents require reinforcement learning methods that can assign credit to intermediate decisions under sparse and delayed rewards. Existi

Learning to Harvest Without Collapse in a Regenerative Commons: A Lagrangian Framework

arXiv:2609.36478v1 Announce Type: new Abstract: The tragedy of the commons poses a multi-agent safety problem: reward-seeking agents can deplete a shared resource, and cooperation among its users does

Learning What to Remember: Long-horizon Counterfactual Memory Optimization

arXiv:2609.37930v1 Announce Type: new Abstract: Persistent textual memory allows language models to carry information across long interactions, but learning what to remember is fundamentally a credit-

safetyarxiv-cs-cl

LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation

arXiv:2609.38146v1 Announce Type: new Abstract: We introduce LIFT, a unified image-to-video generation framework that complements camera control with Layout-In-FuTure control, enabling users to specif

LLM utility has increased, and deeper understanding to a degree, but I still very much agree with this post from 2020 @GaryMarcus

LLM utility has increased, and deeper understanding to a degree, but I still very much agree with this post from 2020 @GaryMarcus Forgets GPT~3. Real #AI news of 2020 is that @ylecun & Y Bengio have c

safetygary-marcus--x

Looped Actor: Depth-Recurrent Reasoning Models for Reinforcement Learning

arXiv:2609.37432v1 Announce Type: new Abstract: Looped reasoning models repeatedly apply a shared set of parameters, enabling more computation without increasing the model size. These models also supp

Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by Trust

arXiv:2609.37156v1 Announce Type: cross Abstract: World models enable agents to learn and plan in imagination, but predictions beyond their experience can become unreliable and mislead decisions. Exis

Making Duplicate Reimbursement Unrepresentable: A Verified Ethereum E-Invoice System for Humans and AI Agents

arXiv:2609.37819v1 Announce Type: cross Abstract: Electronic invoices are replacing paper invoices worldwide, but today's centralized architectures leave three problems unsolved on the consumption sid

Markovian Nonconvex ADMM for Reinforcement Learning: Bellman-Resolvent Stability Beyond Smooth Blocks

arXiv:2609.36859v1 Announce Type: new Abstract: We identify and study a structural mechanism for Markovian nonconvex ADMM in reinforcement learning. Using finite discounted MDPs as a canonical proving

MeanFlowAdvantage: Stable Reward Fine-Tuning for Few-Step Average-Velocity Generators

arXiv:2609.37670v1 Announce Type: new Abstract: MeanFlow enables efficient few-step generation by predicting interval-average velocities, but this representation creates a mismatch for reward fine-tun

MIRI Newsletter #127

Meeting the moment The global conversation on AI has begun. In the wake of Jacob Coxon’s resignation from Anthropic, calls by AI CEOs and over a thousand employees for a potential industry slowdown, a

safetymiri

Mixture of Self-Improving Branches For Agent Harness Optimization

arXiv:2609.37834v1 Announce Type: new Abstract: Harness optimization provides a practical setting for recursive self-improvement (RSI), where agent-generated modifications inform subsequent changes th

Motion Concept Unlearning in Video Diffusion Models

arXiv:2609.36832v1 Announce Type: new Abstract: Text-to-video (T2V) diffusion models can generate realistic depictions of actions such as kicking, stabbing, and shooting, raising safety concerns that

MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videos

arXiv:2609.37030v1 Announce Type: cross Abstract: Despite rapid progress in video generation models, they still exhibit obvious motion deficiencies, often manifested as incorrect object motion. Howeve

MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation

arXiv:2609.38078v1 Announce Type: new Abstract: Vision-language-action (VLA) models have advanced robotic manipulation, but their zero-shot generalization in new tasks and environments remains limited

safetyarxiv-cs-ro

Multi-Granularity Language-Guided Imitation Learning via Instruction Decomposition

arXiv:2609.37135v1 Announce Type: new Abstract: Using language instructions as conditions to guide robot policy learning has recently become an important research domain. However, existing language-gu

NesTok: Nested Self-Aligned 1D Tokenizer for Autoregressive Image Generation

arXiv:2609.36756v1 Announce Type: cross Abstract: One-dimensional (1D) variable-length visual tokenizers enable adaptive compression by varying the number of tokens, allowing downstream autoregressive

Non-Myopic Active Feature Acquisition via Pathwise Policy Gradients

arXiv:2605.05511v2 Announce Type: replace Abstract: Active feature acquisition (AFA) considers prediction problems in which features are costly to obtain and the learner adaptively decides which featu

NVIDIA proposes an AI agent kill switch in silicon after a year of sandbox escapes

This year agents at OpenAI, Anthropic, and Google escaped environments meant to contain them. What decided severity was time-to-detect and time-to-kill, from 12 minutes to seven months. NVIDIA's Open

safetyarize-ai

OFBD: Object-Focused Background Debiasing for Long-Tailed Learning

arXiv:2609.37331v1 Announce Type: new Abstract: Balancing performance trade-offs on long-tailed data distributions remains a long-standing challenge in visual recognition. Existing methods mainly impr

OmniTaskonomy: When Does Visual Generation Improve Visual Understanding?

arXiv:2609.38079v1 Announce Type: new Abstract: Training a model to generate visual content can encourage it to learn rich perceptual capabilities related to geometry, spatial relationships, and objec

On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning

arXiv:2405.14749v3 Announce Type: replace-cross Abstract: Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in high-stakes applications. While traditional RL m

One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models

arXiv:2609.36101v1 Announce Type: cross Abstract: Contrastive vision-language models learn shared embedding spaces by aligning matched image-text pairs, yet their representations remain separated by a

OpenAI-HuggingFace: A Reproduction & Lessons for Alignment Testing

arXiv:2609.35799v1 Announce Type: new Abstract: In July 2026, OpenAI's agents coordinated over channels outside their intended environment to breach Hugging Face's secured infrastructure. Could existi

Optimal Multi-Reward Reinforcement Learning

arXiv:2609.36486v1 Announce Type: new Abstract: We study an unknown-transition finite-horizon Markov decision process (MDP) with a finite collection of known reward functions {r^1, r^2, ldots, r^M}. T