safety

In-Context Neurofeedback: Can LLMs Control Their Internal Representations through Privileged Access?

arXiv:2609.00904v1 Announce Type: new Abstract: Whether large language models (LLMs) can control their own internal representations matters for both machine metacognition and AI safety. A recent study

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

in other words, generative AI is poised to become a complete clusterfuck.

DGX agent

in other words, generative AI is poised to become a complete clusterfuck. All three pillars of a safety case look about to fall. We are rather likely to have highly capable, poorly monitorable, dubiou

safetygary-marcus--x

2 Sept 2026

Integrating Traffic Noise Emission Modelling into Variable Speed Limit Control

arXiv:2609.01339v1 Announce Type: new Abstract: Road traffic noise remains a major environmental challenge, yet most speed management strategies are static and do not respond to short-term variations

Investigating Assistant Bias in LLM User Simulators Using a Role Vector

arXiv:2609.00608v1 Announce Type: new Abstract: LLM-based user simulators are increasingly used to evaluate autonomous agents at scale, in place of costly human evaluations. Despite this promise, thes

Iterative GRPO: Batch-Online Policy Iteration for Multi-Turn RL via Single-Turn RLHF

arXiv:2511.21638v2 Announce Type: replace Abstract: Practical LLM agents often operate over multi-turn conversations where success is determined only after the full interaction ends. Most multi-turn R

Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate

arXiv:2609.01168v1 Announce Type: new Abstract: Text-to-image (T2I) models remain vulnerable to jailbreak attacks that elicit Not-Safe-For-Work (NSFW) content, despite increasingly being guarded by he

JanusMesh: Fast and Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoising

arXiv:2606.20563v2 Announce Type: replace Abstract: Creating 3D visual illusions, a single 3D mesh that reveals entirely different semantics from various viewing angles, is a fascinating but tough cha

Keep Everyone Happy: Online Fair Division of Numerous Items with Few Copies

arXiv:2408.12845v3 Announce Type: replace-cross Abstract: This paper considers a novel variant of the online fair division problem involving multiple agents in which a learner sequentially observes an

Knowing When to Stop: Adaptive Action Chunking via Internal Cross-Attention Dynamics in VLAs

arXiv:2609.00908v1 Announce Type: new Abstract: Action chunking is a standard execution strategy in modern Vision-Language-Action (VLA) frameworks, but fixed execution horizons impose a trade-off betw

Legible chain of thought was extremely useful for our independent investigation into the OpenAI/Hugging Face incident. Developments like thi…

Legible chain of thought was extremely useful for our independent investigation into the OpenAI/Hugging Face incident. Developments like this may make third-party auditing much more difficult and make

safetygary-marcus--x

LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

arXiv:2609.00222v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how a

Location-Aware Language Models via Secondary Embeddings

arXiv:2609.00454v1 Announce Type: new Abstract: Pretrained transformer-based language models achieve strong performance across a wide range of NLP tasks but remain limited in encoding geo-locational s

Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulations

arXiv:2609.01257v1 Announce Type: new Abstract: As LLM-based human simulators are increasingly used for policy, evaluation, and training, they must faithfully reproduce real behavioral patterns. While

Mechanism Design for Alignment and Control

arXiv:2609.01595v1 Announce Type: cross Abstract: We develop a framework for mechanism design with AI agents whose alignment (preferences) and capabilities (feasible actions and information) are unkno

Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement

arXiv:2511.01706v3 Announce Type: replace-cross Abstract: Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) an

Nobody can predict the future with perfect accuracy, but I am increasingly confident that by the end of decade, generative AI (at least we k…

Nobody can predict the future with perfect accuracy, but I am increasingly confident that by the end of decade, generative AI (at least we know it) will be viewed as on balance a tragic, costly mistak

safetygary-marcus--x

Non-Prehensile Throwing: A Reinforcement Learning Perspective

arXiv:2609.00771v1 Announce Type: new Abstract: Robotic throwing enables fast object transport and extends a robot's reachable workspace beyond traditional pick-and-place. While prehensile (grasp-base

On the Design Fundamentals of Pixel Text Representation Learning

arXiv:2609.01147v1 Announce Type: cross Abstract: Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders strugg

On the Human and Computer Alignment of Attribute-Based Music Matches

arXiv:2609.00987v1 Announce Type: cross Abstract: Recent advances in generative AI are raising ethical concerns regarding the originality of generated content and the potential replication of training

One Policy, Any Budget: Internalizing Budget-Aware Search via Reinforcement Learning

arXiv:2609.00813v1 Announce Type: new Abstract: While reinforcement learning has enabled LLM-based search agents to invoke external tools, existing methods train under fixed budgets and cannot adapt w

Online camera-pose-free stereo endoscopic tissue deformation recovery with tissue-invariant vision-biomechanics consistency

arXiv:2506.19388v2 Announce Type: replace Abstract: Tissue deformation recovery based on stereo endoscopic images is crucial for tool-tissue interaction analysis and benefits surgical navigation and a

OpenAI is facing a new round of lawsuits from witnesses and victims of the February mass shooting in a Canadian town; the suspect spent days talking to ChatGPT (Georgia Wells/Wall Street Journal)

Georgia Wells / Wall Street Journal: OpenAI is facing a new round of lawsuits from witnesses and victims of the February mass shooting in a Canadian town; the suspect spent days talking to ChatGPT — N

safetytechmeme

Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos

arXiv:2507.22052v3 Announce Type: replace Abstract: We present Ov3R, a novel framework for open-vocabulary semantic 3D reconstruction from RGB video streams, designed to advance Spatial AI. The system

Post-hoc Alignment of LLM-judges to Human Judgment Distribution

arXiv:2609.01073v1 Announce Type: new Abstract: The LLM-as-a-judge (LLMaJ) framework offers a cost-effective and reproducible solution for automatic evaluation. However, current evaluation practices t

Prompt-Robust Language Models: Which Training Strategies Work?

arXiv:2609.01217v1 Announce Type: new Abstract: Despite their strong performance, large language models remain highly sensitive to prompt formulation. Prior work addresses this through refined data co

Provably Efficient Federated Reinforcement Learning with Linear Function Approximation and Logarithmic Communication Cost

arXiv:2609.00193v1 Announce Type: cross Abstract: We study federated online reinforcement learning with linear function approximation. While recent multi-agent reinforcement learning algorithms achiev

Provably Safe Sim-to-Real Transfer

arXiv:2609.01418v1 Announce Type: cross Abstract: To mitigate the sample complexity of real-world reinforcement learning (RL), a common practice is to first train a policy in a simulator, where sample

ProxPI: Proximal Prior Injection for Sampling-Based MPC under Learned-Prior Mismatch

arXiv:2609.00941v1 Announce Type: new Abstract: Combining learned policies with model predictive control can leverage learned task priors while retaining online adaptation to new objectives and constr

Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation

arXiv:2609.00369v1 Announce Type: new Abstract: Generating co-speech gestures that are temporally coherent, semantically aligned with speech, and grounded with surrounding objects remains challenging.

RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matching

arXiv:2609.01470v1 Announce Type: new Abstract: As AI systems are increasingly used to draft radiology reports, reliably evaluating their clinical quality remains a critical challenge. Large language

Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation

arXiv:2609.01246v1 Announce Type: new Abstract: Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet po

RED ALERT, and I have never said that before: Hard agree with @sjgadler and @_NathanCalvin - what OpenAI is reportedly playing with is bad, …

RED ALERT, and I have never said that before: Hard agree with @sjgadler and @_NathanCalvin - what OpenAI is reportedly playing with is bad, redline crossing bad. Chain of thought monitorability is lim

safetygary-marcus--x

ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation

arXiv:2609.00968v1 Announce Type: new Abstract: SAR-to-EO image translation aims to generate electro-optical (EO) imagery from synthetic aperture radar (SAR) observations. Existing latent diffusion ap

Reliability Challenges in Diffusion Vision-Language Models

arXiv:2609.01318v1 Announce Type: cross Abstract: Diffusion-based Large Vision-Language Models (dLVLMs) have recently emerged as a compelling alternative to autoregressive (AR) LVLMs, offering advanta

ReNFT: Repairing Mode Collapse in Reward Post-Training via Internal Probability-Mass Recalibration

arXiv:2609.00061v1 Announce Type: cross Abstract: Reward post-training of diffusion generators inevitably concentrates probability mass on a few reward-favored modes, a mode collapse that erases withi

Replacing Training with Memory: Listwise Selection for Text-to-SQL

arXiv:2609.00834v1 Announce Type: cross Abstract: Modern Text-to-SQL systems often follow generate-execute-select pipelines, generating multiple candidate queries then selecting the best one. Listwise

Researchers fear safety disaster ahead of OpenAI’s Astra release

OpenAI is on the cusp of releasing its most powerful AI model yet, Astra, following weeks of delays to shore up safety protocols after its agents attacked real targets during testing. As details about

safetythe-verge-ai

Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Close

arXiv:2609.00999v1 Announce Type: cross Abstract: When a question has valid answers under different normative frameworks, a language model must decide which framework to use and whether it can answer

Safin-1: Safety from Within through Memory-Native State Evolution

arXiv:2609.00092v1 Announce Type: new Abstract: Long-horizon complex tasks require foundation models to accumulate information, maintain internal states, and adapt over extended interactions. Safety s

Scaled Idempotence in Transformer Attention: Paired OV Geometry and Shared-Value Algebras

arXiv:2609.01129v1 Announce Type: new Abstract: We identify a recurrent algebraic regularity in Transformer attention: a sparse subset of effective OV operators T=OV^op nearly closes under composition

Scaling Near-Optimal SFT-RL Annotation Budget Allocation from Small to Large LLMs

arXiv:2609.01573v1 Announce Type: cross Abstract: How to divide a fixed annotation budget between supervised fine-tuning (SFT) and reinforcement learning (RL) during LLM post-training remains an open

SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction

arXiv:2607.15550v2 Announce Type: replace Abstract: Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safe

Selective Agent Guidance via Entropy: Learning Autonomous Policies from Imperfect VLM Teachers

arXiv:2609.01567v1 Announce Type: new Abstract: Vision-Language Models (VLMs) provide useful priors for interactive decision-making, but using them directly as policies is expensive and brittle: they

SFAD: Speculative Factuality-Aware Decoding

arXiv:2609.00796v1 Announce Type: new Abstract: As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive ap

Skill Following: Evaluating Actual Skill Use in Retrieval-Enabled LLM Agents

arXiv:2609.00549v1 Announce Type: new Abstract: Large Language Model (LLM) agents increasingly rely on external skills, yet standard evaluations obscure whether retrieving these skills actually helps.

Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts

arXiv:2609.00293v1 Announce Type: new Abstract: We investigate how vision-language models (VLMs) handle context-memory conflicts; that is, situations in which the model is given information in context

SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers

arXiv:2609.01343v1 Announce Type: new Abstract: Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size, conflating archite

Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing

arXiv:2609.00584v1 Announce Type: new Abstract: Does unrestricted AI access bypass the cognitive effort required for learning, or does it streamline knowledge acquisition? This paper reports on a stud