Search: “red-teaming”

LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios

arXiv:2602.03255v2 Announce Type: replace Abstract: Computer-use agents (CUAs) execute multi-stage tasks through tools, where an early unsafe decision can propagate to consequential actions. Evaluatin

5 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs

DGX agent

arXiv:2610.01058v1 Announce Type: cross Abstract: Quantized large language models are increasingly deployed on edge devices for their low latency and energy efficiency. However, model quantization wea

safetyarxiv-cs-ai

2 Oct 2026

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop

arXiv:2610.00559v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have shown strong multimodal reasoning capabilities, yet whether they truly capture the physical consistency underlying re

2 Oct 2026

Video-Index: A Curated Meta-Benchmark for Video Understanding

arXiv:2610.00960v1 Announce Type: new Abstract: A video benchmark should reward the capability it claims to measure, yet models can exploit answer options, question text, or partial visual evidence. W

2 Oct 2026

Refusals That Bend: Measuring and Predicting Task Malleability in Embodied VLM Planners

arXiv:2609.38971v1 Announce Type: cross Abstract: Embodied vision-language models (VLMs) are increasingly deployed as high-level planners for robots because they generalize across diverse environments

safetyarxiv-cs-ro

1 Oct 2026

The Geometry of Harmfulness in Multi-Turn Attacks

arXiv:2609.38389v1 Announce Type: cross Abstract: Large language models (LLMs) remain vulnerable to adversarial attacks that circumvent safety alignment to elicit harmful outputs. It remains unclear h

1 Oct 2026

Who Verifies the Graph? Misspecification Attacks on Causal Action Verification for Language Agents

arXiv:2609.40027v1 Announce Type: new Abstract: Causal action verifiers gate an agent's state-changing tool calls by checking whether each proposed intervention is identifiable against a committed act

1 Oct 2026

CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering

arXiv:2609.36570v1 Announce Type: cross Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that s

30 Sept 2026

Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-based Robots

arXiv:2412.16633v4 Announce Type: replace Abstract: LLM-based robots use Large Language Models (LLMs) as planners to translate natural language instructions into policies such as grasp(), move_to(), a

safetyarxiv-cs-ro

29 Sept 2026

From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMs

arXiv:2505.14368v3 Announce Type: replace-cross Abstract: Recent studies demonstrate that Large Language Models (LLMs) are vulnerable to attacks that generate harmful or sensitive outputs. As open-sou

28 Sept 2026

Stealth Apart, Harm Together: Skill Cascading Attacks on Skill-Based Agent Systems

arXiv:2609.30383v1 Announce Type: new Abstract: A skill is a modular package of natural-language instructions, executable scripts, and reference resources that an agent can load at runtime to extend i

28 Sept 2026

Why your startup needs open models alongside frontier APIs

model-releasesgoogle-cloud-ai

Every week, I talk with founders who are building at an unbelievable pace. Teams are moving from inception to product-market fit faster than ever, with foundation models wired deeply into their core p

28 Sept 2026

Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges

arXiv:2609.23894v1 Announce Type: cross Abstract: Agentic AI extends LLM security beyond generated content to persistent state, autonomous actions, tool use, and interactions with humans and other age

agentsarxiv-cs-ai

23 Sept 2026

Controlling risks of AI in chemical science with agents

arXiv:2312.06632v2 Announce Type: replace Abstract: Artificial intelligence is rapidly advancing scientific discovery, but this progress carries risks of misuse, such as the creation of harmful substa

23 Sept 2026

Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations

arXiv:2609.19101v1 Announce Type: new Abstract: As models scale, reward hacking becomes more frequent, more sophisticated, and more consequential. Does it leave a telltale signature in model represent

17 Sept 2026

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

arXiv:2607.18063v2 Announce Type: replace-cross Abstract: LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. We present a 21-scenario benchmark f

Industry

Democrats and Republicans proposed many bills to address major AI threats over the past two years, but they remain stymied by partisan disputes and lobbying (New York Times)

New York Times: Democrats and Republicans proposed many bills to address major AI threats over the past two years, but they remain stymied by partisan disputes and lobbying — For years, Democrats and

industrytechmeme

Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agents

arXiv:2609.15422v1 Announce Type: new Abstract: AI agents are provisioned the same as employee-owned hosts in many enterprise settings with a static credential set fixed at deployment which includes a

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

arXiv:2609.15134v1 Announce Type: new Abstract: Computer-use agents increasingly interact with browsers, terminals, file systems, and external services, introducing safety risks that emerge through ru

Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representations

arXiv:2609.14861v1 Announce Type: cross Abstract: A deployed language model may refuse a harmful request in English yet comply with its faithful translation, revealing a cross-lingual safety failure t

Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?

arXiv:2606.23189v2 Announce Type: replace-cross Abstract: Computer-use agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-appli

14 Sept 2026

GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science

arXiv:2606.16000v3 Announce Type: replace Abstract: We introduce GRACE-DS, a Guarded Reward-guided Agent Correction Environment in Data Science for pre-deployment evaluation of LLM-powered AutoML agen

14 Sept 2026

An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks

arXiv:2609.09404v1 Announce Type: cross Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read

11 Sept 2026

AgentFairBench: Do LLM Agents Discriminate When They Act?

arXiv:2606.16723v2 Announce Type: replace Abstract: Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is

Industry

Anthropic gives EU cybersecurity agency ENISA testing access to Mythos 5 after months of talks that began in late May; ENISA still lacks access to Mythos 5.1 (Gian Volpicelli/Bloomberg)

Gian Volpicelli / Bloomberg: Anthropic gives EU cybersecurity agency ENISA testing access to Mythos 5 after months of talks that began in late May; ENISA still lacks access to Mythos 5.1 — Anthropic P

industrytechmeme

Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability

arXiv:2609.10036v1 Announce Type: cross Abstract: Large language model agents produce fluent action sequences across a wide range of tasks, yet they fail in characteristic ways once the environment be

safetyarxiv-cs-lg

Building Trustworthy Graph-Agentic RAG for Social Good: Architectures, Failure Propagation, and Assurance by Construction

arXiv:2609.06391v1 Announce Type: new Abstract: Graph-agentic retrieval-augmented generation combines structured evidence with adaptive controllers that can plan retrieval, traverse relations, verify

agentsarxiv-cs-ai

VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models

arXiv:2609.09396v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric

CoRL: Co-Evolutionary Reinforcement Learning for Adaptive Indirect Prompt-Injection Attacks and Defenses

model-releasesarxiv-cs-lg

arXiv:2609.07529v1 Announce Type: new Abstract: Tool-augmented language agents are vulnerable to indirect prompt injection (IPI). Unlike direct prompt injection, IPI hides adversarial instructions in

9 Sept 2026

GradeTrap: Authority Cues in Images Shift VLM Judgments Despite Explicit Instructions to Ignore Them

arXiv:2609.06058v1 Announce Type: new Abstract: As vision-language models (VLMs) become increasingly capable and are deployed in consequential real-world settings, they must evaluate evidence independ

9 Sept 2026

RL-ADA: A World-Feedback Framework for Adversarially Robust Enterprise Dialogue Agents

arXiv:2609.02902v1 Announce Type: new Abstract: Deploying task-oriented dialogue agents in enterprise customer support faces a persistent annotation bottleneck: robust training requires labelled inter

agentsarxiv-cs-cl

4 Sept 2026

SENTINEL-RL: Offloading Topological Reasoning from LLM Agents in the Security Operations Center

arXiv:2609.04159v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly proposed as autonomous SOC analysts, but two limitations make them unreliable at enterprise scale:

4 Sept 2026

Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs

arXiv:2609.03955v1 Announce Type: new Abstract: Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for c

4 Sept 2026

As AI security concerns rise, HiddenLayer raises 100M and Huskeys bags 27M

Artificial intelligence security startup HiddenLayer Inc. today announced a 100 million Series B round, and a separate 27 million Series A went to edge security company Huskeys Security Ltd. Austin-ba

safetysiliconangle

3 Sept 2026

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

arXiv:2609.01691v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used to make decisions from visual inputs. We introduce FAIRLENS, a benchmark and evaluation framework fo

3 Sept 2026

SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment

arXiv:2609.02786v1 Announce Type: new Abstract: The performance of LLM-based agents is jointly shaped by the base model and the harness used when interacting with the environment. This exposes them to

3 Sept 2026

RestoreBench: Can AI Agents Restore Power Flow Convergence?

arXiv:2609.00384v1 Announce Type: new Abstract: Large Language Model (LLM) agents increasingly automate multi-step engineering workflows through tool use, interpretation of intermediate results, and i

2 Sept 2026

SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction

arXiv:2607.15550v2 Announce Type: replace Abstract: Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safe

2 Sept 2026

Controlling Refusal Behavior of LLMs via Stiefel-Constrained Rotation Steering

arXiv:2608.30986v1 Announce Type: cross Abstract: Activation steering has emerged as a lightweight approach for controlling model refusal at inference time. A growing line of research explores trainab

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees

arXiv:2603.22978v2 Announce Type: replace Abstract: In the maintenance of complex systems, fault trees are used to locate problems and provide targeted solutions. To enable fault trees stored as image

MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

arXiv:2603.02482v2 Announce Type: replace-cross Abstract: Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfo

Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection

arXiv:2608.30041v1 Announce Type: cross Abstract: Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later priv

Statutory AI: Aligning Large Language Models With Legal Norms

arXiv:2608.28593v1 Announce Type: new Abstract: With the increasing development of AI regulatory frameworks, ensuring that artificial intelligence systems, particularly generative models, operate in a

Securing Multi-Agent GIS Systems: Risk Evaluation and Prompt Hardening Optimization

arXiv:2606.17092v2 Announce Type: replace-cross Abstract: Agentic systems are increasingly integrated with geographic information systems (GIS), where multi-agent coordination enables complex conversa

agentsarxiv-cs-cl

31 Aug 2026

FaulT-Bench: Towards Benchmarking Network Troubleshooting LLM Agents under Unreliable User Tickets

arXiv:2608.27021v1 Announce Type: cross Abstract: LLM-based agents are increasingly proposed for network fault diagnosis, but existing benchmarks evaluate them only on accurate tickets and always assu

28 Aug 2026

SkillChain: Closing the Loop on Skill Evolution for Image-Based E-Commerce AI Assistants

arXiv:2606.12984v2 Announce Type: replace Abstract: Image-based AI assistants are now deployed at production scale on e-commerce platforms, where a single uploaded image can trigger fundamentally diff

28 Aug 2026

A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks

arXiv:2608.26008v1 Announce Type: cross Abstract: Large language models (LLMs) remain vulnerable to jailbreak attacks that exploit techniques such as role-playing, obfuscation, code transformation, an

27 Aug 2026

Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model

arXiv:2608.23663v1 Announce Type: cross Abstract: Aligning deployed language models requires knowing when their outputs can be trusted, yet on-device models now ship to hundreds of millions of devices