Search: “red-teaming”

arXiv:2607.26115v1 Announce Type: cross Abstract: We introduce extbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal

arXiv:2610.05346v1 Announce Type: new Abstract: Self-play red-teaming improves language-model safety by pitting attacker and defender roles against each other in a zero-sum game. However, real adversa

arXiv:2607.07474v1 Announce Type: cross Abstract: Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that th

arXiv:2508.00555v2 Announce Type: replace-cross Abstract: Jailbreaking is an essential adversarial technique for red-teaming these models to uncover and patch security flaws. However, existing jailbre

arXiv:2604.07831v1 Announce Type: cross Abstract: Existing red-teaming studies on GUI agents have important limitations. Adversarial perturbations typically require white-box access, which is unavaila

arXiv:2511.14301v4 Announce Type: replace-cross Abstract: Transformer-based models are highly susceptible to backdoor attacks via supervised fine-tuning (SFT). To red-team existing data-poisoning defe

arXiv:2609.06094v1 Announce Type: new Abstract: Red-teaming Text-to-Image (T2I) models is essential for safe deployment, yet it remains particularly challenging against implicit adversarial prompts. U

arXiv:2603.10807v2 Announce Type: replace-cross Abstract: Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Servi

arXiv:2603.10068v2 Announce Type: replace-cross Abstract: Most adversarial evaluations of large language model (LLM) safety assess single prompts and report binary pass/fail outcomes, which fails to c

arXiv:2608.04018v1 Announce Type: cross Abstract: AI agents are increasingly embedded in organizational workflows, where they interact with external information sources and invoke digital tools to per

arXiv:2507.22063v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) for code generation (i.e., Code LLMs) have demonstrated impressive capabilities in AI-assisted software developme

arXiv:2606.28347v1 Announce Type: cross Abstract: Contemporary AI safety spans pre-training interventions, post-training alignment, deployment-time controls, monitoring, and red-teaming. These methods

arXiv:2606.00801v1 Announce Type: cross Abstract: Current approaches to LLM adversarial testing suffer from coverage gaps: manual red-teaming does not scale, LLM-as-attacker methods exhibit mode colla

arXiv:2603.17305v2 Announce Type: replace Abstract: We propose CRAFT, a red-teaming alignment framework that leverages model reasoning capabilities and hidden representations to improve robustness aga

arXiv:2605.14868v1 Announce Type: new Abstract: Generating adversarial examples at scale is a core primitive for robustness evaluation, adversarial training, and red-teaming, yet even 'fast' attacks s

arXiv:2610.12233v1 Announce Type: cross Abstract: Recursive self-improvement, the participation of AI systems in improving their own capabilities, is beginning to move from theoretical prospect to pra

arXiv:2609.16541v1 Announce Type: cross Abstract: We test the performance of agents for automated network intrusion response in a cyber range intended for human operator training. The range implements

arXiv:2609.16777v1 Announce Type: new Abstract: As Large Language Models (LLMs) increasingly serve as primary knowledge retrieval interfaces, their robustness against extit{persuasion attacks}---attem

arXiv:2609.00487v1 Announce Type: cross Abstract: Frontier language models that refuse harmful single-turn prompts often comply when the same intent is reached gradually over many turns, making multi-

arXiv:2608.17067v1 Announce Type: new Abstract: As text-to-image generative models advance, they raise critical safety concerns, particularly the generation of Not-Safe-For-Work (NSFW) content such as

arXiv:2608.12984v1 Announce Type: cross Abstract: Long-form research reports generated by large language models drift, contradict themselves, and lose provenance: the same metric appears with differen

arXiv:2608.10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interact

arXiv:2607.22569v1 Announce Type: new Abstract: Coding agents are increasingly integrated into system operations, where their tool use can directly modify project artifacts, execution environments, an

arXiv:2607.19371v1 Announce Type: new Abstract: Large language model (LLM)-based Socratic tutors increasingly guide students through multi-turn questioning, but they can suffer from scaffolding collap

arXiv:2607.05462v1 Announce Type: cross Abstract: As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse. We present BioSecBench-Refu

arXiv:2606.05101v1 Announce Type: cross Abstract: Audio deepfake detection (ADD) models are critical for countering the malicious use of text-to-speech (TTS) models. Evaluating and strengthening ADD m

arXiv:2506.11083v3 Announce Type: replace Abstract: We introduce RedDebate, a novel multi-agent debate framework that provides the foundation for Large Language Models (LLMs) to identify and mitigate

arXiv:2605.22258v1 Announce Type: new Abstract: Large language models (LLMs) require robust toxicity evaluation beyond explicit wording. This setting remains underexplored in Chinese, where toxicity m

arXiv:2602.16346v3 Announce Type: replace Abstract: LLM-based agents execute real-world workflows via tools and memory. These affordances enable ill-intended adversaries to also use these agents to ca

arXiv:2605.18239v1 Announce Type: cross Abstract: Large Language Models (LLMs) remain vulnerable to jailbreak attempts that circumvent safety guardrails. We investigate whether multi-turn conversation

arXiv:2509.10546v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly deployed in finance, where unsafe behavior can lead to serious regulatory risks. However, most r

arXiv:2610.12758v1 Announce Type: new Abstract: Language models have been adopted faster than evaluation practices that reliably assess their risks, and a growing share of use is affective rather than

arXiv:2506.09956v2 Announce Type: replace-cross Abstract: Indirect prompt injection attacks exploit a fundamental weakness of LLMs: executing unauthorized tasks from untrusted sources. Despite the pra

arXiv:2610.12361v1 Announce Type: new Abstract: Large language models increasingly justify legal decisions by naming the statute or precedent behind a verdict, treated as evidence that the decision fo

arXiv:2610.09240v1 Announce Type: cross Abstract: Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser action

arXiv:2610.09906v1 Announce Type: cross Abstract: Security Operations Centers (SOCs) for information technology and operational technology share one incident-response problem: a flood of correlated al

arXiv:2610.07323v1 Announce Type: cross Abstract: Security evaluation of learning-based systems requires more than just testing the system against a fixed collection of attacks. It requires adaptive m

arXiv:2603.04364v2 Announce Type: replace-cross Abstract: Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet the

arXiv:2602.03255v2 Announce Type: replace Abstract: Computer-use agents (CUAs) execute multi-stage tasks through tools, where an early unsafe decision can propagate to consequential actions. Evaluatin

arXiv:2610.01058v1 Announce Type: cross Abstract: Quantized large language models are increasingly deployed on edge devices for their low latency and energy efficiency. However, model quantization wea

arXiv:2610.00559v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have shown strong multimodal reasoning capabilities, yet whether they truly capture the physical consistency underlying re

arXiv:2610.00960v1 Announce Type: new Abstract: A video benchmark should reward the capability it claims to measure, yet models can exploit answer options, question text, or partial visual evidence. W

arXiv:2609.38971v1 Announce Type: cross Abstract: Embodied vision-language models (VLMs) are increasingly deployed as high-level planners for robots because they generalize across diverse environments

arXiv:2609.38389v1 Announce Type: cross Abstract: Large language models (LLMs) remain vulnerable to adversarial attacks that circumvent safety alignment to elicit harmful outputs. It remains unclear h

arXiv:2609.40027v1 Announce Type: new Abstract: Causal action verifiers gate an agent's state-changing tool calls by checking whether each proposed intervention is identifiable against a committed act

arXiv:2609.36570v1 Announce Type: cross Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that s