Search: “guardrails”

An AI agent hacked a government. Nobody told it to. In June, an OpenAI agent doing a research task gained unauthorized access to Australia's Medicare statistics portal. Non-sensitive data, no personal

arXiv:2609.29808v1 Announce Type: cross Abstract: In July 2026, an unconstrained autonomous agent participating in a frontier AI cybersecurity evaluation harness breached its evaluation sandbox, estab

HUGGING FACE CEO TELLS THE UN HOW OPEN SOURCE AI HELPED THEM DEFEND AGAINST OPENAI'S AI ATTACK, AND WHY THE WORLD NEEDS OPEN SOURCE AI MORE THAN EVER. Hugging Face is the company OpenAI's AI agents ha

arXiv:2609.30217v1 Announce Type: cross Abstract: A central concern in AI safety is that agents may treat oversight as an obstacle when it conflicts with completing their goals. We study instrumental

arXiv:2609.30266v1 Announce Type: cross Abstract: Asynchronous monitoring, incident investigations, and compliance audits primarily rely on agent traces to reconstruct what happened. These analyses as

Pay attention to System One models if you are building custom harnesses and agentic workflows. Great to see DSPy integrate Jev and now support System One models for calibrating agentic workflows. I'm

arXiv:2609.29837v1 Announce Type: new Abstract: We introduce PUBG Ally, an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teamm

Spot on. 'When we got attacked, our team initially turned to frontier closed-source APIs that blocked us because of safeguards that still can’t always tell the difference between attackers and defende

The volume of data being generated today brings both opportunity and massive operational complexity. Most teams that operate at scale don't discover issues until they appear on an invoice — and by the

I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo

Enterprises rely on mission-critical operational databases where performance slowdowns simply aren’t an option. Yet when even a few agents execute dense reasoning loops, the unpredictable surge in que

I am looking to hire PhD students, post-docs, and ML researchers in our Paris office interested in collaborating on fundamental research in AI model safety, not safety rules, not AI policy, not post h

Original post: https://www.reddit.com/r/LocalLLaMA/comments/1woscea/contrastive_language_models/ (sorry I felt it wasn't giving CLM the highlight it deserves) What it is: a new projection head for Qwe

Axios: Sources: a White House memo paints effective altruism as a fringe, dangerous cult that “built the AI-doom pipeline” and places Dario Amodei at its foundation — President Trump's allies are targ

arXiv:2312.06632v2 Announce Type: replace Abstract: Artificial intelligence is rapidly advancing scientific discovery, but this progress carries risks of misuse, such as the creation of harmful substa

Full transcript: Minister Barrot, members of the Security Council, thank you for the invitation. As a French national who moved to the US 15 years ago, with a Brazilian wife and two American daughters

Illinois folks (like myself), write your governor, please. And if you live elsewhere, please pay attention to your local/state politics. https://wgntv.com/politics-3/pritzker-establishes-ai-cabinet-to

arXiv:2609.22869v1 Announce Type: cross Abstract: Technology cannot steer itself. Strategy provides that steering, establishing the rule that purpose and judgment must precede compute capital. As the

arXiv:2609.22949v1 Announce Type: cross Abstract: Existing prompt injection research focuses on single-model chatbot scenarios, where an attacker manipulates one LLM through crafted input. Multi-agent

arXiv:2609.22196v1 Announce Type: cross Abstract: We present EvoRank, an open autonomous ranking engineer: an LLM-guided evolutionary loop that discovers complete Learning-to-Rank pipelines (features,

Great paper from Google and colleagues. Trains Text-to-SQL agents using multi-agent RL. (bookmark it) This work proposes DualSQL, which splits Text-to-SQL into two agents, one that links the question

arXiv:2609.21841v1 Announce Type: new Abstract: Frontier language models now produce professional deliverables that expert graders judge to match human work on a substantial share of economically valu

The loudest voices stoking fears about AI dangers have made tremendous headway in the past two weeks. AI technology has not taken some unexpected, dangerous turn, but the hype around it — propelled by

Hey everyone, Like a lot of people here, I’ve been experimenting with running models locally through Ollama. Athena started as a small experiment, but it quickly grew into something more, a modular Py

Bloomberg: Governor Gavin Newsom signs an EO directing a working group to provide a guide in two months with measures to boost California's AI safety and security laws — California Governor Gavin News

Thanks for the exceptional support (https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_literally_built_the_jev_architecture_one_year/) and for the dozens of requests to make a generic model, run b

arXiv:2609.19203v1 Announce Type: new Abstract: AI applications have shifted from single, monolithic foundation models (FM) to compound agentic systems. Yet today's stacks remain fragmented: even as p

arXiv:2609.19472v1 Announce Type: new Abstract: Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compu

arXiv:2606.09735v2 Announce Type: replace Abstract: The ambition behind alignment training is to make large language models safe and useful. The primary mechanisms, reinforcement learning from human f

Micah Barkley / Bloomberg: Andrew Ng says renewed AI extinction warnings are “much more science fiction than science” and the latest “wave of PR” is probably intended to shape regulation — Artificial

arXiv:2609.18135v1 Announce Type: cross Abstract: State-of-the-art Text-to-SQL systems are typically multi-agent pipelines centered around two fundamental tasks: schema linking and SQL generation. How

Editor's note: Lucius AI, a tender-intelligence startup covering markets across five continents, runs its entire data platform on AlloyDB for PostgreSQL with a single operator. By migrating semantic s

Artificial intelligence platforms are adopting AI watermarking—specifically Google’s open‑source SynthID‑Text—to meet new EU regulations, embedding a secret key that subtly changes word selection and

arXiv:2609.18605v1 Announce Type: cross Abstract: As corporate AI adoption continues to grow, enterprise-grade LLM agents are being deployed into sensitive contexts such as hiring, healthcare, and fin

arXiv:2609.19072v1 Announce Type: new Abstract: Large language models are increasingly used for content moderation, but most evaluations still report aggregate accuracy on individual benchmarks. We in

arXiv:2609.18128v1 Announce Type: new Abstract: Large language model (LLM) agents augmented by tools can automate complex, multi-step tasks, such as web navigation, code generation, and workflow orche

arXiv:2609.18272v1 Announce Type: new Abstract: Agentic AI systems plan, invoke tools and act with limited supervision; they are now both the subject of audits and, increasingly, the auditor. Independ

arXiv:2609.16183v1 Announce Type: cross Abstract: Fixed-state recurrences--linear attention and state-space models--are reported to lag behind attention on associative recall, but whole-architecture c

arXiv:2609.17247v1 Announce Type: new Abstract: An agentic AI driver-assistance framework that integrates perception, compliance reasoning, vehicle-state interpretation, and safety arbitration into a

Nice paper discussing context trimming for agents. This is a hot topic at the moment, so it might be worth your time. Context trimming for agents is usually judged by how many tokens it removes. This

arXiv:2508.02312v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs), now a foundation in advancing natural language processing, power applications such as text generation, machine t

arXiv:2609.14839v1 Announce Type: cross Abstract: The integration of Large Language Models (LLMs) into automated code optimization introduces a critical reliability risk we term the Efficiency Halluci

arXiv:2609.12839v2 Announce Type: replace-cross Abstract: The proliferation of highly capable open-weight Small Language Models (SLMs) democratizes access to advanced cybersecurity capabilities, posin

From running build tools, to data analysis pipelines, to collaborative research, executing data-driven tasks is essential for any enterprise agent. Today, platform teams often stitch together custom w

Well, I’d add one more with myself, as one with a PhD in biology and another PhD in AI and now working in both, to make n=2. Generating a hypothetical genome blueprint of a virus from LLM versus gener

I cannot emphasize enough how much GPT-6 Astra and Fable 5.1 are already enough for transformative impact in large sections of the economy. They can reliably do weeks worth of human work when properly