Search: “papers”

// Your Agents are Aging Too // Huh!? They need 'sleep,' and now they are aging? Joke aside, great write-up on reliable agentic engineering. This new research introduces AgingBench, a longitudinal rel

arXiv:2605.24076v1 Announce Type: cross Abstract: Modern Artificial Intelligence achieves remarkable predictive power by optimizing statistical risk functionals over vast corpora. Yet a gap separates

System scaling is the next real bottleneck in agentic AI. If you build agent orchestration layers, this is a clean map of where the engineering leverage actually sits. The labs own the model. You own

📢 Accepted to TMLR, with reproducibility certification 🏅 v2 of our JEPA-WM study (arXiv:2512.24497) is out, with new data-scaling experiments, a Lipschitz analysis of multistep rollout training, and e

// The Efficiency Frontier in LLMs // (bookmark this one) How much are you overpaying for context you do not need? It turns out that context costs dominate production LLM bills, and the right strategy

As more people come to recognize the tells of AI, which mostly happens as you start to work with AI a lot, the scales are going to fall from their eyes and they are going to realize what some of us al

Can frontier models forecast scientific progress? Mostly no, but here is why. This work looks at 4,760 scientific events across disciplines. Frontier models can identify plausible research directions

Image diffusion models like Flux natively output at 1k resolution, but what if we want to generate much higher resolution images (6k+)? SEGA modifies the RoPE encodings during the diffusion process to

Gated DeltaNet has been one of my favorite 'hybrid attention' newcomers in the good old transformer stack. Excited to see Gated DeltaNet-2. Adding it to my reading stack. In the meantime, I have a pri

Today we release a study on decoupling the benefits of subword tokenization for language model training, by simulating each suspected benefit one at a time inside a 1.7B byte-level pretraining pipelin

What we find most useful about this study is the decomposition. For byte-level pretraining, the ordering is throughput first and boundary signal second; the boundary benefit can be recovered without a

What we find most useful about CNA is that the intervention is simple yet powerful. The steering is a multiplicative ablation on a sparse set of MLP neurons, which makes CNA a clean addition on top of

just as i feared AI can be a powerful tool for opening people up to new perspectives, yet we find that people actually prefer to use “sycophantic” AI systems that reinforce their pre-existing beliefs.

There are a lot of coding and reasoning benchmarks for AI agents, but not a lot for document understanding - which is a prerequisite for all downstream knowledge work. We released ParseBench ~a month

arXiv:2605.14855v1 Announce Type: cross Abstract: Forecasting within signal processing pipelines is crucial for mitigating delays, particularly in predicting the dynamic movements of objects such as N

Open-ended coding training data may no longer be the bottleneck: AI can scale open-ended tasks—and even outperform human-expert curation. FrontierCS team is releasing FrontierSmith: a system for synth

Very thoughtful post on the role and success of open source, including potential applications for AVs and AI. Long, but worth the read. -- 'Open source is no longer just how good software gets built.

// Harnessing Agentic Evolution // Pay attention to this one if you run iterative agentic search loops. (bookmark it) AEvo splits the self-improvement loop into two jobs: > One proposes the next candi

Making humans responsible for their AI use seems like an incredibly reasonable way to address problems & opportunities in the use of AI for academic research, at least in the short term (autonomous sc

We’re training models wrong and it’s due to chatGPT. Even the modern coding agents used daily still use message-based exchanges: They send messages to users, to themselves (CoT) and to tools, and rece

Cool idea from Nous Research. What if you could speed up long-context pretraining with a subquadratic wrapper that you remove before deployment? That is the idea behind Lighthouse Attention. The metho

This NVIDIA remains the strongest platform for large-model inference at scale. Prefill/decode disaggregation, Blackwell-native quantization, custom kernels, and rack-scale NVLink turn GB200 into faste

tl;dr - let's not just remove negative behavior from models, but also add positive ones 👍 If anyone builds it, everyone thrives. Over the past decade, a lot of important work on AI alignment has focus

The inability of AI models to produce creative variation is a huge gap. The fact that they generate similar ideas limits their ability to do science & the same-y writing limits their usefulness in man

// The Memory Curse in LLM Agents // (bookmark it) Long histories apparently degrades agents as they become increasingly history-following and risk-minimizing. Across 7 LLMs and 4 social dilemma games

32,000 views for my tweet; 2.2 million for the nonsense it critiques. so typical. lies and distortions beating debunking by a factor of roughly 100. Wanna get a million views? Make stuff up. Take a ti

So many people misremember (or never read) what I said in in 2022 in “Deep learning is hitting a wall”, which was neither about revenue or AI’s potential upper limits. Rather, it was an argument that

“Combined free cash flow across Microsoft, Alphabet, Amazon, Meta, and Oracle is projected to FALL more than -70%, to ~100 billion, by the end of …. AI capital expenditure is consuming nearly every do

Badass tank that beats your sports car in a drag race Only when you drive the Cybertruck do you realize how incredible it is: a bulletproof tank that moves like a million dollar sports car! Reason for

Only when you drive the Cybertruck do you realize how incredible it is: a bulletproof tank that moves like a million dollar sports car! Reason for the angular shape is that the thick, ultra-hard stain

🧠 Introducing NeuralBench: a unified, open-source framework to benchmark NeuroAI models. v1.0: 36 EEG tasks, 94 datasets, task-specific + foundation models. MEG/fMRI ready. MIT-licensed, FAIR's Brain

arXiv:2512.07578v3 Announce Type: replace-cross Abstract: Global SHAP explanations are typically presented as feature-importance rankings, which identify variables that matter to a black-box model but

I want to tell you the story of a young woman who you have probably never heard of. Her name is Mary Anne. She was born on a remote island in Scotland, where life was harsh and unforgiving. On May 2,

Introducing nanowhale 🐳! A tiny DeepSeek model fully pretrained by an agent. Inspired by @karpathy's nanochat, we gave ml-intern the task of training a tiny MoE with all the architectural advancements

There's a clay tablet with the founding charter of a 12-partner company on it. Twelve merchants pooled 33 pounds of gold to start the firm. The contract has the partner names, the starting capital, th

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely b

音声AIの素早さと賢さを両立できるか？ 私たち人間は会話の中で、言いたいことを全部まとめてから話し始めるのではなく、話しながら考えを整理していきます。応答の速い Speech-to-Speech モデルは、この「話しながら考える」を実現しましたが、そのぶん思考が浅くなりがちです。かといって知識豊富な LLM を挟むカスケード型では、遅延が生じるため「話しながら」が成立しません。 そこで Sakan

In the future you have a choice. Do you engage brain? or Do you cheat? There will be other choices too, such as: Do you go to the casino? Or to the library or maker space? I fear most will make the ea

ml-intern is fully on mobile now you can launch 8 A100s from your phone. while on the couch. while commuting. wherever I just did this while biking. same sessions as your desktop too — start a run on

// OCR-Memory // Well this is a unique approach to store memory for long-horizon agents. Most of the agent memory systems compress trajectories into text summaries and hope the model remembers what ma

Two Heads Are Better Than One: Async Knowledge Injection for Speech AI with Tandem Architecture Blog: https://pub.sakana.ai/kame/ 🐢 KAME: Tandem Architecture for Enhancing Knowledge in Real-Time Speec

Warning: GenAI-induced cognitive surrender may kill innovation. Consider the following, from CEO @wadhwa’s newsletter: “I have been speaking with recent graduates in India, many of them highly recomme

// When to Retrieve During Reasoning // Pay attention to this one, AI devs. (bookmark it) Most RAG systems retrieve once, before the model starts reasoning. Large reasoning models like o1 and R1 don't

// Agentic Harness Engineering // Pay attention to this one, AI devs. (bookmark it) Most coding-agent harnesses are still tuned by hand or brittle trial-and-error self-evolution. This new work introdu

For years, voice AI has been stuck in a rigid loop: think, then speak. But real human conversation is messy, overlapping, and asynchronous. In our new #ICASSP2026 work, we built a tandem architecture

// Latent Agents // Multi-agent debate makes models reason better. It also burns tokens generating long transcripts before any answer comes out. This new research distills the entire debate into a sin