Search: “benchmark”

Nous Research announced significant developments or milestones for Hermes Agent, their AI agent framework, indicating multiple new capabilities, updates, or releases during that particular week. The a

Learn more about the latest from @james_y_zou and our Frontier Agents Research team! To evaluate frontier AI agents, we need more complex tasks. But such tasks are also more prone to have design mista

The efficiency frontier! Where do you think GPT-5.6 will land? Claude Opus 4.8 has landed on DeepSWE Bench, posting a 58% Pass@1 and taking #2 overall behind GPT-5.5. It continues a broader trend: sli

I tried the liteparse's web browser version today to convert a couple of PDF to text and was shocked at the speed. I had to recheck twice to see whether it even did the complete processing or not 😅 ht

I'm proud to share that @Glean has surpassed 300M ARR, just five months after crossing 200M and growing ~3x over the past 15 months. This is an exciting milestone for Glean, and it's a signal about wh

just noticed today - the dataset is already past 1k+ downloads. opensource / openresearch ftw ! @evo__hq would be opensourcing as many datasets, evals and autoresearch runs as we can in our pursuit of

Stronger models do not always need lighter harnesses. Everyone believes more structured harnesses universally improve reliability, and that higher-capability models need proportionally less structural

Super happy that a bunch of people are finding marlin useful. Thank you for the inference support @ZeroGPU_AI @huggingface 🥰🤝 We’ve got hands on more compute now so we’ll also release a series of blog

Forecasting Scientific Progress with Artificial Intelligence https://arxiv.org/abs/2605.22681 Turns out AI is just as bad at forecasting biology and physics breakthroughs as we are. To be fair, most b

Neurosymbolic AI is saving deep learning from hitting the wall (*exactly* as I said my 2022 paper “deep learning is hitting a wall”) Really sad to see someone as smart @peterwildeford confusing the or

1. Agreed w @scaling01 that Mythos appears to be better GPT 5.5 on many metrics. 2. Mythos is definitely a major wakeup call wrt security, and will pose problems for real-world systems that aren’t wel

this has been my experience as well. there definitely were improvements, specifically wrt shell based computer use, but also regressions, especially in the last 3 version bumps of flicker and gerperte

I truly miss the age of science and transparency in AI. Especially given how much money and political power and governance and scientific understanding is at stake. We don’t know for example • How man

Cursor's new Composer 2.5 takes third on the Artificial Analysis Coding Agent Index and is ~10-60x lower cost than the higher-effort Opus 4.7 and GPT-5.5 variants above it. This release puts Composer

Google’s new Gemini 3.5 Flash is the clear leader on the Intelligence vs Speed Pareto frontier and makes large gains on GDPval-AA (real-world agentic tasks), but is 5x the cost of Gemini 3 Flash @Goog

this has probably been the most interesting project i've ever worked on. we created the best agent at finding problems with your agent. and we integrated it with the best observability platform to spi

Today we release Contrastive Neuron Attribution (CNA), a method for steering LLM behavior by identifying and ablating sparse circuits in the MLP basis without training a sparse autoencoder, modifying

This post appears to be a congratulatory message acknowledging exceptional work or contributions from members of the AI engineering community (@aiDotEngineer, @agrimsingh, and @swyx), suggesting they

20 years ago, my first startup was all about enterprise search. Two decades later, we’re still building search engines. The technology has shifted from NLP to NN and the users from humans to agents. b

A new set of open-weight models is topping the leaderboard for document understanding 🔥 INF just released two models: Infinity-Parser2-Pro (35B) and Infinity-Parser2-Flash (2B) that top our @huggingfa

Today we release Lighthouse Attention, a selection-based hierarchical attention for long-context pre-training that delivers a 1.4-1.7× wall-clock speedup at 98K context. It runs the same forward+backw

Are scaling laws finally working for time series foundation models? Today, @datadoghq is releasing Toto 2.0 weights in Apache 2.0 on @huggingface. It's a family of open-weights TSFMs from 4M to 2.5B p

// Harnessing Agentic Evolution // Pay attention to this one if you run iterative agentic search loops. (bookmark it) AEvo splits the self-improvement loop into two jobs: > One proposes the next candi

A lot of people have been wondering about Mythos, Glasswing, and the vulns we / our partners are fixing. Today, I’m excited for us to start sharing more. (For context, I lead Glasswing @AnthropicAI.)

clarifying how much “task horizon” falls off as a function of the increasing accuracy criterion directly in the graph (not across tabs) would definitely improve this graph, @METR_Evals. as a variant o

// δ-mem: Efficient Online Memory for LLMs // One of the more elegant memory mechanisms I've seen this month. Most long-term memory work either inflates context or retrains the model. This paper shows

Llama-eval is a proposed evaluation tool for llama.cpp designed to standardize and improve comparability of community-run evaluations. The tool aims to address inconsistencies in how different users b

When you want to move from single agent SQLite on something like QMD, PostgreSQL is a great choice for multi agent and production quality, but not as snappy. So we made it much more snappy with BM25 &

gpt-realtime-2 is a great voice model (with a typically bad OpenAI name). Voice models are natively processing speech, not transcribing it, so the intelligence of the model matters. The old voice mode

Cool paper from PwC. 'Earlier is always better' is the default intuition for agent clarification. New paper claims that's mostly wrong. Goal clarification loses nearly all of its value after just 10%

One of the most important properties of LLMs that we take for granted is that newer, bigger models are just better at everything. The AI Labs are pouring effort into economically valuable fields like

ERNIE 5.1 is here 🚀 ERNIE 5.1 significantly reduces pretraining cost while compressing total parameters to ~1/3 and activated parameters to ~1/2 — using only ~6% of the pretraining cost compared to mo

Hot take on METR’s new graph that so many people are flipping about today. • Claude Code is a real advance; Mythos probably builds on some of what is learned there. But… • If you read the graph carefu

This is confused, but popular. Popular because it tells a bunch of people what they want to hear. Confused for a couple reasons: first, Mythos probably isn’t a pure LLM. (Claude Code isn’t, and it pro

Yann LeCun closed 1.03B for AMI Labs on March 10. Three days later, this paper dropped from his NYU collaborators. 15M parameters. Single GPU. A few hours of training. LeWorldModel is the first JEPA t

Hermes is an AI model developed by Nous Research that focuses on instruction-following and reasoning capabilities. Based on Nous Research's focus, this likely covers the model's architecture, performa

Nous Research announced improvements or updates to their Hermes Agent system, highlighting enhanced capabilities or performance. The post was shared on X (formerly Twitter), suggesting it covers recen

Multi-Token Prediction (MTP) for LLaMA.cpp! Running Gemma4 local model 1.5x faster. We patched LLaMA.cpp. Quantized Gemma 4 assistant models into GGUF format. We ran tests on a MacBook Pro M5Max. Gemm

Last week I gave a talk at AI Dev ’26 by @DeepLearningAI on “AI can’t read PDFs, how do we fix it” . I’m sharing the slides publicly if others are interested in doing a deep dive into document underst

NEW paper from Microsoft Research. (bookmark it) The entire interpretability literature is built around human readers. As more analysis gets delegated to agents, the right target of interpretability s

The last sentence in this abstract is really important, in a way that professional programmers will immediately recognize: the models favored big single files rather than breaking things into modules.

Some things never change. If you don’t understand this one, you don’t understand what’s happening AI. Marcus, 1998: neural nets have trouble generalizing far beyond the data. Marcus, 2001, 2012, 2019,

🚀Today we ship @FlyMy_AI Agents. The world's first all-in-one agentic cloud. The modern way to build, integrate, and scale production AI agents. 3 steps to a production agent: 1. Connect your work too

(Sorry, after seeing so many of these, could not resist): 🚨 BREAKING: Google just dropped a NEW paper that completely deletes RNNs from existence. No recurrence. No convolutions. Nothing. Just one mec

I cannot verify the authenticity of this post, as the URL structure and timestamp appear inconsistent with actual X (Twitter) posts. The claim about fine-tuning a '1930 vintage LLM' is anachronistic,

This post from Swyx discusses Grok (xAI's AI assistant) in the context of efficiency, likely highlighting performance metrics, speed improvements, or resource optimization of the model compared to alt