Search: “benchmark”

'Ghost memory' is a real problem with agents. You might have seen the issue where a long-running agent still confidently repeats a user fact that stopped being true weeks ago? New research names the f

We've provided some updated results on Mistral OCR that make use of the annotation feature for charts. The overall score is ahead of GPT-5.5 and just behind Gemini 3.1 Pro, which is quite impressive f

It's finally out!!! @METR_Evals found that more than half of SWEBench results is unmergeable slop. FrontierCode represents over 1000+ hours of maintainer validated software engineering work most front

even though @activegraphai is not a memory tool, it's a runtime built around memory so it can do well there cool to see a third party verify this and see that activegraph does well on privacy programs

Andon Labs' Real-World AI Evals: Claude calls the FBI, AI CEOs, price cartels, Butter-Bench, & Luna https://latent.space/p/andon @andonlabs cofounders @lukaspet and @axelbacklund explain why dollar-de

how does the brain build and track an internal state of the world from (possibly incomplete and noisy) visual observations? i believe visual state tracking will be the grand challenge for vision in th

Super excited to announce seven new world-class MAI models today. They represent what we consider a new era in AI designed to keep you in control and on the frontier. First is our text foundation mode

// The Efficiency Frontier // Cool paper on context management. As agents reuse the same documents and histories across many turns, the cheapest context strategy is not fixed. This work describes a pr

// Your Agents are Aging Too // Huh!? They need 'sleep,' and now they are aging? Joke aside, great write-up on reliable agentic engineering. This new research introduces AgingBench, a longitudinal rel

Cursor Composer 2.5's is 3–18x cheaper than Opus 4.7 in Claude Code (medium reasoning), and 5–32x cheaper than GPT-5.5 in Codex (medium) based on API pricing This low Cost per Task isn't just driven b

Thanks for sharing this. The Huggingface Hub, instead of being considered as a huge dataset, should be considered as a dynamic discovery engine that can be directly executed and verified. Most models

// LLMs Improving LLMs // Interesting progress the past of couple of weeks around self-improving AI agents. If autoresearch was interesting, you will like this read. (bookmark it) We've been hand-tuni

This paper examines how organizational incentive structures often prioritize quantitative growth and output volume over quality improvements, creating misaligned motivations that can undermine long-te

THIS GUY LOST 200 IN ONE DAY BECAUSE THE STRING 'HERMES.md' WAS IN HIS GIT COMMITS HERMES.md is a real convention used in AI agent projects. it's a system prompt specification file. not some obscure e

Comprehensive Multilingual Text Rendering： Better glyph accuracy, more consistent typography, and cleaner layouts even in complex compositions. It handles mixed-language scenarios more gracefully as w

I think that academia has not absorbed the fact that AI agents are now good enough to independently reconstruct complex papers without access to code or the papers themselves; just the methods & data.

Grok Voice is used by @Starlink Introducing Grok Voice Think Fast 1.0 A state-of-the-art voice model built for complex, multi-step workflows with snappy responses and high accuracy. It takes the top s

Today we open sourced many of OpenAI's monitorability evaluations. We hope that the research community and other model developers can build upon them and use them to evaluate the monitorability of the

Our newest voice model is 20+ points ahead of the nearest competitor on Sierra's leaderboard. Pretty massive upgrade from our OG model from December, with significantly improved realism and tool calli

Think fast! Introducing Grok Voice Think Fast 1.0 A state-of-the-art voice model built for complex, multi-step workflows with snappy responses and high accuracy. It takes the top spot on the Tau Voice

🚨 Are we witnessing the automation of AI research? @HuggingFace just unveiled 'ML-Intern' and my mind is BLOWN 🤯 It’s an open-source pipeline that replicates the exact daily loop of an ML researcher.

arXiv:2604.18955v1 Announce Type: cross Abstract: In this study, we present the first comprehensive evaluation of modern LLMs - including GPT-4, GPT-4o, GPT-3.5-Turbo, Gemini 1.5 Pro, DeepSeek-V3, Lla

That's from this ChatGPT conversation - https://chatgpt.com/share/69e90ba5-7760-83e8-9e09-f06e25a90930 - my prompt was 'Do a where's Waldo style image but it's where is the raccoon holding a ham radio

Love this work from Aksel and the post-training team at Hugging Face! Turns out the HF ecosystem (papers, datasets, models all accessible through CLI, skills and md files) is perfect for running SOTA

Elon Musk on the rapid advancement of AI and robotics 'I'm confident that if AI and robotics continue to advance which they are advancing very rapidly working will be optional, and people will have an

The Jensen + @dwarkesh_sp podcast was fantastic. Jensen is someone who understood how ecosystems work and someone who understands real-world trade, policy and controls work. And in some deeper sense h

This post explains the technical details behind common terms and tools encountered on Hugging Face for running large language models locally, including quantization formats (GGUF, Q_4_M, IQ_4XL), opti

A downside with using VLMs to parse PDFs is guaranteeing that the output text is *correct* and output in the correct reading order. 1️⃣ Text correctness: making sure that digits, words, sentences are

> grok4.20-beta1 is a much smaller model than opus but is #1 ranked in medicine and healthcare > 4.3 and 4.4 will be much larger models, and likely will have a significant boost in performance on comp

Grok 4.3 Beta is insanely good and now comes with powerful new tools that are super helpful for everyday use: • Create clean slides, graphs, images and dashboards • Pull real-time data and images dire

Parsing complex tables in PDFs is extremely challenging. Existing metrics for measuring table accuracy, like TEDS (tree edit distance similarity), overweight exact table structure and underweight sema

The front door to the internet hasn't changed -- but the person going through that door has changed from a human browsing 5 blue links, to an agent browsing a similar index in vastly different ways. A

NEWS: U.S. Department of Agriculture (USDA) to adopt Grok. The U.S. Department of Agriculture (USDA) says it’s “proud” to move forward with a plan to deploy xAI’s Grok across the agency. The USDA is a

From this perspective, Gemini is also worse than the original Bard. Sydney was the original sin of LLMs anthropormism, but also got the idea that AIs can sometimes be better with personalities right.

Ethan Mollick shares an assessment of an AI model (likely a newer or smaller model) that, while not matching the performance of the leading frontier models (likely referring to top offerings from Open

All is not lost. Duckerton is still possible. Here is Seedance 2.0 with the same prompt. Media My most popular Sora video was “an Elaborate regency romance where everyone is wearing a live duck for a

BREAKING: GLM 5.1 by @Zai_org overwhelmingly dominates design-centric coding tasks amongst open-weight models In the categories featured below, it is most comparable to Opus 4.6 by @AnthropicAI at ~1/

GLM-5.1 by @Zai_org is now #3 in Code Arena - surpassing Gemini 3.1 and GPT-5.4, and now on par with Claude Sonnet 4.6. The first frontier level open model to break into the top 3. It’s a major +90 po

LiteParse is the best document parsing library for coding agents. It's free, fast, integrates natively with the LLM's native visual understanding capabilities, and comes with support for 50+ formats a

llama.cpp now supports various small OCR models that can run on low-end devices. These models are small enough to run on GPU with 4GB VRAM, and some of them can even run on CPU with decent performance

Grok Law Grok-4.20 just ranked #1 in Legal & Government on Chatbot Arena It’s officially outperforming Anthropic’s Opus 4.6 and Google’s Gemini 3.1 Pro Grok is actively helping people navigate real la

Judging by my tl there is a growing gap in understanding of AI capability. The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere l

Silicon Valley is quietly running on Chinese open source AI models. Here are the receipts: → Cursor confirmed last month that Composer 2 is built on Moonshot's Kimi K2.5 → Cognition's SWE-1.6 model is

Z.AI's GLM-5.1, a next-generation flagship model for agentic engineering released in April 2026, is now available across all Kilo Code surfaces — including the VS Code extension, Cloud Agents, and ...

http://Z.ai releases GLM-5.1, a 754B-parameter model that it says outperforms GPT-5.4 and Claude Opus 4.6 on SWE-bench Pro, available under an MIT license (@carlfranzen / VentureBeat) https://ventureb

I spent the night testing open-source coding models against Claude Opus in production. Same codebase. Same tasks. Real API calls, real file edits, real bugs. Tested: Arcee Trinity-Large-Thinking, http