Search: “benchmark”

BREAKING: Anthropic just dropped Claude Fable 5—this is Mythos, made safe for public release. It is the best coding model in the world. We've been testing it internally @every for the last week or so

LiteParse is a parsing tool that demonstrates exceptionally fast performance, reportedly so rapid that Claude 3.5 Sonnet struggles to believe the benchmarked results are genuine. The post highlights L

Claude Fable 5 is compared against other AI models on various evaluation metrics through Cursor's benchmarking tool. The evaluation likely covers performance across different tasks such as coding, rea

Great tips. In practice, this is how it roughly looks to run agents autonomously for hours or days. /goal or /loop to keep it going. Verification is crucial here. Seeing a number of benchmarks showing

Cognition AI published detailed model results and technical implementation information on their blog, likely covering performance benchmarks, architecture specifics, and methodology for their frontier

The post appears to reference designer Vincent conducting frontier capability tests, likely related to AI model evaluation or benchmarking. The tone suggests this is commentary on ongoing AI developme

# Reality: The Final Eval This post argues that real-world task completion rate is the ultimate metric for evaluating AI systems, prioritizing actual production performance over benchmark scores like

AECbench is a public benchmark developed by Nomic AI that addresses the complex question of how to evaluate performance in the architecture, engineering, and construction (AEC) domain. The project rep

Our team is at CVPR 2026 if you want to come say hi :) We're presenting ParseBench at CVPR 2026! ParseBench is the most comprehensive document understanding benchmark for VLMs. ✅ It contains 2k pages

Quarq's team has developed an AI agent built on LangGraph that currently leads the LongMemEval-S benchmark, demonstrating advances in long-context memory evaluation. The work represents a notable achi

The Quarq agent is constructed using LangGraph, a framework that simplifies the development of complex memory systems. Quarq has achieved a top ranking on the LongMemEval benchmark, demonstrating the

BREAKING: Anthropic just dropped Opus 4.8—and it is a MONSTER We've been testing for about a week @every and our verdict is they could've just called it Opus 5, it's that good. Here's our vibe check:

Claude Opus 4.8 is now available as a model option in the Cursor code editor. According to Cursor's benchmarking, Opus 4.8 demonstrates improved efficiency compared to its predecessor Opus 4.7. The up

Announcing ESMFold2, our new state-of-the-art structure prediction model capable of predicting structure from single sequences or MSAs. ESMFold2 improves on benchmarks of protein-protein interaction a

An interesting work on Physical AI: PhysX-Omni. First unified sim-ready generation framework for rigid, deformable, and articulated objects, with a diverse dataset and new benchmark. 🌐 https://physx-o

Shipper’s “After Automation” is the best framing I’ve read of why AI keeps generating more work, not less. The line that stuck with me: the frame is not the framer. Models climb whatever benchmark we

brain dump of how/why we use Evals to measure agents before & after shipping to prod 1. Good Evals simulate what our real users will do and encounter. They’re not really random benchmark tasks, they r

Read more on what we've learned while developing long-horizon agent evals: Curious finding while creating evals and benchmarks for long-horizon (100+ turn) agents While it’s generally thought that a d

GLM-5.1 from @Zai_org is now live on OrcaRouter • #1 open-source model on SWE-Bench Pro • Beats closed source models on real-world repo repair benchmarks • MIT licensed • 200K context • Built for long

EBM are so back! @ylecun has been pointing here for years: AI reasoning needs systems that check structure before they answer. Aleph from @logic_int now leads the major formal reasoning benchmarks – l

Grok Voice Think Fast 1.0 is officially the most well-rounded agentic voice AI on the market right now It now ranks #1 in the latest τ-Voice agentic performance benchmarks in real-world tests on Artif

我們開源了這顆星球🌎上速度最快的低成本 bm25 引擎。 so we built psql_bm25s. exact BM25 retrieval. native Postgres access method. ~23x faster than pg_search on the standard benchmark. retrieval stops being a budget item. the

Grok Voice is #1! Announcing agentic performance benchmarking for Speech to Speech models on Artificial Analysis. We use 𝜏-Voice to measure tool calling and customer interaction voice agent capabiliti

Meet physics-intern🧑‍🎓, our agentic framework for theoretical physics. It takes Gemini 3.1 Pro from 17.7% to 31.4% on CritPt, a new SOTA on one of the hardest benchmarks for LLMs. Theoretical physics

Try Grok Voice Grok Voice Think Fast 1.0 ranks #1 on the Artificial Analysis τ-Voice benchmark for real-world agentic customer service resolution Absolutely outperforming GPT-Realtime-2 (High) and Gem

Jarred tried rewriting Bun in Rust and it passes 99.8% of the existing test suite we're not being ambitious enough there will be a blog post about this. on what this means for bun, benchmarks, memory

That doesn't mean that there should not be regulation and vetting, but it does suggest that is hard to write a criteria right now that is not somewhat vague. More R&D into non-lab benchmarks is urgent

Francois Chollet shared a blog post analyzing failure modes of frontier AI models, specifically examining performance on the ARC (Abstraction and Reasoning Corpus) AGI benchmark with models including

AI21 Labs achieved state-of-the-art performance on the BrowseComp-Plus benchmark with 95.18% accuracy using their AI21 Maestro model with agent optimization techniques. The post appears to discuss aut

I cannot verify the authenticity of this post or its specific claims about Grok 4.3's performance metrics. The source appears to reference xAI's Grok model improving significantly on benchmark leaderb

DeepSeek V4 Pro is crazy good at bug fixing. Costs counted in cents not dollars/tens of dollars. It’s the next level quiet confidence not to get caught up by benchmarks & rankings, and just let us use

AI21 Labs announced that their Chief Product Officer Or Dagan presented at the DeepLearning.AI conference, discussing how the company achieved state-of-the-art results on the Browsecomp-Plus benchmark

Want to see which frontier models do the best on document understanding? Check out our ParseBench leaderboard on @kaggle! https://www.kaggle.com/benchmarks/llamaindex-org/parsebench For more details o

The only way to fully appreciate the jaggedness of the AI frontier is up close. When you use it for a task you know well you find tons of tiny points where AI requires human help. Some are tedious (mo

More of my notes on DeepSeek V4 - the really big news is the pricing: both DeepSeek-V4-Flash and DeepSeek-V4-Pro are the cheapest models in their categories while benchmarking close to the frontier mo

CursorBench is a benchmarking tool or evaluation framework developed by Cursor, likely designed to measure and compare the performance of AI code completion or development features. The post provides

GPT-5.5 achieved state-of-the-art performance on the ARC-AGI-2 benchmark, with scores ranging from 85.0% on maximum difficulty tasks to 33% on low difficulty tasks. The model demonstrated consistent i

Kimi K2.6 achieved the top ranking on MathArena, a benchmark for evaluating mathematical problem-solving capabilities in open-source language models. This announcement highlights the model's superior

⚡️⚡️Run Qwen3.6-27B locally! @UnslothAI Qwen3.6-27B can now run locally! 💜 Run on 18GB RAM via Unsloth Dynamic GGUFs. Qwen3.6-27B surpasses Qwen3.5-397B-A17B on all major coding benchmarks. GGUFs: htt

LM Performance：With only 27B parameters, Qwen3.6-27B outperforms the Qwen3.5-397B-A17B (397B total / 17B active, ~15x larger!) on every major coding benchmark — including SWE-bench Verified (77.2 vs.

ml-intern by @huggingface is wild 🔥 You drop a high-level prompt (“build the best scientific reasoning model” or “crush healthcare benchmarks”) and this open-source agent does the entire post-training

Today we’re introducing two big steps for health at OpenAI: - ChatGPT for Clinicians, a free version of ChatGPT designed for clinical work - HealthBench Professional, a new benchmark to evaluate real

HUGGING FACE JUST AUTOMATED THEIR ENTIRE POST-TRAINING TEAM WITH AN AGENT. It reads papers, runs GPU experiments, iterates, and builds research-backed models autonomously. Pushed a benchmark from 10%

life when you discover an open-source model that runs 300 parallel agents, executes for 12+ hours straight, beats GPT-5.4 and opus 4.6 on multiple benchmarks... and the weights are on huggingface Medi

🤯QWEN 3.6 35B-A3B IS INSANE AT CODING @unslothai recently dropped Qwen3.6-35B-A3B-GGUF The first open Qwen3.6 model and it’s solid. This MoE beast is cooking on benchmarks: 🧠SWE-bench Verified: 73.4 (

This post announces Ornstein3.6-35B-A3B-GGU, a fine-tuned variant of Qwen 3.6 35B model available on Hugging Face, with the author claiming it represents a high-quality optimization of the base model.