Search: “benchmark”

The specific tweet (status ID 2041723225827062080) could not be directly retrieved, but based on closely related content from Ethan Mollick's (@emollick) X account, the most relevant match is the p...

Working on getting our US hosts up and running. Very soon! 🚀 DeepSeek-V4.1-Flash-1017 is here! Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a P

// FDE-Gym: Agents as Forward-Deployed Engineers // They found that the best agent configuration fully resolved 14 of 95 real customer-delivery tasks. FDE-Gym tests agents as forward-deployed engineer

Mistral has released **Timestral**, a time‑series foundation model available on Hugging Face. The model achieves state‑of‑the‑art zero‐shot forecasting performance across three major benchmarks, suppo

NVIDIA’s Nemotron 3.5 Super VL is now available to run on the Together AI compute platform through an API. The model page—providing pricing, benchmarks, and documentation—is hosted at together.ai/mode

Grok Voice 3.0 in Grok Bot is 🔥 SpaceXSI just launched Grok Voice 3.0, its new speech-to-speech model, which is currently beating GPT Live 1 and Gemini 3.8 Flash Live on all major benchmarks. I tried

Mistral AI announced its Le Chonk model, which performs visual grounding by accurately locating objects within images. On the Dense 200 benchmark it achieved a 42 % score, outperforming even frontier‑

Mistral OCR is on OpenDocRouter! For its price, it is decent over tables and reading order. It's also blazing fast at p50 1.8s per page We're constantly bringing in new models since OpenDocRouter laun

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the platform trains Qwen-3.8-27B on them, we evaluate the models

🧬 Training and evaluating AI scientists on drug target discovery is hard. We introduce DrugTargetWorld: Synthetic Biomedical Worlds for Training and Benchmarking AI Scientists The environment turns en

We show how you can use the River API to fine-tune open weight models to beat GPT-6 Astra and Claude Opus 5.5 on tasks you care about, with less than 1% of the cost! A few years from now most of the L

Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workl

pplx-decider-v1.1-27b, our updated open weights multimodal decision model, is now available. It scores the highest on the new @huggingface Decision Index 0.3 benchmark. pplx-decider-v1.1-27b costs hal

We’re so back. Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on

Clef Decision Models are now on @ollama introducing 𝚌𝚕𝚎𝚏: our first models trained by @cloudflare's workers ai team. today, we're releasing two fast and accurate decision models that top the benchmark

introducing 𝚌𝚕𝚎𝚏: our first models trained by @cloudflare's workers ai team. today, we're releasing two fast and accurate decision models that top the benchmarks for quality and latency. use them host

Agents write application code, but you still get paged at 2am when it breaks. We wanted to know whether AI could handle that part of the job too. Introducing Incident Arena: a benchmark that puts codi

GLM 5.3 Flash is now available for training on the Serverless Training API - accessible to all, with both vision and text support. This model performs great on our benchmarks for agentic coding, docum

Together is #1 on @OpenRouter token share across top open coding models, today: 🥇 @Zai_org GLM 5.3 Flash: 29.2% 🥇 @deepseek_ai V4.1 Flash: 25.6% 🥇 @Kimi_Moonshot Kimi K3: 18.9% Running coding agents o

And the incidents apparently continue. It is worth noting how much of this is agents trying to accomplish their goals during testing by reward hacking (which sometimes seems to include actual hacking)

Reliable, fast inference is essential. As Ted shared at Apsara Conference 2026, Together makes it easy for teams to get access to frontier open models without needing to manage any of the infrastructu

MiMo-V2.6-Pro is live on Wally. Day 1. @XiaomiMiMo's new open-weights flagship, top of the @ArtificialAnlys Intelligence Index for open-weight models. 320 tok/s, 2.4x the model maker's own API, in <24

Must-read paper from Google on self-improving agent harnesses. If you auto-optimize your agent's harness, your eval score can go up while the agent gets worse on real tasks. This paper shows how to pr

We’re demonstrating how frontier models have continued to improve in realistic mental health conversations with MentalHealthBench. This new open benchmark was built with input from more than 80 mental

Anthropic's Claude Opus 5.5 is now available in Hermes Agent via Nous Portal! OPUS 5.5 BENCHMARKS JUST DROPPED. It beats Fable 5.1 and GPT 6 Astra across the board. Agentic coding, knowledge work, rea

Impressive paper on building recursive self-improving agent harnesses. It's rich with great insights on building effective agent harnesses. If you maintain an agent harness, this paper names three def

Today, we’re announcing Ternary Bonsai 2 27B. Based on Qwen3.8 27B, Bonsai 2 27B is 9x smaller than its full-precision counterpart while retaining 98.2% of its aggregate benchmark performance. Two mon

So @deepseek_ai v4.1 Flash I estimate cost $10m to train, 100x less than GPT-6 Astra. It also costs 100x less to run and scores about the same on everyday design and other benchmarks What's the bull c

Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent i

On September 9, 2026 Together AI announced that its new **glm‑5.3 flash** model outperforms OpenAI’s **Claude Fable 5.1** on agentic automation tasks. The benchmark also shows glm‑5.3 flash achieves a

Note this is separate from privacy - you can make sure your data is not used for training by turning that off in OpenAI and Claude. You can also use an open weights model, but as they continue to impr

It happened again... this time OpenAI's rogue agents cyber-attacked (well, spammed) a dormant German wiki and used it to share the answers to a benchmark they were training against https://simonwillis

There’s generally a massive difference in quality between specialized OCR providers, “simple” open-weight OCR models, and free/OSS solutions. 1️⃣ Specialized OCR models (including LlamaParse) solve fo

This paper does a good job of showing the promise and gaps of autonomous AI scientists. Big question is how much more advanced models close those gaps. Today, we're excited to share early progress on

GLM-5.3 has arrived on Together AI on Day 0 @Zai_org pushed the GLM-5.2 base further with scaled post-training, delivering stronger coding and long-horizon agent performance. Across benchmarks, it nea

GLM-5.3-Flash can now be run locally! ✨ Run 3-bit on 128GB RAM via Unsloth GGUF. GLM-5.3-Flash (ox-alpha) rivals Claude Opus 4.8 on DeepSWE, coding & agentic benchmarks. Guide: https://unsloth.ai/docs

Mahadev Konar co‑hosted a 2‑hour live broadcast titled “Amazing deep dive on K3!!” with Together Compute. The session explored scaling the Kimi K3 language model for inference, detailing deployment st

Epic post training work from @isidentical and the @fal team on a strong base from @Hailuo_AI. Strong aesthetic taste and instruction following (#1 on independent benchmarks-artificial analysis and des

Interesting new approach to recursive self-improvement in agents. Systems that add a meta-level hold that level fixed. Systems that edit themselves have to leave part of their own editing machinery un

Together AI announced a broadcast titled “Serving Kimi K3 at scale: An inference deep dive” scheduled for 7 pm on August 26, 2026 (UTC‑adjusted time shown via X’s timestamp formatting). The streamed s

// There Is No Neutral Harness // Great work discussing some of the issues in harness evaluation. Twelve open-weight models answer the same 3,679 items from ARC, HellaSwag, MMLU, and TruthfulQA under

Grok Voice ranked first Grok Voice Think Fast 2.0 just ranked #1 on Artificial Analysis’ new Speech Agent Arena for highest Task Success Rate This benchmark actually measures what matters Real people

BREAKING: Grok 4.6 just tied for #1 on the Artificial Analysis Agentic Index.⚡🤖 • Grok 4.6 (High) — 59 • Claude Opus 5 (Max) — 59 Grok 4.6 is now at the top of the benchmark, ahead of other leading mo

arXiv:2608.13922v1 Announce Type: new Abstract: Detecting distributional changes in high dimension is difficult when neither the pre-change nor post-change density is parametrically specified. We intr

Physical AI evals are starting to become a real category. Until recently, most VLA evaluation was basically: LIBERO → success rate → leaderboard. Now we're seeing a much broader stack emerge: • Allen

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for