Search: “gemini”

The same model, with the same weights, scores 62% in one agent harness and 33% in another. @adithya_s_k and the @huggingface team just released the ultimate guide to multi-harness RL, and it's one of

Personal agents gone wrong. As we embrace more personal agents to carry out personalized tasks in the real world, interesting dynamics and behaviors will emerge. I think personal agents as they stand

Model personality matters in a way that benchmarks can't capture. Opus models went through a rough patch from around 4.7 to 5 where they just didn't feel 'Claude-y' anymore, more like an watered-down

Gavin Baker owned 15% of Nvidia and 10% of Tesla sub-2 billion. His biggest regret won't be either. It'll be the price you'll pay for SpaceX after the IPO. 'A company like this comes once in your care

Interesting new paper from Google DeepMind. Studies agents that follow bad advice from users. Users often suggest a fix that sounds right and is wrong. XYEval adds one confident, misleading hint to ta

Must-read paper from Google on self-improving agent harnesses. If you auto-optimize your agent's harness, your eval score can go up while the agent gets worse on real tasks. This paper shows how to pr

Big model release today - I wrote about Claude Opus 5.5, GPT-6 Sol, and GPT-6 Luna - plus comparison grids of pelicans by the different model families at different reasoning levels https://simonwillis

Grok 4.7 xHigh is now at the top with just ONE point away from Claude Fable 5.1 Max on Artificial Analysis’ AA-Briefcase benchmark Claude Fable 5.1 Max — 59% Grok 4.7 xHigh — 58% Just a 1-point differ

I am reminded of the fact that the infamous Milgram Experiment is actually quite weak because most people suspected the shocks were fake (kind of obvious in retrospect) & thus went along with shocking

Jev-style logit read on a 4B open VLM, measured: http://glance.yohei.me 👀 vs the same model writing JSON: ⏳ ~1/3 less time for a yes/no on a full photo 🤑 up to ~85% less GPU cost with many Qs per imag

LlamaIndex 发布了一个面向 AI Agent 的文档解析评测基准「ParseBench」 @llama_index 100 种解析方法参与评测（vlm - 67、专用解析器 - 33），2000 页真实企业文档（人工两轮校验），169011 条确定性规则，完全不用 LLM 当裁判，可复现。评测五个维度取无加权平均：表格结构、图表抽取、内容忠实度、格式语义、视觉定位。 https://ww

Weekend project, getting AEO-pilled. Also a good test (for me) of Claude Projects. Have been meaning to update Eleventy and do a few other redesign tasks on https://cybercultural.com, which tbh has be

🚀 Meet Qwen3.8-Omni-Flash, Qwen's first omni-modal model built around agentic capabilities! Native audio-video understanding, reasoning, and tool use come together in one model: understand the content

agreed, one of the biggest bottlenecks right now is my own ability to internalize the information that these models convey the latest models (astra, fable 5.1) are either way too verbose, or try too h

Another brilliant paper from Meta. This one is worth reading if you work on production-grade ranking or recommendation systems. Meta deployed an autonomous agent that runs the ML iteration cycle acros

This is a really neat benchmark. It's super interesting that Astra starts strong and holds the lead for up to 19 hours, but Fable 5.1 catches up in the final hours. Not entirely clear why. Different m

A couple years later, with GPT-6. https://underdark-duel.netlify.app Apparently the mindflayer has a '101-bone skeletal rig' (This is 5E 2014 rules, and I didn't see major errors, though the mind flay

Jensen Huang says that AGI has been achieved. I strongly disagree. GPT-6 performs really well on benchmarks, including ARC-AGI-3. But benchmark performance is not the same as general intelligence. GPT

Another neat one: I had GPT-6 Astra build a model of a famous never-built building, Boullée’s 1784 Centotaph. There were only a few black and white sketches to work with, along with a description. It

🦙 @ollama is used by 9 million developers and 85% of the Fortune 500, giving co-founder and CEO Jeffrey Morgan (@jmorgan) a unique view into which AI models people are actually using and how that’s ch

We've massively improved our document parsing capabilities across the board in the past ~3 months. Our LlamaParse cost-effective mode is one of the cheapest OCR solutions out there at <0.4c per page,

you can now build @LangChain agents w/ agentic video understanding! google is consistently at the forefront of multimodal processing; this is the video analog of 'agentic vision' token usage + cost is

Just WOW! Absolutely unacceptable. I look forward to understanding how that is possible. LLM tools get going and you can’t be 100% certain what they’re gonna do. Just like any LLM, you can ask coding

LlamaParse is officially a verified connector on Claude! 🦙 Claude is perfectly fine for the occasional text-only PDF. But most of the context your agent needs lives in messy PDFs, spreadsheets, scanne

People don't understand. You can create a rl env to teach it to find quarters. This will make money while you sleep. Media I DON’T THINK PEOPLE REALIZE HOW MASSIVE THIS IS YET @HuggingFace and @Pollen

Neat paper suggesting that human augmentation and task automation are not necessarily related. Models that are really good at doing work are not always good at helping humans do work better. Given the

The research on financial advice https://x.com/emollick/status/2085174123743842448?s=20 This paper by researchers from MIT and Stanford finds that most people would be financially better off if they f

GLM-5.3 (Max) by @Zai_org has shifted the Pareto frontier for Code Arena: WebDev! GLM-5.3 (Max) would sit at #2 among open models (upon open weights release, and #8 overall) with 1597 pts in the Code

GLM-5.3 achieves 60 on the Artificial Analysis Intelligence Index, on par with Kimi K3 and up 7 points from GLM-5.2. Once the weights are released it will be tied as the leading open weights model @Za

One prompt, one shot. Qwen3.8-27B creates something incredible. 😎Now it's your turn to run it on your own laptop. Can't wait to see what you come up with!👀@ItsmeAjayKV Here's a comparison i never thou

teams want to understand what their agents are doing but it’s hard to sift through all that trace data at scale but you can basically find any signal by fine-tuning a small, cheap model on data showin

Try Grok 4.6! Grok 4.7, which is a major upgrade, is coming soon. Grok 4.6 is up to 5× cheaper than GPT-5.6 Sol while delivering the exact same performance on this benchmark Both score 61 on Artificia

Grok 4.6 is smart, super fast & affordable Grok 4.6 just broke into the Top 3 on the Artificial Analysis Healthcare & Medical Index Grok 4.6 (high) scores 50 - just ONE point off the top score of 51 I

Every document extraction system has a perception blind spot. We mapped them. For ExtractBench, we tested 14 systems on documents that weren't born digital: 1950s regulatory filings, hand-filled tax f

⚡️A coalition that secures long-term AI capacity: We’re aggregating long-term compute demand in Europe to determine what capacity is built, where it’s located, and whom it serves. Through these multi-

Claude Haiku is my current least favorite model - it hallucinates wildly, and is out-performed now by other similarly priced models like GPT-5.6-Luna Even worse: it seems to still be used by the Claud

Alibaba’s Qwen team announced the new Qwen‑3.8‑Max model and thanked users for extensive testing, noting its ability to provide highly detailed world observations. A community member highlighted that

The tweet is a question from user Simon Willison (posted on 6 Aug 2026) asking which OpenAI API model corresponds to the ChatGPT “GPT‑5.6 Instant” version. No answer or clarification is included in th

We re-tested GPT-5.6 Luna from @OpenAI on ARC-AGI (Verified) following its recent 80% price reduction: - ARC-AGI-2: 59.6%, 0.18/task - ARC-AGI-1: 90.7%, 0.07/task The new results match Luna's original

Wake me when Astra solves a significant open-world problem that doesn’t revolve around formal verification. Or at least fixes poor @skdh’s video problems. I have tried many times to get ChatGPT, Claud

OK, GPT-5.6 Luna is a bit of a beast. Given the 80% price drop today I decided to try it in Datasette Agent, and it's furiously quick and generates all the SQL, HTML and JavaScript (for Datasette Apps

Great technical paper from Google. Great read on why context beats scale for agents working against unfamiliar APIs. (bookmark it) GPU kernel optimization has KernelBench to hillclimb on. TPUs had not

The Gemma series is an amazing set of highly performant open-weight models. They have proven extremely effective in industrial settings where site-deployed agents need exactly this as a base for domai

An interesting thing I'm observing from the blog/system card is that on a good chunk of the reported benchmarks (~20-30% from a skim), Opus 5 max thinking leads to a degradation in performance compare

We benchmarked Opus 5 comprehensively on document understanding through ParseBench. It is roughly on par with Opus 4.8 - it does a few points worse on dense tables, but does slightly better on parsing

Announcing OpenWorker! An open-source agent that doesn't just chat with you, but delivers finished work -- like hand you a polished document, send a slack message, or update a calendar entry. Ask it t