Model Releases
The benchmark for non-verifiable domains is often the opinions of humans. That is how we determine whether writing or an idea or a pitch is …
The benchmark for non-verifiable domains is often the opinions of humans. That is how we determine whether writing or an idea or a pitch is good in the real world And we know how to measure & benchmar
The benchmark for non-verifiable domains is often the opinions of humans. That is how we determine whether writing or an idea or a pitch is good in the real world And we know how to measure & benchmark this stuff: time for AI folks to read up on qualitative research methodology!
Related
- I find that open weights models over-perform on benchmarks compared to actual real-world usage, and Kimi feels like no exception. For exampl…
- Basically every remaining good AI benchmark score has an implied asterisk next to it which reads: * could be signficantly higher with a bett…
- The metrics discussion at OpenAI is a little confusing to me. I appreciate the clarification about bad benchmarks, but they spent a lot of m…
- I think Artificial Analysis does a good job overall and provides transparency in benchmarking, but GDPval-AA is not a good benchmark and nee…
- And now a new DeepSeek model, and appears to be fully open weights. Good benchmarks, but with open models, that isn't always as meaningful. …
Source: Ethan Mollick (X) | 2026-08-16