Search: “benchmark”

@GaryMarcus was right. It was the harness, not the model. That’s a much bigger deal (and hidden danger) than the benchmark score. If the real capability lives in opaque, vendor-controlled scaffolding

Other investors in the round include Benchmark, Bessemer, Kleiner Perkins, Greylock, Lightspeed, Altimeter, Bond Capital, Meritech, Atreides, Valor, T. Rowe Price, Lux, 8VC, D1, 137 Ventures, DST, Pos

This model represents a step-function improvement on many benchmarks, and its training is ongoing. Our internal model group arrived at the Navier–Stokes solution in 88 hours, using around 10,000 coord

arXiv:2609.04557v1 Announce Type: cross Abstract: Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under

arXiv:2602.11558v2 Announce Type: replace Abstract: Brain foundation models (BFMs) are advancing neurotechnology by learning transferable representations from neural signals, with broad potential in c

arXiv:2609.04859v1 Announce Type: new Abstract: As vision-language models (VLMs) rapidly advance in image understanding, cross-modal reasoning, and complex instruction execution, instruction-following

arXiv:2609.05395v1 Announce Type: new Abstract: Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across

arXiv:2609.04452v1 Announce Type: new Abstract: Modern misinformation is often heard before it is read, yet fact-checking systems are still evaluated mainly on clean written claims. Spoken dialogue re

We are excited to bring the first open benchmarking of Google's TPUs to the world Running every day, on many models + scenarios $/token is better than B200 and B300 Huge shout-out to Google @inferact

arXiv:2609.03673v1 Announce Type: new Abstract: Video generators build long videos by composing shorter parts, either by generating segments one after another or by autoregressively extending chunks.

arXiv:2609.03331v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly deployed in multi-turn settings where users may describe visual content with incorrect assumptions. Yet e

arXiv:2609.03588v1 Announce Type: new Abstract: As LLMs increasingly act through tools, they must reconcile user instructions, parametric knowledge, and dynamic environmental observations before takin

arXiv:2609.03467v1 Announce Type: cross Abstract: Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. Howev

OpenAI’s new language‑model system **Astra** has achieved state‑of‑the‑art performance on three key workflow benchmarks: the Agents’ Last Exam, AutomationBench, and ScreenSpot Pro. These suites evalua

arXiv:2609.02502v1 Announce Type: cross Abstract: Text-to-image (T2I) models have achieved remarkable success at faithfully rendering specified objects and attributes, yet their ability to produce vis

Meet E-Commerce Bench, a new benchmark for long-horizon autonomous business operations. 🚀 Agents start with ¥100,000 to run online stores for 365 days, handling sourcing, negotiation, pricing, promoti

'One remaining area of important capability we haven't seen demonstrated yet by any model is open-ended invention and discovery.' Benchmarks keep going up. Open-endedness keeps moving up in importance

Paper Instruments benchmarked sandbox providers for their RL rollouts, and E2B's tool execution came in up to 3x faster than every other sandbox provider they tested, reducing idle GPU time and traini

arXiv:2609.02140v1 Announce Type: cross Abstract: Encrypted traffic classification infers semantics beyond the flow record from transport-layer observables, and supervised training rests on labels tha

arXiv:2609.00866v1 Announce Type: cross Abstract: The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pat

arXiv:2603.11915v2 Announce Type: replace Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Multimoda

arXiv:2609.01161v1 Announce Type: new Abstract: Large language models can reproduce memorized text verbatim, yet copyright defenses are usually evaluated under incompatible protocols. We introduce Cop

arXiv:2609.00661v1 Announce Type: new Abstract: Satellite foundation models offer a globally available alternative to census data for commuting origin-destination (OD) generation, yet no study has sys

Google: Google launches Gemini 3.8 Flash Cyber for partners in its new Fairwind Program and says Gemini 3.8 Flash beats Claude Opus 5 and GPT-5.6 Sol on some benchmarks — Our newest Gemini models deli

arXiv:2609.01306v1 Announce Type: cross Abstract: Triangle-based neural rendering bridges neural scene representations and conventional graphics pipelines by optimizing explicit geometric primitives c

arXiv:2609.01036v1 Announce Type: cross Abstract: A lack of suitable datasets has limited the research into the privacy risks of novel smart city sensors, such as thermal cameras, depth cameras, and l

arXiv:2609.00918v1 Announce Type: new Abstract: Large language models are increasingly used as interactive recommender assistants. Their evaluation should therefore go beyond plausible item recommenda

We benchmark Qwen3.6-35B-A3B on an M5 Max MacBook Pro. Across ten prompt lengths and ten decode contexts, Lily was faster than MLX-LM, averaging 1.23× higher prefill throughput and 1.35× higher decode

arXiv:2608.30536v1 Announce Type: new Abstract: Reliable execution of long-horizon mobile manipulation tasks remains challenging because overall task success depends on the successful completion of mu

arXiv:2608.30418v1 Announce Type: cross Abstract: Resting-state functional magnetic resonance imaging (rs-fMRI) functional connectivity (FC) matrices are widely used for individual-level prediction, b

BenchMIRT is a new method for auditing LLM benchmarks question by question, revealing which capabilities they actually measure and helping researchers build smaller, more focused, and easier-to-interp

DeepSeek V4 Flash now comes with native vision on Fireworks. Available now on serverless and priority tiers. Better benchmarks than the text-only 0731 version at the same low price: 0.22 input / 0.66

arXiv:2608.29711v1 Announce Type: new Abstract: While MLLMs have made significant strides in chart comprehension and video understanding, current evaluations largely isolate these capabilities, leavin

arXiv:2608.28667v1 Announce Type: cross Abstract: The rapid proliferation of Large Language Models (LLMs) has raised concerns about their environmental impact during inference. While Green AI research

Introducing Fleet: GPU benchmarking entirely in your browser. Run WebGPU compute kernels drawn from real AI workloads directly on your hardware and earn a personalized card built for your device. Rate

arXiv:2608.29677v1 Announce Type: cross Abstract: Despite rapid advances in MIS, fair and reproducible comparisons of segmentation models remain challenging due to heterogeneous datasets, inconsistent

arXiv:2608.30903v1 Announce Type: new Abstract: Dynamic stance classification models how a reply responds to its direct parent message, rather than how a post relates to a fixed topic. Existing work h

arXiv:2608.29331v1 Announce Type: new Abstract: Video traffic constitutes a significant share of global web traffic. To reduce its volume, video codecs have been developed and continuously improved. W

arXiv:2608.29324v1 Announce Type: new Abstract: Annotation schemas are not neutral. When applied to classical literature, tag sets developed for modern journalistic texts impose source-culture definit

arXiv:2608.30820v1 Announce Type: new Abstract: Occlusion boundaries (OBs) are pixel-level image boundaries corresponding to surface visibility discontinuities caused by occlusion. Through precise bou

arXiv:2608.28607v1 Announce Type: new Abstract: Pharmaceutical sponsors developing a drug for both the United States and the European Union must reconcile guidance issued independently by the FDA and

arXiv:2608.30872v1 Announce Type: new Abstract: Objective assessment of surgical technical skill is important for surgical training and structured feedback, but current workflows remain dependent on l

glm-5.3 now beats gpt-5.6 sol and claude fable 5 on agentic benchmarks 5.3 flash is right behind it glm-5.3 didn’t even need a new base model to get there @zai_org kept the glm-5.2 base and scaled pos

arXiv:2608.28481v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have demonstrated strong capabilities in natural language understanding and mathematical reasoning. Howe

The official benchmark card for Ling-3.0-flash-Fin is a useful reminder that the unit being tested is rarely just “the model.” The release says most runs used temperature 1, top_p 0.95 and the highest

arXiv:2608.26992v1 Announce Type: new Abstract: Representation learning has attracted great atten- tion and managed to reach good performances as a pretraining method for downstream tasks or as a firs