Knowledge catalogue
Another informal nerd benchmark that AIs had struggled with has been passed: Astra successfully designed a Magic the Gathering deck and then used it to defeat a bot on Arena. (the deck may not be stun
Benchmark two 30B Mixture-of-Experts models, Qwen3-Coder-30B and NVIDIA Nemotron-3-Nano-30B, across G5, G6, G6e, and G7 GPU instances on Amazon SageMaker AI. Compare throughput, latency, and cost-per-
@GaryMarcus was right. It was the harness, not the model. That’s a much bigger deal (and hidden danger) than the benchmark score. If the real capability lives in opaque, vendor-controlled scaffolding
Other investors in the round include Benchmark, Bessemer, Kleiner Perkins, Greylock, Lightspeed, Altimeter, Bond Capital, Meritech, Atreides, Valor, T. Rowe Price, Lux, 8VC, D1, 137 Ventures, DST, Pos
This model represents a step-function improvement on many benchmarks, and its training is ongoing. Our internal model group arrived at the Navier–Stokes solution in 88 hours, using around 10,000 coord
arXiv:2609.04557v1 Announce Type: cross Abstract: Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under
arXiv:2602.11558v2 Announce Type: replace Abstract: Brain foundation models (BFMs) are advancing neurotechnology by learning transferable representations from neural signals, with broad potential in c
arXiv:2609.04859v1 Announce Type: new Abstract: As vision-language models (VLMs) rapidly advance in image understanding, cross-modal reasoning, and complex instruction execution, instruction-following
arXiv:2609.05395v1 Announce Type: new Abstract: Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across
arXiv:2609.04452v1 Announce Type: new Abstract: Modern misinformation is often heard before it is read, yet fact-checking systems are still evaluated mainly on clean written claims. Spoken dialogue re
We are excited to bring the first open benchmarking of Google's TPUs to the world Running every day, on many models + scenarios $/token is better than B200 and B300 Huge shout-out to Google @inferact
arXiv:2609.03673v1 Announce Type: new Abstract: Video generators build long videos by composing shorter parts, either by generating segments one after another or by autoregressively extending chunks.
arXiv:2609.03331v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly deployed in multi-turn settings where users may describe visual content with incorrect assumptions. Yet e
arXiv:2609.03588v1 Announce Type: new Abstract: As LLMs increasingly act through tools, they must reconcile user instructions, parametric knowledge, and dynamic environmental observations before takin
arXiv:2609.03467v1 Announce Type: cross Abstract: Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. Howev
OpenAI’s new language‑model system **Astra** has achieved state‑of‑the‑art performance on three key workflow benchmarks: the Agents’ Last Exam, AutomationBench, and ScreenSpot Pro. These suites evalua
arXiv:2609.02502v1 Announce Type: cross Abstract: Text-to-image (T2I) models have achieved remarkable success at faithfully rendering specified objects and attributes, yet their ability to produce vis
Meet E-Commerce Bench, a new benchmark for long-horizon autonomous business operations. 🚀 Agents start with ¥100,000 to run online stores for 365 days, handling sourcing, negotiation, pricing, promoti
'One remaining area of important capability we haven't seen demonstrated yet by any model is open-ended invention and discovery.' Benchmarks keep going up. Open-endedness keeps moving up in importance
Paper Instruments benchmarked sandbox providers for their RL rollouts, and E2B's tool execution came in up to 3x faster than every other sandbox provider they tested, reducing idle GPU time and traini
arXiv:2609.02140v1 Announce Type: cross Abstract: Encrypted traffic classification infers semantics beyond the flow record from transport-layer observables, and supervised training rests on labels tha
arXiv:2609.00866v1 Announce Type: cross Abstract: The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pat
arXiv:2603.11915v2 Announce Type: replace Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Multimoda
arXiv:2609.01161v1 Announce Type: new Abstract: Large language models can reproduce memorized text verbatim, yet copyright defenses are usually evaluated under incompatible protocols. We introduce Cop
arXiv:2609.00661v1 Announce Type: new Abstract: Satellite foundation models offer a globally available alternative to census data for commuting origin-destination (OD) generation, yet no study has sys
Google: Google launches Gemini 3.8 Flash Cyber for partners in its new Fairwind Program and says Gemini 3.8 Flash beats Claude Opus 5 and GPT-5.6 Sol on some benchmarks — Our newest Gemini models deli
arXiv:2609.01306v1 Announce Type: cross Abstract: Triangle-based neural rendering bridges neural scene representations and conventional graphics pipelines by optimizing explicit geometric primitives c
arXiv:2609.01036v1 Announce Type: cross Abstract: A lack of suitable datasets has limited the research into the privacy risks of novel smart city sensors, such as thermal cameras, depth cameras, and l
arXiv:2609.00918v1 Announce Type: new Abstract: Large language models are increasingly used as interactive recommender assistants. Their evaluation should therefore go beyond plausible item recommenda
We benchmark Qwen3.6-35B-A3B on an M5 Max MacBook Pro. Across ten prompt lengths and ten decode contexts, Lily was faster than MLX-LM, averaging 1.23× higher prefill throughput and 1.35× higher decode
arXiv:2608.30536v1 Announce Type: new Abstract: Reliable execution of long-horizon mobile manipulation tasks remains challenging because overall task success depends on the successful completion of mu
arXiv:2608.30418v1 Announce Type: cross Abstract: Resting-state functional magnetic resonance imaging (rs-fMRI) functional connectivity (FC) matrices are widely used for individual-level prediction, b
BenchMIRT is a new method for auditing LLM benchmarks question by question, revealing which capabilities they actually measure and helping researchers build smaller, more focused, and easier-to-interp
DeepSeek V4 Flash now comes with native vision on Fireworks. Available now on serverless and priority tiers. Better benchmarks than the text-only 0731 version at the same low price: 0.22 input / 0.66
arXiv:2608.29711v1 Announce Type: new Abstract: While MLLMs have made significant strides in chart comprehension and video understanding, current evaluations largely isolate these capabilities, leavin
arXiv:2608.28667v1 Announce Type: cross Abstract: The rapid proliferation of Large Language Models (LLMs) has raised concerns about their environmental impact during inference. While Green AI research
Introducing Fleet: GPU benchmarking entirely in your browser. Run WebGPU compute kernels drawn from real AI workloads directly on your hardware and earn a personalized card built for your device. Rate
arXiv:2608.29677v1 Announce Type: cross Abstract: Despite rapid advances in MIS, fair and reproducible comparisons of segmentation models remain challenging due to heterogeneous datasets, inconsistent
arXiv:2608.30903v1 Announce Type: new Abstract: Dynamic stance classification models how a reply responds to its direct parent message, rather than how a post relates to a fixed topic. Existing work h
arXiv:2608.29331v1 Announce Type: new Abstract: Video traffic constitutes a significant share of global web traffic. To reduce its volume, video codecs have been developed and continuously improved. W
arXiv:2608.29324v1 Announce Type: new Abstract: Annotation schemas are not neutral. When applied to classical literature, tag sets developed for modern journalistic texts impose source-culture definit
arXiv:2608.30820v1 Announce Type: new Abstract: Occlusion boundaries (OBs) are pixel-level image boundaries corresponding to surface visibility discontinuities caused by occlusion. Through precise bou
arXiv:2608.28607v1 Announce Type: new Abstract: Pharmaceutical sponsors developing a drug for both the United States and the European Union must reconcile guidance issued independently by the FDA and
arXiv:2608.30872v1 Announce Type: new Abstract: Objective assessment of surgical technical skill is important for surgical training and structured feedback, but current workflows remain dependent on l
glm-5.3 now beats gpt-5.6 sol and claude fable 5 on agentic benchmarks 5.3 flash is right behind it glm-5.3 didn’t even need a new base model to get there @zai_org kept the glm-5.2 base and scaled pos
arXiv:2608.28481v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have demonstrated strong capabilities in natural language understanding and mathematical reasoning. Howe
The official benchmark card for Ling-3.0-flash-Fin is a useful reminder that the unit being tested is rarely just “the model.” The release says most runs used temperature 1, top_p 0.95 and the highest
arXiv:2608.26992v1 Announce Type: new Abstract: Representation learning has attracted great atten- tion and managed to reach good performances as a pretraining method for downstream tasks or as a firs
arXiv:2608.26332v1 Announce Type: new Abstract: Managed LLM services are now part of real production systems, but model selection and service planning still rely heavily on capability benchmarks that
arXiv:2608.27334v1 Announce Type: new Abstract: Industrial sites contain large volumes of read-only telemetry, but few benchmarks specify how to compile these records into executable multi-turn agent
arXiv:2608.27442v1 Announce Type: cross Abstract: In real-world software development, code review typically involves iterative interactions between developers and reviewers to improve software quality
arXiv:2603.03230v2 Announce Type: replace-cross Abstract: The electric vehicle routing problem with time windows (EVRPTW) extends the classical VRPTW by introducing battery capacity constraints and ch
arXiv:2608.26613v1 Announce Type: new Abstract: This study provides a comprehensive benchmarking of conventional machine learning (ML), ensemble learning, deep neural networks, recurrent architectures
arXiv:2608.25520v1 Announce Type: new Abstract: Audio-visual cross-modal Fine-Grained Visual Categorization (FGVC) aims to identify fine-grained categories by jointly leveraging visual and auditory in
Hi all, I'm looking for the best model for a hobby project and trying to make sense of the various data I came across. I know benchmarks do not often translate to the real world, especially to your pa
No reward hacking was found in GLM-5.2: solve the task, not the benchmark. Introducing reward hacking score corrections to the Artificial Analysis Coding Agent Index In v1.4 of the Artificial Analysis
Hello, I've been self-hosting LLMs on various budget hardware for a while (6x RTX 3060 12 GB, Intel Arc Pro B60 24 GB, RX 9070 XT, etc). Over the last few months, I wrote about it in 4 articles: Gener
arXiv:2608.23857v1 Announce Type: new Abstract: Urban heat islands (UHIs) are intensifying under climate change, exacerbating thermal exposure risks. Their two primary observations, land surface tempe
With only 6B active parameters, Qwen3.8-Flash-Next-Base tops 8 of 14 benchmarks, including MMLU-Pro, SuperGPQA, BBH and GSM8K. And it remains competitive with Qwen3.7-Plus-Base on the rest. Its 51B N-
Advances in models, chips, or devices will continually improve local-first capabilities. We will open our benchmarks soon. Read the full post: https://www.perplexity.ai/hub/blog/a-local-first-agent-fo