Search: “pretraining”

Megatron‑LM PR #7262 introduces a workflow that records ordered per‑rank tensor fingerprints to pinpoint and correct nondeterminism in large‑scale pretraining. By implementing low‑overhead kernel twea

Skild AI: Skild AI unveils S1, a robotics foundation model that it says can learn tasks never seen during pretraining, using a single video demo, without fine-tuning — UNSEEN TASKS10-MINUTE HORIZONSON

Financial Times: Sources: ByteDance is pretraining an AI model with up to 10T parameters, roughly 3x larger than Kimi K3 and larger than the 8T estimate for Anthropic's Mythos 5 — TikTok owner trainin

This article from Latent Space provides guidance on securing employment at cutting-edge AI research laboratories, with a focus on roles related to pretraining large language models. It likely covers p

Anthropic: Anthropic researchers detail “model spec midtraining”, which adds a stage between pretraining and fine-tuning to improve generalization from alignment training — Sara Price2, Samuel Marks2,

Recent advancements in large language models have intensified the need for efficient and deployable models within limited inference budgets. Structured pruning pipelines have shown promise in token ef

The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling their use across diverse hardware and software plat

Visual generative models (e.g., diffusion models) typically operate in compressed latent spaces to balance training efficiency and sample quality. In parallel, there has been growing interest in lever

COMPASS is a scalable cross‑embodiment robot navigation framework that builds on NVIDIA’s X‑Mobility pretrained policies, adding lightweight RL fine‑tuning specialists for new robot‑scene pairs to avo

Auto-generated index of all papers mentioned across the wiki.

Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total pretraining data budget they still fall short of monolingual models. We show

An Anthropic PBC researcher has resigned over concerns that artificial intelligence labs are “gambling with our lives.” Jacob Coxon was part of the company’s AI pretraining team until today. He announ

Auto-generated index of all models mentioned across the wiki.

Standard Bots develops AI‑native industrial robot arms (used for machine tending, welding, assembly) that rely on a shared pretrained base model with 1–2 billion parameters. The model is adapted to ea

Auto-generated index of all tools mentioned across the wiki.

Automated lint: 14 errors, 54840 warnings, 3 info

Automated lint: 12 errors, 54839 warnings, 3 info

Automated lint: 12 errors, 53990 warnings, 3 info

It is impossible to make large language models fully secure against hacks because of a fundamental flaw in how they work, a team of researchers argue in a paper presented at the International Conferen

Automated lint: 26 errors, 6728 warnings, 3 info

At Google Cloud Next ’26 we announced Cloud Storage Rapid, a family of object storage capabilities for data-intensive workloads like AI and analytics. Out of the gate, Cloud Storage Rapid consists of

This article outlines Sebastian Raschka's systematic approach to learning and understanding Large Language Model (LLM) architectures, likely covering foundational concepts, key components, and methodo

Auto-generated index of all companies mentioned across the wiki.

Auto-generated index of all people mentioned across the wiki.

NVIDIA’s Seattle Robotics Lab and Isaac engineering team created reusable robotic systems to automate GB300 tester‑tray assembly, achieving over 95 % success on busbar tasks with a 160‑second cycle ti

Automated lint: 19 errors, 64915 warnings, 3 info

Auto-generated synthesis of 18054 entries about arxiv-cs-cv

Automated lint: 17 errors, 59669 warnings, 3 info

NVIDIA Cluster Readiness Engine (NVCRE) is an open‑source Kubernetes controller that runs real distributed workloads across topology‑aware node groups to certify GPU cluster readiness before productio

Discrete flow matching generates text by iteratively transforming noise tokens into coherent language, but may require hundreds of forward passes. Distillation uses the multi-step trajectory to train

The Guardian: Matt Clifford steps down as the chair of the UK government's science and tech research unit after joining Anthropic, following conflict of interest concerns — Matt Clifford forced to sta

Michael Nuñez / VentureBeat: The Agentic AI Foundation updates MCP with a fully stateless architecture, a hardened authentication model, a formal 12-month deprecation policy, and more — The Model Cont

How we built the OlmoEarth Platform to fine-tune geospatial models and run continent-scale satellite inference while managing massive data pipelines, distributed compute, and automatically recovering

Automated lint: 20 errors, 8743 warnings, 3 info

Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding This is an interesting new open weights (MIT licensed) model, the first model release from DeepReinforce. [...] with variants including 9B Dense, 3

If Claude Fable stops helping you, you'll never know Jonathon Ready highlights one of the more eyebrow-raising details from the 319 page system card for Fable 5 and Mythos 5. Here's a longer excerpt,

At Google Cloud Next, we are announcing innovations across every layer of our storage stacks — performance, intelligence, and management — to ensure your data is as fast and as useful as the AI models

This paper was accepted at the Workshop on Navigating and Addressing Data Problems for Foundation Models (NADPFM) at ICLR 2026. Principled domain reweighting can substantially improve sample efficienc

This paper was accepted at the Workshop on Memory for LLM-Based Agentic Systems at ICLR. Language models have consistently grown to compress more world knowledge into their parameters, but the knowled

Muse Spark is the first model from Meta's Superintelligence Labs division and the debut entry in the new Muse model family, representing a 'ground-up overhaul' of the company's AI efforts. It is ...