AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “pretraining”

GridTimelineEvolution
6,758 results
9 Oct 2026

Residual Modeling Closes the Regression and Generative Policy Gap in Robot Learning

SafetyDGX agent

arXiv:2610.12231v1 Announce Type: new Abstract: Learning from demonstration has enabled impressive robot behaviors. A common choice for policy learning is to use diffusion or flow matching (Flow-Polic

Scalable AI Uncertainty Quantification via Generalized Laplace Active Subspaces

Local AiDGX agent

arXiv:2610.11738v1 Announce Type: new Abstract: Reliable uncertainty quantification (UQ) is essential for deploying neural networks in scientific and high-stakes applications, but full Bayesian infere

Smoothing the Top-k Exposure Boundary for Sparse Mixture-of-Experts

Model ReleasesDGX agent

arXiv:2610.11575v1 Announce Type: new Abstract: Sparse Mixture-of-Experts models scale parameter capacity efficiently while maintaining a fixed compute budget per token. However, traditional training

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

Tell Robot What Not to Do: A Negation Understanding Perspective

Model ReleasesDGX agent

arXiv:2610.11952v1 Announce Type: new Abstract: Instruction following enables robots to perform diverse tasks specified in natural language, making it a fundamental capability for human-robot interact

8 Oct 2026

Are Parameter-Efficient Fine-tuning Methods Really Different?

Model ReleasesDGX agent

arXiv:2610.09122v1 Announce Type: new Abstract: Parameter-efficient fine-tuning (PEFT) offers many parameterizations, yet their methodological and functional differences remain unclear. We compare six

{Delta}WAM: Distilling Action Tangent Fields into World Action Models

Local AiDGX agent

arXiv:2610.09734v1 Announce Type: new Abstract: World Action Models (WAM) improve robot policies by augmenting sparse action supervision with dense future prediction. However, much of the predictable

Juno: Taming Predictive Latents for Vision-Language-Action Models

SafetyDGX agent

arXiv:2610.09940v1 Announce Type: cross Abstract: Joint-embedding predictive architectures (JEPAs) predict masked or future observations in representation space, offering a natural source of predictiv

Mechanics of Long-Context Hybrid Models Part 1.1: From Hybrid Attention to Hybrid Position

TutorialsDGX agent

arXiv:2610.10114v1 Announce Type: new Abstract: The architectural design of Large Language Models (LLMs) is shifting from traditional full-attention-only models to hybrid models, which combine differe

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition

ResearchDGX agent

arXiv:2610.10245v1 Announce Type: new Abstract: Despite the ubiquity of sensors in wearable and mobile devices and the abundance of human movement data they generate, translating unlabeled recordings

Q-Learning with Scalar Adjoint Matching

SafetyDGX agent

arXiv:2610.10437v1 Announce Type: new Abstract: Flow policies capture rich and diverse action distributions, and fine-tuning them with off-policy RL to improve beyond the demonstrations has drawn grow

RDGSplat: Render-Dedicated Geometry for Novel View Synthesis

ResearchDGX agent

arXiv:2610.09173v1 Announce Type: new Abstract: 3D foundation models enable efficient novel view synthesis by carrying a Gaussian head on the representation they already use for reconstruction. Howeve

Right Number, Wrong State? Measuring Cross-Jurisdiction Substitution in LLM Recall of State Policy

Model ReleasesDGX agent

arXiv:2610.09458v1 Announce Type: new Abstract: When an LLM answers a state-specific policy question wrongly, it may be hallucinating, or it may be returning a real value that holds in another state.

SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models

ResearchDGX agent

arXiv:2610.09317v1 Announce Type: new Abstract: Synthetic aperture radar (SAR) and electro-optical (EO) imagery provide complementary observations: SAR enables day-and-night, weather-resilient sensing

Video Prediction Policy 2: Predict Better, Act Better

Model ReleasesDGX agent

arXiv:2610.10270v1 Announce Type: new Abstract: World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learnin

7 Oct 2026

A doctrine-grounded visual question answering dataset for Tactical Combat Casualty Care

ResearchDGX agent

arXiv:2610.07339v1 Announce Type: cross Abstract: Tactical Combat Casualty Care (TC3) requires responders to connect visual observations of injuries and interventions with established clinical guidanc

Beyond Training from Scratch: Foundation Models for Data-Efficient and Generalizable Cardiac MRI Reconstruction

Model ReleasesDGX agent

arXiv:2610.08109v1 Announce Type: new Abstract: Cardiac magnetic resonance imaging reconstruction aims to recover high-quality images from undersampled acquisitions, enabling faster scans while preser

CETUS: How Far Do Representations Trained on Earth Transfer to Cassini SAR of Titan?

TutorialsDGX agent

arXiv:2610.07576v1 Announce Type: new Abstract: Cassini synthetic aperture radar (SAR) images reveal the dunes, plains, and lake basins of Titan, providing an instance of representations learned from

Complementary Supervised and Self-Supervised Representations for Out-of-Distribution Graph Learning

ResearchDGX agent

arXiv:2610.07628v1 Announce Type: new Abstract: Out-of-distribution (OOD) generalization remains challenging for graph neural networks (GNNs), as graph distributions can vary substantially across time

DepthWorld: 3D World Model for Robot Manipulation

SafetyDGX agent

arXiv:2610.08780v1 Announce Type: cross Abstract: World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and pl

Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations

ResearchDGX agent

arXiv:2610.07906v1 Announce Type: new Abstract: We study sequential content sufficiency by investigating whether a representation retains the ordered target information available in its input. An info

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?

Model ReleasesDGX agent

arXiv:2610.08215v1 Announce Type: new Abstract: Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments. Evaluating this ability is therefore important for

Learning Grasp Targeting from Point Clouds for Log Pile Clearing on a Hydraulic Crane

SafetyDGX agent

arXiv:2610.07613v1 Announce Type: cross Abstract: In mill yards, log loaders clear dense piles by a sequence of bundle grasps: hundreds of logs rest in contact, and each removal changes the pile avail

MASKerade: Token-Routed Mask Experts for Dense-to-MoE Upcycling

Model ReleasesDGX agent

arXiv:2610.07809v1 Announce Type: new Abstract: Sparsely activated Mixture-of-Experts (MoE) models increase model capacity without a proportional increase in per-token computation. Dense-to-MoE upcycl

No Transformer Beats Six Covariates: Long-Horizon Prediction of Depressive Symptoms from Childhood Essays

ResearchDGX agent

arXiv:2610.07764v1 Announce Type: cross Abstract: Natural language processing (NLP) models can detect depression-related language in text written near the time symptoms are measured, but whether pretr

One for All, All for One: Coordinated Multi-Agent Diffusion Steering via Stochastic Optimal Control

AgentsDGX agent

arXiv:2610.08595v1 Announce Type: cross Abstract: Deep generative models often produce structured outputs composed of interacting components. Modelling these outputs with a single model requires learn

Pronunciation Editing for Finnish Speech using Phonetic Posteriorgrams

SafetyDGX agent

arXiv:2507.02115v3 Announce Type: cross Abstract: Synthesizing second-language (L2) speech is potentially highly valued for L2 language learning experience and feedback. However, due to the lack of L2

Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning

ResearchDGX agent

arXiv:2610.07385v1 Announce Type: new Abstract: Fine-tuning vision-language models (VLMs) is typically evaluated at a single downstream checkpoint, obscuring whether a semantic capability was never ac

TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity

Model ReleasesDGX agent

arXiv:2610.07559v1 Announce Type: new Abstract: Recent progress in tabular foundation models suggests that training on synthetic tasks can substantially improve in-context learning capabilities, with

WavePrune: One period is often enough for RoPE

HardwareDGX agent

arXiv:2610.06963v1 Announce Type: new Abstract: Rotary Position Embedding (RoPE) encodes token positions by rotating each two-dimensional channel of the query and key vectors at a channel-specific fre

WiSPER: Pose-Supervised Predictive and Residual Flow Refinement For Multi-Person 3D Pose Estimation With WiFi CSI

Local AiDGX agent

arXiv:2610.07025v1 Announce Type: new Abstract: Multi-person 3D pose estimation with WiFi channel state information (CSI) is challenging because reflections from different people overlap without direc

6 Oct 2026

Bridging the EHR Divide: Asymmetric Contrastive Learning for Cross-National Medical Representation Transfer

ApplicationsDGX agent

arXiv:2610.04946v1 Announce Type: new Abstract: Cross-system transfer of longitudinal Electronic Health Record (EHR) representations is challenging because clinical coding, patient populations, and he

Clean: Second-order LLM Training at Linear Memory Cost via Nystrom Sketching

Model ReleasesDGX agent

arXiv:2610.04204v1 Announce Type: new Abstract: Training large language models (LLMs) entails a fundamental trade-off: memory-efficient optimizers such as Adam discard cross-parameter curvature, where

Differentially Private Mixing of Public Datasets Improves Private Learning

ResearchDGX agent

arXiv:2610.06636v1 Announce Type: new Abstract: Many machine learning applications involve sensitive data and therefore require training under differential privacy (DP). However, DP training often deg

Diffusion Transformers are Provably Optimal In-context Generators

TutorialsDGX agent

arXiv:2610.05333v1 Announce Type: new Abstract: Generative foundation models are attracting interest for their ability to produce desired outputs from demonstrations given at inference time, without u

DriftSR: One-Step Real-World Image Super-Resolution via Distribution Drifting

Local AiDGX agent

arXiv:2610.04819v1 Announce Type: new Abstract: One-step real-world image super-resolution (Real-ISR) offers efficient inference, but recovering realistic and perceptually rich details often relies on

EagleDepth: Efficient Fine-Grained Depth Estimation via Pixel Diffusion Decoder

ResearchDGX agent

arXiv:2610.04554v1 Announce Type: new Abstract: Recovering detailed geometry from high-resolution images is critical for precise perception of the surroundings and objects. However, existing methods w

From Sight to Foresight: Predictive Spatial Reasoning in Vision-Language Models

Model ReleasesDGX agent

arXiv:2610.04139v1 Announce Type: new Abstract: Predicting future spatial states supports collision avoidance and timely decision-making in dynamic environments. However, existing vision-language mode

Gefen: Optimized Stochastic Optimizer

Model ReleasesDGX agent

arXiv:2606.13894v3 Announce Type: replace-cross Abstract: AdamW is a default optimizer for deep learning, but its moment states add two parameter-sized buffers to training memory, increasing the cost

In-Context Time Series Classification with Random Convolutional Features

Local AiDGX agent

arXiv:2607.19234v2 Announce Type: replace Abstract: Time series classification is central to domains such as medical signal analysis, industrial monitoring, and sensor-based activity recognition, wher

Learning Conditional Source Distribution via Flow Reversal for Temporal Flow Matching

ResearchDGX agent

arXiv:2610.05349v1 Announce Type: new Abstract: We introduce CNP-Flow, a flow matching framework for temporal generation that learns conditional source distributions through flow reversal. Whereas sta

Learning to Learn a Language: in-context learning of natural language from a synthetic non-linguistic prior [R]

Model ReleasesDGX agent

Learning from data as we observe it is easy for humans, but most machine learning models have limited ability to learn from new data that they have not seen during training. Prior-fitted networks (the

Level-of-Token Diffusion

AgentsDGX agent

arXiv:2610.05816v1 Announce Type: new Abstract: Image and video diffusion models allocate equal computation to every region, even when the intended scene calls for varying levels of detail. The spatia

Lightweight Semantic EEG Foundation Model for Frozen Cross-Disorder Transfer

TutorialsDGX agent

arXiv:2610.05503v1 Announce Type: new Abstract: Large-scale EEG foundation models have demonstrated promising transferability across neurological disorders, but often require millions of parameters an

Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

SafetyDGX agent

arXiv:2610.05872v1 Announce Type: new Abstract: A long-standing goal of AI is a model that can continually learn and improve itself. On post-trained models, supervised finetuning (SFT) on new data oft

OncoNoteBERT: A Foundation Representation Model for Natural Language Processing of Real-World Outpatient Oncology Notes

Local AiDGX agent

arXiv:2610.03829v1 Announce Type: cross Abstract: Real-world outpatient oncology notes contain specialised terminology, tumour staging expressions, treatment names, toxicity descriptions, and institut

Orchestrating Level-K Policies Against Unknown Opponents in Partially-Observable Dynamic Games

SafetyDGX agent

arXiv:2610.04937v1 Announce Type: cross Abstract: Level-K reasoning generates a hierarchy of policies specialized to opponents with different reasoning levels. When an opponent's level is unknown, a c

Pythia: Toward Foundation World Models for Multimodal Time Series

TutorialsDGX agent

arXiv:2610.05240v1 Announce Type: new Abstract: Time-series foundation models offer a unified approach to forecasting across heterogeneous domains. Textual context and auxiliary observations provide c

Scaling Down the Scaling Laws: Parameter Efficiency and Compute-Optimal Training in Resource-Constrained Large Language Models

Model ReleasesDGX agent

arXiv:2610.06387v1 Announce Type: new Abstract: Large language models (LLMs) have achieved substantial performance gains through increases in model size, training data, and computational resources. Ho

Spatial Supervision Without Attribution Optimization: Improving Post-Hoc Class Activation Maps via Box-Guided Evidence Routing

ResearchDGX agent

arXiv:2610.05891v1 Announce Type: new Abstract: Post-hoc class activation maps (CAMs) are a standard tool for inspecting the evidence behind an image classifier's predictions, yet nothing in ordinary

Transform Trained Transformer for Accelerating Native 4K Video Generation

ResearchDGX agent

arXiv:2512.13492v2 Announce Type: replace Abstract: Native 4K (2176imes3840) video generation remains a critical challenge due to the quadratic computational explosion of full-attention as spatiotempo

Your Temporal Link Predictor Is Blind to Who Is Active: A Missing Factor That Transfers Across Models

ResearchDGX agent

arXiv:2610.04869v1 Announce Type: new Abstract: An interaction has two parts: someone decides to act, and then chooses whom to act on. Temporal link prediction has concentrated on the second, and we s

5 Oct 2026

Beyond Random Splits: Evaluating Drug-Target Affinity Models Under Chemically and Biologically Motivated Distribution Shifts Copy

ResearchDGX agent

arXiv:2610.03456v1 Announce Type: new Abstract: Drug-target affinity (DTA) prediction is widely used to prioritize candidate compounds before costly experimental screening. DTA models are often compar

CreateScore: Domain-Theory-Informed Bayesian Routing for LLM-Based CV Screening

Local AiDGX agent

arXiv:2610.02972v1 Announce Type: new Abstract: Large language models (LLMs) can support rubric-based screening of CVs, but applying a high-capability model to every candidate and criterion is costly.

DeepStratNet: A Context-Aware Coordinate Regression Framework for Seismic Horizon Tracking under Sparse Labels

ResearchDGX agent

arXiv:2610.02494v1 Announce Type: new Abstract: Automatic horizon tracking is a foundational task in 3D seismic interpretation. Most existing deep learning approaches formulate it as dense semantic se

FlowHMR: Physically Plausible Motion Capture from Video

SafetyDGX agent

arXiv:2610.03691v1 Announce Type: new Abstract: We present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning-based methods typical

Mitigating Private Data Leakage in LLMs with Whiteout

SafetyDGX agent

arXiv:2610.02418v1 Announce Type: cross Abstract: Modern large language models (LLMs) are trained on massive, largely unfiltered datasets, including content scraped from nearly every accessible websit

PointWAM: 3D World Action Modeling for Dexterous Robotic Manipulation

TutorialsDGX agent

arXiv:2610.02840v1 Announce Type: cross Abstract: World action models jointly learn to forecast world dynamics and predict robot actions, such that the learned internal world dynamics guide accurate a

RAOA: Alternating-Operator Neural Computation with Programmable Radio Propagation

ResearchDGX agent

arXiv:2610.02683v1 Announce Type: new Abstract: Can programmable radio propagation serve as computational depth rather than only as a communication channel or one-shot analog transform? We introduce t

Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis

SafetyDGX agent

arXiv:2610.02718v1 Announce Type: cross Abstract: Vision-language models such as CLIP exhibit strong semantic generalization, but remain limited in fine-grained visual perception. A recent work named

4 Oct 2026

I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens

Model ReleasesDGX agent

First of all, thank you for reading. I trained a small MoE model completely from scratch (no external base weights) and wanted to share the results + a couple of lessons. Apex-2 - Architecture: Decode

← Previous
1…3334353637…113
Next →