Search: “pretraining”

arXiv:2610.08595v1 Announce Type: cross Abstract: Deep generative models often produce structured outputs composed of interacting components. Modelling these outputs with a single model requires learn

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Pronunciation Editing for Finnish Speech using Phonetic Posteriorgrams

DGX agent

arXiv:2507.02115v3 Announce Type: cross Abstract: Synthesizing second-language (L2) speech is potentially highly valued for L2 language learning experience and feedback. However, due to the lack of L2

safetyarxiv-cs-cl

7 Oct 2026

Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning

arXiv:2610.07385v1 Announce Type: new Abstract: Fine-tuning vision-language models (VLMs) is typically evaluated at a single downstream checkpoint, obscuring whether a semantic capability was never ac

TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity

arXiv:2610.07559v1 Announce Type: new Abstract: Recent progress in tabular foundation models suggests that training on synthetic tasks can substantially improve in-context learning capabilities, with

Hardware

WavePrune: One period is often enough for RoPE

arXiv:2610.06963v1 Announce Type: new Abstract: Rotary Position Embedding (RoPE) encodes token positions by rotating each two-dimensional channel of the query and key vectors at a channel-specific fre

hardwarearxiv-cs-cl

WiSPER: Pose-Supervised Predictive and Residual Flow Refinement For Multi-Person 3D Pose Estimation With WiFi CSI

arXiv:2610.07025v1 Announce Type: new Abstract: Multi-person 3D pose estimation with WiFi channel state information (CSI) is challenging because reflections from different people overlap without direc

local-aiarxiv-cs-cv

Applications

Bridging the EHR Divide: Asymmetric Contrastive Learning for Cross-National Medical Representation Transfer

arXiv:2610.04946v1 Announce Type: new Abstract: Cross-system transfer of longitudinal Electronic Health Record (EHR) representations is challenging because clinical coding, patient populations, and he

applicationsarxiv-cs-lg

Clean: Second-order LLM Training at Linear Memory Cost via Nystrom Sketching

arXiv:2610.04204v1 Announce Type: new Abstract: Training large language models (LLMs) entails a fundamental trade-off: memory-efficient optimizers such as Adam discard cross-parameter curvature, where

Differentially Private Mixing of Public Datasets Improves Private Learning

arXiv:2610.06636v1 Announce Type: new Abstract: Many machine learning applications involve sensitive data and therefore require training under differential privacy (DP). However, DP training often deg

Diffusion Transformers are Provably Optimal In-context Generators

arXiv:2610.05333v1 Announce Type: new Abstract: Generative foundation models are attracting interest for their ability to produce desired outputs from demonstrations given at inference time, without u

DriftSR: One-Step Real-World Image Super-Resolution via Distribution Drifting

arXiv:2610.04819v1 Announce Type: new Abstract: One-step real-world image super-resolution (Real-ISR) offers efficient inference, but recovering realistic and perceptually rich details often relies on

local-aiarxiv-cs-cv

EagleDepth: Efficient Fine-Grained Depth Estimation via Pixel Diffusion Decoder

arXiv:2610.04554v1 Announce Type: new Abstract: Recovering detailed geometry from high-resolution images is critical for precise perception of the surroundings and objects. However, existing methods w

From Sight to Foresight: Predictive Spatial Reasoning in Vision-Language Models

model-releasesarxiv-cs-cv

arXiv:2610.04139v1 Announce Type: new Abstract: Predicting future spatial states supports collision avoidance and timely decision-making in dynamic environments. However, existing vision-language mode

Gefen: Optimized Stochastic Optimizer

model-releasesarxiv-cs-cv

arXiv:2606.13894v3 Announce Type: replace-cross Abstract: AdamW is a default optimizer for deep learning, but its moment states add two parameter-sized buffers to training memory, increasing the cost

In-Context Time Series Classification with Random Convolutional Features

arXiv:2607.19234v2 Announce Type: replace Abstract: Time series classification is central to domains such as medical signal analysis, industrial monitoring, and sensor-based activity recognition, wher

local-aiarxiv-cs-lg

Learning Conditional Source Distribution via Flow Reversal for Temporal Flow Matching

arXiv:2610.05349v1 Announce Type: new Abstract: We introduce CNP-Flow, a flow matching framework for temporal generation that learns conditional source distributions through flow reversal. Whereas sta

Learning to Learn a Language: in-context learning of natural language from a synthetic non-linguistic prior [R]

model-releasesr-machinelearning

Learning from data as we observe it is easy for humans, but most machine learning models have limited ability to learn from new data that they have not seen during training. Prior-fitted networks (the

Agents

Level-of-Token Diffusion

arXiv:2610.05816v1 Announce Type: new Abstract: Image and video diffusion models allocate equal computation to every region, even when the intended scene calls for varying levels of detail. The spatia

agentsarxiv-cs-cv

Lightweight Semantic EEG Foundation Model for Frozen Cross-Disorder Transfer

arXiv:2610.05503v1 Announce Type: new Abstract: Large-scale EEG foundation models have demonstrated promising transferability across neurological disorders, but often require millions of parameters an

Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

arXiv:2610.05872v1 Announce Type: new Abstract: A long-standing goal of AI is a model that can continually learn and improve itself. On post-trained models, supervised finetuning (SFT) on new data oft

safetyarxiv-cs-lg

OncoNoteBERT: A Foundation Representation Model for Natural Language Processing of Real-World Outpatient Oncology Notes

arXiv:2610.03829v1 Announce Type: cross Abstract: Real-world outpatient oncology notes contain specialised terminology, tumour staging expressions, treatment names, toxicity descriptions, and institut

local-aiarxiv-cs-lg

Orchestrating Level-K Policies Against Unknown Opponents in Partially-Observable Dynamic Games

arXiv:2610.04937v1 Announce Type: cross Abstract: Level-K reasoning generates a hierarchy of policies specialized to opponents with different reasoning levels. When an opponent's level is unknown, a c

safetyarxiv-cs-ro

Pythia: Toward Foundation World Models for Multimodal Time Series

arXiv:2610.05240v1 Announce Type: new Abstract: Time-series foundation models offer a unified approach to forecasting across heterogeneous domains. Textual context and auxiliary observations provide c

Scaling Down the Scaling Laws: Parameter Efficiency and Compute-Optimal Training in Resource-Constrained Large Language Models

arXiv:2610.06387v1 Announce Type: new Abstract: Large language models (LLMs) have achieved substantial performance gains through increases in model size, training data, and computational resources. Ho

Spatial Supervision Without Attribution Optimization: Improving Post-Hoc Class Activation Maps via Box-Guided Evidence Routing

arXiv:2610.05891v1 Announce Type: new Abstract: Post-hoc class activation maps (CAMs) are a standard tool for inspecting the evidence behind an image classifier's predictions, yet nothing in ordinary

Transform Trained Transformer for Accelerating Native 4K Video Generation

arXiv:2512.13492v2 Announce Type: replace Abstract: Native 4K (2176imes3840) video generation remains a critical challenge due to the quadratic computational explosion of full-attention as spatiotempo

Your Temporal Link Predictor Is Blind to Who Is Active: A Missing Factor That Transfers Across Models

arXiv:2610.04869v1 Announce Type: new Abstract: An interaction has two parts: someone decides to act, and then chooses whom to act on. Temporal link prediction has concentrated on the second, and we s

Beyond Random Splits: Evaluating Drug-Target Affinity Models Under Chemically and Biologically Motivated Distribution Shifts Copy

arXiv:2610.03456v1 Announce Type: new Abstract: Drug-target affinity (DTA) prediction is widely used to prioritize candidate compounds before costly experimental screening. DTA models are often compar

CreateScore: Domain-Theory-Informed Bayesian Routing for LLM-Based CV Screening

arXiv:2610.02972v1 Announce Type: new Abstract: Large language models (LLMs) can support rubric-based screening of CVs, but applying a high-capability model to every candidate and criterion is costly.

local-aiarxiv-cs-ai

DeepStratNet: A Context-Aware Coordinate Regression Framework for Seismic Horizon Tracking under Sparse Labels

arXiv:2610.02494v1 Announce Type: new Abstract: Automatic horizon tracking is a foundational task in 3D seismic interpretation. Most existing deep learning approaches formulate it as dense semantic se

FlowHMR: Physically Plausible Motion Capture from Video

arXiv:2610.03691v1 Announce Type: new Abstract: We present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning-based methods typical

safetyarxiv-cs-cv

Mitigating Private Data Leakage in LLMs with Whiteout

arXiv:2610.02418v1 Announce Type: cross Abstract: Modern large language models (LLMs) are trained on massive, largely unfiltered datasets, including content scraped from nearly every accessible websit

safetyarxiv-cs-ai

PointWAM: 3D World Action Modeling for Dexterous Robotic Manipulation

arXiv:2610.02840v1 Announce Type: cross Abstract: World action models jointly learn to forecast world dynamics and predict robot actions, such that the learned internal world dynamics guide accurate a

tutorialsarxiv-cs-cv

RAOA: Alternating-Operator Neural Computation with Programmable Radio Propagation

arXiv:2610.02683v1 Announce Type: new Abstract: Can programmable radio propagation serve as computational depth rather than only as a communication channel or one-shot analog transform? We introduce t

Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis

arXiv:2610.02718v1 Announce Type: cross Abstract: Vision-language models such as CLIP exhibit strong semantic generalization, but remain limited in fine-grained visual perception. A recent work named

safetyarxiv-cs-ai

I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens

model-releasesr-localllama

First of all, thank you for reading. I trained a small MoE model completely from scratch (no external base weights) and wanted to share the results + a couple of lessons. Apex-2 - Architecture: Decode

4 Oct 2026

A Large Scale Investigation of Scaling Limits in Chemical Language Models

arXiv:2508.13408v3 Announce Type: replace Abstract: Chemical Language Models (CLMs) are increasingly used in de novo drug design, driven by recent growth in model scale, compute, and dataset size. How

Autoregressive Drillhole Modelling Under Distribution Shift

arXiv:2610.01204v1 Announce Type: new Abstract: Autoregressive modelling has achieved remarkable success in language and sequence tasks by learning to predict future states from previous observation.

EndoLive: Real-Time Style Transfer for Endoscopic Endonasal Skull Base Surgical Video

arXiv:2610.01956v1 Announce Type: new Abstract: Complex surgical procedures around critical anatomy, such as the endoscopic endonasal skull base surgery, requires significant practice and training on

tutorialsarxiv-cs-cv

Hardware

Fast Polynomial Transcendentals for LLMs

arXiv:2610.00049v1 Announce Type: new Abstract: Graphics processing unit (GPU) generations scale matrix, special-function, and memory pipelines at different rates, so kernel bottlenecks move as hardwa

hardwarearxiv-cs-lg

Applications

Generalist Representation, Specialist Detection: TS-Router for Time-Series Anomaly Detection

arXiv:2610.00978v1 Announce Type: cross Abstract: Time-series anomaly detection (TSAD) is difficult to generalize across datasets because heterogeneous temporal dynamics imply different notions of nor

applicationsarxiv-cs-ai

Generative Cinematographer: Composing Camera and Object Motion in 3D

arXiv:2610.02180v1 Announce Type: cross Abstract: Current controllable video generation systems often rely on 2D motion trajectories or sparse drag signals for object motion. These controls are ambigu

local-aiarxiv-cs-ai

Match the Distribution, Not the Compute: Post-Training Multi-Token Prediction Heads

arXiv:2610.00888v1 Announce Type: cross Abstract: Multi-token prediction (MTP) improves the throughput of autoregressive generation by enabling the language model to draft multiple next tokens per for

MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning

model-releasesarxiv-cs-cv

arXiv:2610.01352v1 Announce Type: new Abstract: Open multimodal reasoning models have benefited from large-scale reasoning supervision, yet reliable post-training remains challenging due to uneven dat

Not Too Hard, Not Too Easy: Learning from Intermediate States for LLM Structured Reasoning

arXiv:2609.33149v2 Announce Type: replace Abstract: A common principle of effective learning is to practice material that is neither already mastered nor too difficult to permit progress. We ask how t

Parameter-Efficient Distributionally Robust Adaptation of Tabular Foundation Models under Subpopulation Shift

arXiv:2610.01143v1 Announce Type: cross Abstract: Despite strong mean accuracy, tabular foundation models (TFMs) can perform poorly on underrepresented groups under subpopulation shift, where group pr

Personalized Image Generation with Reasoning and Reflection

arXiv:2610.00737v1 Announce Type: cross Abstract: Personalized image generation has remained narrowly focused on conditional synthesis from curated visual exemplars, rather than capturing who a user i

PG-SFT: Balancing Capability Acquisition and Retention in Offline Agent Fine-Tuning

arXiv:2610.00949v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) on offline agent trajectories is the standard approach for training specialized tool-using agents, but forcing models to im