Search: “pretraining”

The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention

arXiv:2610.00910v1 Announce Type: cross Abstract: Human reasoning depends on how objects are related within propositions. extit{How do relations organize the language representations of contextual con

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Tutorials

The hidden advantage of mask resampling: a theory of masked autoencoders

DGX agent

arXiv:2610.01578v1 Announce Type: cross Abstract: Why can masked prediction learn useful representations that unmasked reconstruction misses? We study this question in a high-dimensional model of a ma

tutorialsarxiv-cs-lg

2 Oct 2026

TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models

arXiv:2610.00899v1 Announce Type: cross Abstract: Autoregressive Vision-Language-Action models often represent continuous robot actions as discrete token sequences, enabling action prediction with sta

safetyarxiv-cs-ai

Training-Aware Target Coverage for Synthetic Data Selection

arXiv:2610.00814v1 Announce Type: cross Abstract: Synthetic data are increasingly used to scale LLM training, yet more synthetic data do not necessarily produce better models. Useful synthetic data mu

Applications

UniTrackPLA: Unified Panorama-Language-Action Model for Instruction-Guided Navigation and Dynamic Person Tracking

arXiv:2610.00878v1 Announce Type: cross Abstract: General-purpose embodied robots should support both navigation toward language-specified destinations and dynamic person tracking under arbitrary init

applicationsarxiv-cs-cv

Universal interpolation for deep residual self-attention networks

arXiv:2610.01981v1 Announce Type: new Abstract: Universal approximation is a necessary qualitative property of learning architectures to benefit from scaling laws. While it is generically verified on

Varda-single-1.0: deterministic data-driven weather forecasting at 1 km resolution over Switzerland's complex topography

arXiv:2610.01835v1 Announce Type: cross Abstract: We present Varda-single-1.0, a medium-range data-driven weather prediction system built for the Alpine domain. It provides hourly deterministic region

local-aiarxiv-cs-lg

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

arXiv:2609.39810v1 Announce Type: new Abstract: Source-Free Universal Domain Adaptation (SF-UniDA) extends Universal Domain Adaptation by removing access to source data at adaptation time while still

A Generalizable and Explainable Framework for Synthetic Video Detection Using First-Digit Gradient Statistics

arXiv:2609.39585v1 Announce Type: new Abstract: AI video generators have not only become harder to detect but are used to generate a diverse set of scenarios from landscapes to street views to animal

Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration

arXiv:2603.01623v2 Announce Type: replace Abstract: Diffusion models have become the dominant tool for high-fidelity image and video generation, yet are critically bottlenecked by their inference spee

An Uncertainty-Guided Digital Twin Framework for Online Adaptive Proton Therapy in Head and Neck Cancer: A Feasibility Study

arXiv:2609.39010v1 Announce Type: cross Abstract: Objective: Head and neck (HN) proton therapy spans six to seven weeks of anatomical change, while offline replanning takes about a week. We present an

AneumoBench: A Source-Linked Benchmark for Synthetic-Geometry Transfer in Aneurysm CFD

arXiv:2505.14717v2 Announce Type: replace-cross Abstract: Scientific machine learning uses simulation data to train surrogate models for fast physical-field prediction across geometries. Local shape e

Behavior-Grounded Semantic Enrichment for Financial Fraud Modeling and Reasoning

arXiv:2609.34211v2 Announce Type: replace Abstract: In financial fraud detection, rich semantic context can provide important evidence for transaction behavior modeling and fraud reasoning. However, p

Beyond Spatial-Domain Supervision: A Relation Constrained Space for Multi-Modal Image Fusion

arXiv:2609.38968v1 Announce Type: new Abstract: Multi-modal image fusion (MMIF) aims to form a single image by integrating shared information, preserving complementary cues, and coordinating cross-mod

safetyarxiv-cs-cv

Beyond the Current Scene: Event-Referential Grasping with Active View Selection

arXiv:2609.39375v1 Announce Type: cross Abstract: A robot that observes people interacting with objects should be able to carry out later requests that refer back to those interactions. Such requests

CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models

arXiv:2609.39441v1 Announce Type: cross Abstract: Online reinforcement learning has been extended to flow matching for diffusion model (DM) image generation. However, this paradigm faces three limitat

CellMSA: Context Modeling for Single-Cell Representation Learning

arXiv:2609.38908v1 Announce Type: cross Abstract: Single-cell transcriptomics enables profiling of cellular states at unprecedented resolution, but its high dimensionality, sparsity, and technical bat

safetyarxiv-cs-ai

Tutorials

CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding

arXiv:2609.39924v1 Announce Type: cross Abstract: Vision-language models face a fundamental scaling bottleneck: the number of visual tokens grows with both temporal duration and spatial resolution, ma

tutorialsarxiv-cs-ai

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

model-releasesarxiv-cs-cl

arXiv:2603.26164v2 Announce Type: replace-cross Abstract: Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters

DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence

model-releasesarxiv-cs-cv

arXiv:2609.39222v1 Announce Type: new Abstract: High-compression tokenizers are essential for scaling latent image generative models. However, aggressive compression creates a fundamental tradeoff bet

Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention

arXiv:2609.39494v1 Announce Type: new Abstract: Self-distillation with privileged context adapts a language model from demonstrations by letting the model, once conditioned on a reference response, te

Enhancing Autoregressive Video Generation via Representation Adversarial Distillation

arXiv:2609.40037v1 Announce Type: new Abstract: Few-step autoregressive video generation enables efficient streaming synthesis, but errors introduced in early temporal blocks are reused as context and

safetyarxiv-cs-cv

EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model -- From Semantic Understanding through Visual Foresight to Action

arXiv:2609.39973v1 Announce Type: new Abstract: Vision-language-action (VLA) policies emphasize semantic understanding, whereas world-action models (WAMs) learn predictive representations of environme

safetyarxiv-cs-ro

GestAdapt: Workspace-Conditioned Co-Speech Gesture Generation for Humanoid Robots

arXiv:2609.38400v1 Announce Type: cross Abstract: Co-speech gestures for robots must adapt not only to speech and embodiment, but also to the workspace available for performing the motion. Since the s

GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed

arXiv:2609.39600v1 Announce Type: cross Abstract: Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We vi

Instruction Retrieval at Inference Time for Small Language Models

arXiv:2510.13935v3 Announce Type: replace-cross Abstract: The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which nee

Learning Functional Subspaces for Neural Network Compression

model-releasesarxiv-cs-cl

arXiv:2609.40127v1 Announce Type: cross Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping

Learning Reliable GUI Agents under Imperfect Priors

arXiv:2609.39547v1 Announce Type: new Abstract: GUI agents built on large language and vision-language models still struggle on unseen applications and complex multi-step tasks, as completing real GUI

Tutorials

Learning Skills from Historical Action Trajectories: Action Experience Dictionary for World Action Models

arXiv:2609.40219v1 Announce Type: cross Abstract: World Action Models (WAMs) couple visual dynamics prediction with action generation, yet they do not explicitly support the reuse of action experience

tutorialsarxiv-cs-ai

Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models

arXiv:2609.39899v1 Announce Type: new Abstract: Cardiac magnetic resonance imaging (CMR) enables assessment of cardiac anatomy, ventricular function, and myocardial tissue characteristics. Clinicians

local-aiarxiv-cs-cv

Tutorials

Matisse: Evidence-Space Reasoning for Active 3D Reconstruction

arXiv:2609.38746v1 Announce Type: new Abstract: How can a 3D reconstruction system acquire and retain useful information to understand the geometry of a scene from partial views under a limited comput

tutorialsarxiv-cs-cv

MatLoom: Layered Text-to-Material Generation in a Compact Program Space

arXiv:2609.40322v1 Announce Type: cross Abstract: Material generation should produce not only an appearance, but also the rules that construct it. We introduce MatLoom, a compact, layer-oriented langu

Mitigating Memorization In Language Models

arXiv:2410.02159v3 Announce Type: replace-cross Abstract: Language models (LMs) can 'memorize' information, i.e., encode training data in their weights in such a way that inference-time queries can le

local-aiarxiv-cs-ai

Molecular Property Prediction under Structural Shift with Tabular Foundation Models

arXiv:2609.38744v1 Announce Type: new Abstract: Predicting molecular properties for compounds that differ structurally from labeled training molecules is important for drug discovery and materials des

Multi-Link Safety Filtering for VLA Policies Around Moving Hazards

arXiv:2609.40007v1 Announce Type: cross Abstract: A vision-language-action (VLA) policy can finish a manipulation task while knocking over objects unrelated to it, so task success alone does not show

safetyarxiv-cs-cv

OccluDex: Hierarchical 3D Visuo-Tactile Representation Learning for Egocentric Dexterous Manipulation under Self-Occlusion

arXiv:2609.39017v1 Announce Type: new Abstract: Reliable dexterous manipulation requires continuous estimation of object geometry and hand-object contact throughout interaction. With egocentric sensin

researcharxiv-cs-ro

OverdoseMoE: A Multi-Expert Framework for Opioid Overdose Risk Prediction

model-releasesarxiv-cs-cl

arXiv:2609.40108v1 Announce Type: new Abstract: Opioid overdose remains a major clinical and public health burden, highlighting the need for scalable approaches to identify patients at high risk. Here

PartiCam: Camera Controlled Video Generation with Reward Guidance

arXiv:2609.39504v1 Announce Type: cross Abstract: We present PartiCam, a training-free Particle filtering rooted method for improved Camera controlled video generation. Generating videos that follow a

local-aiarxiv-cs-ai

Persistent Watermarking of Text-to-Image Models

arXiv:2609.39024v1 Announce Type: new Abstract: Text-to-image (T2I) generation is gaining increasing popularity with the general public, motivating the development of reliable mechanisms for copyright

PhasePlan: Ordered Future-Phase Planning for Robot Brain Models

arXiv:2609.39970v1 Announce Type: new Abstract: Robot brain models integrate vision, language, and robot state to generate actions for complex manipulation tasks. Most predict fixed-length action chun

safetyarxiv-cs-ro

PhyProbe: Rethinking Physical Consistency Evaluation in Generated Videos

arXiv:2609.38377v1 Announce Type: new Abstract: Evaluating the physical consistency of generated videos remains a fundamental challenge. Existing approaches rely on off-the-shelf vision-language model

PISCO: Precise Video Instance Insertion with Sparse Control

arXiv:2602.08277v3 Announce Type: replace-cross Abstract: The landscape of AI video generation is undergoing a pivotal shift: moving beyond general generation - which relies on exhaustive prompt-engin

PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation

arXiv:2609.38597v1 Announce Type: new Abstract: Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and co

Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis

arXiv:2609.40361v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based ob

researcharxiv-cs-cl

Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text

arXiv:2609.40359v1 Announce Type: new Abstract: We find that major reported improvements in decoding words from non-invasive brain recordings are largely reproducible without any brain data. In the in

Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions

arXiv:2609.39243v1 Announce Type: new Abstract: Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural net

Scale-Split Neural Operator for Memory- and Data-Efficient 3D Turbulence Prediction

arXiv:2609.38977v1 Announce Type: cross Abstract: Neural surrogates have emerged as fast alternatives to the numerical simulation of three-dimensional turbulence. However, training them at high resolu

local-aiarxiv-cs-ai

Searching for BSM Experimental Signatures with Large Lagrangian Models

arXiv:2609.38309v1 Announce Type: cross Abstract: The search for physics Beyond the Standard Model (BSM) is generally limited not by the supply of theory descriptions but by the lack of discriminating