Search: “pretraining”

OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning

arXiv:2610.20418v1 Announce Type: new Abstract: Tabular foundation models enable in-context learning by predicting unseen observations from a labeled support table without task-specific parameter upda

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Parameter-Efficient Temporal Domain Generalization

DGX agent

arXiv:2602.11965v2 Announce Type: replace Abstract: Temporal domain generalization (TDG) learns from a sequence of observed domains to predict models for unseen future domains. Existing parameter-dyna

model-releasesarxiv-cs-lg

15 Oct 2026

PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models

arXiv:2606.16569v2 Announce Type: replace Abstract: Registering two captures of the same indoor space taken at different times underpins persistent spatial memory for robots and AR systems, yet the re

Provable Transformer Context Compression for In-Context Reinforcement Learning

arXiv:2610.20004v1 Announce Type: new Abstract: Transformers can perform in-context reinforcement learning without parameter updates. However, existing theoretical constructions often require context

RAN-JEPA: An Action-Conditioned World Model for Energy-efficient Mobile Network Control

arXiv:2610.17584v1 Announce Type: new Abstract: Operational mobile networks are rich in observations but poor in interventions: they continuously generate large volumes of passive telemetry, while con

researcharxiv-cs-ai

ReAl: Refine neural operators via aligning dynamical flow

arXiv:2610.20369v1 Announce Type: new Abstract: Neural partial differential equation (PDE) surrogates trained with mean squared error (MSE) can capture dominant dynamics but often lose high-frequency

safetyarxiv-cs-lg

ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning

arXiv:2610.19700v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning by learning low-rank weight updates. Increasing rank expands adaptation capacity, but

Remember, Revisit, Revise: Scaffold Memory for Long-Horizon 3D Geometry

arXiv:2610.19674v1 Announce Type: new Abstract: Geometry foundation models estimate camera poses and 3D scene structure from video sequences, but maintaining consistent reconstructions over long horiz

local-aiarxiv-cs-cv

ReproBAIT: Can coding agents build their own biological AI tools?

arXiv:2610.19810v1 Announce Type: cross Abstract: Large language models are increasingly competent at leveraging scientific knowledge and existing computational tools to execute complex workflows. At

agentsarxiv-cs-lg

Risk-Controlled DFT Selection for Machine-Learned Interatomic Potentials

arXiv:2610.19630v1 Announce Type: cross Abstract: Automated materials discovery requires evaluating large pools of candidate configurations, with density functional theory (DFT) as a major throughput

researcharxiv-cs-lg

Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation

arXiv:2610.20247v1 Announce Type: new Abstract: Multilingual models share capacity across languages, so adding languages can hurt performance on existing ones. Mixture-of-experts models expand this ca

researcharxiv-cs-cl

Select or Rewrite: Oracle-Guided Fine-Tuning for Generative ASR Post-Editing from N-Best Hypotheses

arXiv:2610.15056v1 Announce Type: cross Abstract: Large language models can post-edit ASR by using linguistic context to compare N-best hypotheses and rewrite the transcript when needed, often improvi

safetyarxiv-cs-ai

SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models

arXiv:2610.19327v1 Announce Type: new Abstract: Robotic ultrasound can standardize image acquisition and reduce operator workload and variability, but policy learning remains limited by the lack of re

Spectral-MoE: Heterogeneous Per-Layer Expert Utilization in Dense-to-MoE Conversion

arXiv:2610.19232v1 Announce Type: new Abstract: Large language models impose substantial memory demands and arithmetic costs during inference. Dense-to-mixture-of-experts (MoE) conversion is a model o

SuPER-Mamba: Surprisal Prioritization for Extended-Context Routing in Mamba

model-releasesarxiv-cs-ai

arXiv:2610.16636v1 Announce Type: cross Abstract: State space models provide efficient linear-complexity processing for long sequences, but their performance frequently degrades as context length grow

TEX-MEX: Similarity-Guided Temporal Contrastive Learning for Time Series

arXiv:2610.19557v1 Announce Type: new Abstract: Contrastive learning for time series depends on deciding which pairs of inputs should be treated as similar and dissimilar. Because few augmentations re

The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation

model-releasesarxiv-cs-cl

arXiv:2610.20456v1 Announce Type: new Abstract: The scaling relationship of compute and performance for language models (LMs) has been extensively studied; more recently, models are being designed to

The Geometry of Language Model Logits: Generic Structure and Learned Departures

arXiv:2610.18520v1 Announce Type: cross Abstract: Autoregressive language models produce, at every prediction step, a vocabulary-sized logit vector whose structure determines the conditional next-toke

researcharxiv-cs-ai

Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2610.19108v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitat

Training LLMs Beyond Next Token Prediction - Filling the Mutual Information Gap

model-releasesarxiv-cs-ai

arXiv:2511.00198v2 Announce Type: replace-cross Abstract: Autoregressive language models are usually trained with next-token prediction under a fixed left-to-right target order. This paper studies tar

Virtual Head Attention

model-releasesarxiv-cs-ai

arXiv:2610.17959v1 Announce Type: cross Abstract: The design of attention mechanisms in long-context large language models faces an ``impossible triangle'' among KV cache size, training and prefill FL

VLRLA: Deployment-Time Adaptation of Vision-Language-Action Models through In-Context Reinforcement Learning

arXiv:2610.19517v1 Announce Type: cross Abstract: Vision-language-action (VLA) models connect visual perception and language understanding with robot control, providing a foundation for instruction-co

safetyarxiv-cs-ro

VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization

arXiv:2503.04940v3 Announce Type: replace-cross Abstract: Emergent Language (EL) focuses on the emergence of communication among artificial agents. Although symbolic communication channels more closel

safetyarxiv-cs-ai

What Matters for World Modeling in Vision-Language-Action Models?

arXiv:2610.19825v1 Announce Type: cross Abstract: Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynami

When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models

arXiv:2610.19814v1 Announce Type: new Abstract: Transformer-based tabular foundation models (TFMs) achieve state-of-the-art performance on small to medium-sized tabular prediction tasks while processi

local-aiarxiv-cs-lg

Why don't Graph Transformers attend to the same token?

arXiv:2610.19324v1 Announce Type: new Abstract: In large language models (LLMs), attention concentrates heavily on the first token. This 'attention sink' is believed to protect deep models against ove

local-aiarxiv-cs-lg

Applications

Acting Through Registers

arXiv:2610.17638v1 Announce Type: new Abstract: World Action Models (WAMs) use generative visual representations to condition robot actions, yet how these representations organize action-relevant info

applicationsarxiv-cs-ro

Agentic4PDM: Host-Governed Evidence Admissibility for Industrial Fault Diagnosis

arXiv:2610.17786v1 Announce Type: cross Abstract: We study industrial fault diagnosis as a classification task that assigns sensor time-series windows to predefined health states or fault types. Scarc

agentsarxiv-cs-lg

AI-Enhanced Hyperspectral Imaging for Intraoperative Detection of Low-Grade Glioma: A Clinical Study

arXiv:2610.17944v1 Announce Type: new Abstract: Current image-guided neuronavigation techniques, such as intraoperative MRI and ultrasound, are limited by high costs, limited sensitivity and operation

ATLAS: A Foundation Neural Sampler for Amorphous Materials

arXiv:2607.19198v2 Announce Type: replace-cross Abstract: Amorphous materials exhibit exceptional mechanical and functional properties, yet their rugged energy landscapes are notoriously difficult to

agentsarxiv-cs-lg

Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding

model-releasesarxiv-cs-cv

arXiv:2610.14589v1 Announce Type: new Abstract: Walking sequences reveal human-interpretable attributes related to body structure, posture, and motion. Gait attribute recognition provides a concrete r

Axiom-0: Hierarchical Self-Refinement for Vision-Language-Action Policies

arXiv:2610.14155v1 Announce Type: cross Abstract: Reliable manipulation requires policies that generalize across tasks, layouts, and object interactions. Vision-language-action (VLA) models combine pr

Barycentric Guidance: Turning Foundation Image Editors into Continuous Affective Controllers

arXiv:2610.16892v1 Announce Type: new Abstract: Emotions are expressed on a continuum and are naturally described in the valence-arousal (VA) space. Existing VA-based image editors are typically train

Tutorials

BridgeWAM: Bridging Video and Action Experts via Latent Bridge Queries

arXiv:2610.14746v1 Announce Type: cross Abstract: World Action Models (WAMs) commonly couple video and action experts through dense, layer-wise Mixture-of-Transformers (MoT) attention, exposing action

tutorialsarxiv-cs-cv

Building age estimation based on widely available multi-source1 geospatial data

model-releasesarxiv-cs-cv

arXiv:2610.16320v1 Announce Type: new Abstract: Building age is fundamental for urban infrastructure safety, operation, and planning, as it directly affects structural safety, and sustainable developm

Co-DSL: Human-LLM Co-design of Reusable Spatial Concepts for Layout Generation

arXiv:2610.15290v1 Announce Type: cross Abstract: We present Co-DSL, a framework in which users and LLMs co-design reusable spatial concepts for layout generation. In the first stage, our system allow

Coarse-Graining Recursive Expert Computation: Continuation-Relative Interfaces and Failure Modes

arXiv:2610.14413v1 Announce Type: new Abstract: Computational coarse-graining aims to replace a sequence of updates with a cheaper module while preserving task performance. We present a continuation-r

COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration

arXiv:2610.17332v1 Announce Type: new Abstract: Vision large language models (VLLMs) are deployed in privacy-sensitive domains, raising training-data exposure concerns. Existing gray-box membership in

researcharxiv-cs-lg

ConDenseDrive: Knowledge-Dense, Token-Sparse Visual Encoder for End-to-End Autonomous Driving

arXiv:2610.16908v1 Announce Type: new Abstract: End-to-end driving requires rich spatiotemporal representations to support scene understanding and trajectory planning under strict latency constraints.

agentsarxiv-cs-cv

Cross-Market Meta-Learning for Electricity Price Forecasting

arXiv:2610.17387v1 Announce Type: new Abstract: Day-ahead (DA) electricity price forecasting is usually posed per market, with a separate model fitted to each grid's own history. This caps accuracy: s

CryoTracer: A Synthetic-to-Real System for Filament Instance Tracing in Cryo-EM

arXiv:2610.18696v1 Announce Type: new Abstract: Particle picking in cryo-electron microscopy is commonly formulated as an object-detection problem in which individual particles are represented by boun

local-aiarxiv-cs-cv

DeKoR: Decomposed Knowledge Routing for Efficient GNN--LLM Fusion

arXiv:2610.14710v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential for text-attributed graph learning by leveraging rich pre-trained knowledge and language unders

local-aiarxiv-cs-lg

Delta Flow Matching for Robot Foundation Models

arXiv:2610.15794v1 Announce Type: new Abstract: Flow matching is a common generative recipe for robot foundation models. Vision-language-action models (VLAs) generate action chunks, while world-action

safetyarxiv-cs-ro

DexStitch: Learning and Composing Dexterous Skills from Human Videos via Contact-Aware Robotic Inpainting

arXiv:2610.17789v1 Announce Type: cross Abstract: A long-horizon manipulation task consists of primitive skills and the transitions connecting them. Learning skills that can be executed in sequence re

Applications

ECHO: Correcting Time Series Foundation Model Forecasts with Historical Analogues and Errors

arXiv:2610.16433v1 Announce Type: new Abstract: Time Series Foundation Models (TSFMs) provide general-purpose forecasting capabilities, yet their forecasts can remain misaligned with the temporal char

applicationsarxiv-cs-lg

Hardware

Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and Inference

arXiv:2610.14277v1 Announce Type: cross Abstract: Tensor parallelism ends every layer with an all-reduce that sits between two dependent matrix multiplications, so its cost cannot be hidden and grows

hardwarearxiv-cs-lg

Efficient Frechet Distributional Decoder Alignment for Latent Generative Models

arXiv:2610.17466v1 Announce Type: new Abstract: Frechet Distributional (FD) objectives provide an effective distribution-level signal for post-training generative models. For latent generative models,

Applications

Efficient Test-Time Adaptation through Latent Subspace Coefficients Search

arXiv:2510.11068v4 Announce Type: replace Abstract: Real-world deployment often exposes vision models to distribution shifts, making test-time adaptation (TTA) important for reliable prediction. Exist

applicationsarxiv-cs-lg