Search: “pretraining”

UniSpace: Unified Visual Representation and Scalable Multimodal Modeling

arXiv:2608.08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation. However, t

11 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

World Tokens: Enhancing Embodied Policies with Training-Time World Modeling

DGX agent

arXiv:2608.09730v1 Announce Type: new Abstract: Vision-language-action (VLA) models are a widely adopted paradigm for embodied policies. They excel at efficient closed-loop control but do not explicit

safetyarxiv-cs-cv

11 Aug 2026

Is SwiGLU's Open Positive Tail Necessary? Evidence from Closed-Tail Gating with MemGLU

arXiv:2608.07323v1 Announce Type: new Abstract: We test whether decoder-only language-model FFNs require SwiGLU's open positive tail. We introduce MemGLU as a closed-tail comparator derived from a mem

Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

arXiv:2608.07436v1 Announce Type: new Abstract: Under the standard split, Muon gets hidden matrices and AdamW embeddings/output head. Muon groks modular addition faster, but its solutions do not hold.

Retrofitting Linear Attention into Diffusion Language Models

arXiv:2608.06628v1 Announce Type: new Abstract: Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding. Recent dLL

Stoicheia: Character-Level Masked Diffusion for Ancient Greek Textual Restoration, Parsing, and Metrical Scansion

model-releasesarxiv-cs-cl

arXiv:2608.07249v1 Announce Type: new Abstract: We introduce Stoicheia, a 405M-parameter character-level masked-diffusion encoder for Ancient Greek whose input factors into five aligned, independently

TEMPO: Semantic-Action Decoupled RL Post-Training for Vision-Language-Action Models

arXiv:2608.07314v1 Announce Type: cross Abstract: Vision-language-action (VLA) models are commonly adapted to downstream manipulation tasks via supervised fine-tuning (SFT) or online reinforcement lea

[NEW MODEL] SupraElegans-500K

model-releasesr-localllama

*SupraLabs released a new experimental model!* SupraElegans-500K is a ~500,000-parameter causal language model built around a sparse, signed, recurrent neural graph. No Transformer, no attention mecha

9 Aug 2026

Agents

iARCS: Iterative Agentic RL for Controllable 3D Scene Generation

arXiv:2608.06161v1 Announce Type: new Abstract: Synthetic 3D scene generation is increasingly used as a data source for computer vision and embodied AI, but existing generators often optimize perceptu

agentsarxiv-cs-ai

Invisible Shortcuts: Why Vision Encoders Know Your Camera

arXiv:2608.05424v1 Announce Type: new Abstract: Deep vision models exploit shortcuts, relying on cues that correlate with supervision signals. Prior work has focused on visible biases, such as object-

researcharxiv-cs-cv

LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models

arXiv:2608.05706v1 Announce Type: new Abstract: World models enable agents to perform forward rollout and planning without real-world interaction. However, their application in open-world embodied int

safetyarxiv-cs-cv

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

arXiv:2604.01280v2 Announce Type: replace-cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires Multimodal Large Language Models (MLLMs) to identify and combine fine-grained visu

MameLoshnLM: Yiddish Language Model and Evaluation Benchmark

arXiv:2608.05850v1 Announce Type: cross Abstract: We present MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish. Despite Yiddish's rich textual tradition, it

ProDVI: Programmatic Dynamics Priors for Value Network Initialization

arXiv:2608.06015v1 Announce Type: cross Abstract: Deep Reinforcement Learning (RL) is notoriously sample inefficient. One contributing factor is that RL agents are typically initialized from scratch,

The Tamed Subgradient Unadjusted Langevin Algorithm beyond Convexity

arXiv:2608.06283v1 Announce Type: new Abstract: We study the problem of sampling from target distributions whose potentials are simultaneously non-smooth, subject to superlinear gradient growth, and n

Local Ai

Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming

arXiv:2608.05663v1 Announce Type: new Abstract: Real-time long-form avatar audio--video generation requires causal, continuous synthesis while maintaining audiovisual synchronization and visual consis

local-aiarxiv-cs-cv

Personalized Federated Sparse Adaptation of Time-Series Foundation Models

arXiv:2608.04695v1 Announce Type: cross Abstract: Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distribute

Tactus: Open-Vocabulary Object Recognition from Low-Cost Pressure Arrays

arXiv:2608.04043v1 Announce Type: new Abstract: Resistive pressure arrays are the cheapest and most widely shipped tactile sensors, yet tactile representation learning has concentrated on optical sens

Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding

arXiv:2608.04127v1 Announce Type: new Abstract: Large language model agents need to perceive human behavior in physical environments. Millimeter-wave (mmWave) radar provides a privacy-friendly and con

The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data

arXiv:2608.04268v1 Announce Type: new Abstract: Generative models trained on artificially generated data have been shown to exhibit model collapse, resulting in significant performance degradation. As

safetyarxiv-cs-cl

Assessment of Conditional Diffusion Model for Synthetic Histopathology Image Generation

arXiv:2608.03990v1 Announce Type: new Abstract: Synthetic histopathology image generation has emerged as an approach that may address data scarcity in computational pathology, yet current evaluation m

Attention is Case-Sensitive

arXiv:2608.03711v1 Announce Type: cross Abstract: In human visual perception, uppercase lettering serves as a natural salience cue that captures attention within lowercase text. In this paper, we pres

researcharxiv-cs-cl

FOUND-AF: Benchmarking ECG Foundation Models for Atrial Fibrillation Detection

arXiv:2608.03597v1 Announce Type: new Abstract: Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia and is associated with increased risks of stroke, heart failure, and mortality.

GROW: Group-Relative Advantage-Weighted On-Policy Reinforcement Learning of Autoregressive-Diffusion Text-to-Speech model

arXiv:2608.03215v1 Announce Type: cross Abstract: Reinforcement learning for flow-matching text-to-speech is complicated by deterministic ODE sampling: trajectory-level policy-gradient methods typical

safetyarxiv-cs-ai

Scaling an Autoregressive Transformer for Single-Cell Generation

arXiv:2608.02961v1 Announce Type: cross Abstract: We study a self-supervised generation task for single-cell gene expression vectors: given a set of vectors from a cell type, we aim to generate additi

SeqLLM: Augmenting LLMs with Behavioral-Sequence Modeling for High-Stakes Decisions at WeChat Pay

model-releasesarxiv-cs-cl

arXiv:2608.03063v1 Announce Type: new Abstract: Merchant risk control at large payment platforms screens tens of millions of merchants daily, where false positives harm legitimate merchants and false

When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings

model-releasesarxiv-cs-cl

arXiv:2608.03994v1 Announce Type: new Abstract: We identify a previously overlooked failure mode of ALiBi positional encoding: its linear bias scaling underflows floating-point precision, which zeroes

Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data

arXiv:2608.02580v1 Announce Type: new Abstract: Learning generalizable robot manipulation policies requires large-scale and diverse demonstration data. Egocentric human manipulation videos offer rich

researcharxiv-cs-ro

Faster-WAM: Do World Action Models Need Deep Action Modules?

arXiv:2608.02365v1 Announce Type: cross Abstract: World Action Models (WAMs) couple robot action prediction with video world models. Existing WAMs with shared-backbone and Mixture-of-Transformers desi

From Digital to Physical Reservoir Computing: Co-Optimizing Soft Robotic Reservoirs via Dynamics Matching

arXiv:2608.00484v1 Announce Type: cross Abstract: Soft robotic substrates are promising for Physical Reservoir Computing (PRC) because their compliant nonlinear dynamics can provide temporal memory, h

InstancePin: Instance-Addressable Layout-to-Image Diffusion via Coordinate Pinning

arXiv:2608.00588v1 Announce Type: new Abstract: Layout-to-image diffusion models have achieved impressive semantic controllability by conditioning generation on category-level segmentation maps. Howev

researcharxiv-cs-cv

TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning

arXiv:2510.03519v2 Announce Type: replace Abstract: Time series reasoning is crucial to decision-making in diverse domains, including finance, energy, and scientific discovery. While existing time ser

safetyarxiv-cs-cl

A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples

arXiv:2607.29122v1 Announce Type: new Abstract: Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels. This is challenging because a single model must capture both

Epistemic-aware Vision-Language Foundation Model for Fetal Ultrasound Interpretation

arXiv:2510.12953v4 Announce Type: replace-cross Abstract: Recent medical vision-language models have shown promise on tasks such as VQA, report generation, and anomaly detection. However, most are ada

Tutorials

PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction

arXiv:2607.29000v1 Announce Type: cross Abstract: Multimodal information can improve the accuracy of click-through rate (CTR) prediction and effectively alleviate item cold-start and long-tail problem

tutorialsarxiv-cs-lg

RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving

arXiv:2602.07339v2 Announce Type: replace Abstract: Diffusion-based trajectory planners can model multi-modal driving behavior, but their iterative denoising process introduces a latency bottleneck fo

safetyarxiv-cs-ai

Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution

arXiv:2605.25333v2 Announce Type: replace Abstract: Video world models should maintain evolving states when evidence is unobserved, yet current generators often freeze hidden states upon interruption.

Arm2Air: Cross-Embodiment Skeleton Transfer for 3D Relay Formation

arXiv:2607.27627v1 Announce Type: new Abstract: Unmanned aerial vehicle (UAV) relay networks can restore connectivity after communication infrastructure is damaged. Urban relay placement is difficult

researcharxiv-cs-ro

Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models

arXiv:2607.27386v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refi

DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement

arXiv:2607.27761v1 Announce Type: cross Abstract: In recent years, multi-view clustering has attracted widespread research interest. However, due to limitations in data collection devices, data across

safetyarxiv-cs-cv

ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring

arXiv:2607.27671v1 Announce Type: new Abstract: The majority of the recently-developed models for automated essay scoring (AES) are evaluated solely on the ASAP corpus. However, ASAP is not without it

researcharxiv-cs-cl

LM-GRASP: Instance-Specific Language Models for Combinatorial Construction via Online Imitation Learning

arXiv:2607.28135v1 Announce Type: new Abstract: Machine learning for combinatorial optimization typically relies on neural constructors trained via reinforcement learning on large offline datasets for

PrintAnything: Learning an Intermediate Representation for 3D printing G-code Generation

arXiv:2607.27729v1 Announce Type: new Abstract: Point clouds are one of the most fundamental and widely used 3D representations, serving as the most basic geometric representation of 3D shapes. Nevert

researcharxiv-cs-cv

Agents

RadHarmony: Radiological Data Handling in the Era of Agentic AI

arXiv:2607.27235v1 Announce Type: cross Abstract: Training deep learning models on radiological images requires integrating heterogeneous datasets across different sources, file formats, directory lay

agentsarxiv-cs-cv

Tutorials

ReDiff: Reliability-Guided Diffusion for Trustworthy Ultra-Low-Field to High-Field MRI Synthesis

arXiv:2603.11325v2 Announce Type: replace Abstract: Low-field to high-field MRI synthesis has emerged as a promising strategy to improve image quality when access to high-field scanners is limited. Ho

tutorialsarxiv-cs-cv

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

arXiv:2603.13707v3 Announce Type: replace-cross Abstract: Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-

safetyarxiv-cs-lg

Region-adaptable retrieval of coastal biogeochemical parameters from near-surface hyperspectral remote sensing reflectance using physics-aware meta-learning

arXiv:2605.05623v2 Announce Type: replace Abstract: Hyperspectral in situ sensing has shown promise in retrieving aquatic biogeochemical (BGC) parameters, such as total suspended solids, dissolved org

Agents

RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents

arXiv:2607.27881v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have attracted growing interest as a scalable approach to robotic manipulation. While these models are effective act

agentsarxiv-cs-ro