Search: “pretraining”

3DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding

arXiv:2605.29416v1 Announce Type: cross Abstract: Vision-Language-Action models have achieved remarkable progress in robotic manipulation, yet they suffer from a critical limitation: a lack of 3D scen

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation

DGX agent

arXiv:2605.29992v1 Announce Type: new Abstract: Sentence embeddings are a foundational component for semantic search, clustering, classification, and retrieval-augmented generation. This paper present

model-releasesarxiv-cs-cl

29 May 2026

BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models

arXiv:2605.30226v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for grounding visual-language understanding into real-world robotic manipulat

BrahmicTokenizer-131K: An Indic-Capable Drop-In Replacement for o200k_base

model-releasesarxiv-cs-cl

arXiv:2605.29379v1 Announce Type: new Abstract: We present BrahmicTokenizer-131K, a 131,072-vocabulary byte-level BPE tokenizer that closes the Brahmic compression gap at the 131K-vocabulary class whi

Agents

Code-QA-Bench: Separating Code Reasoning from Documentation Memorization in Repository-Level QA

arXiv:2605.29277v1 Announce Type: cross Abstract: We present Code-QA-Bench, a fully automated framework for synthesizing repository-level code understanding benchmarks that separates genuine code comp

agentsarxiv-cs-ai

CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective

model-releasesarxiv-cs-cl

arXiv:2502.03805v2 Announce Type: replace Abstract: Large language models have revolutionized natural language processing but face significant challenges of high storage and runtime costs, due to the

Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model

arXiv:2510.27607v3 Announce Type: replace Abstract: Augmenting vision-language-action models (VLAs) with world models is promising for robotic policy learning but faces challenges in jointly predictin

safetyarxiv-cs-cv

Eigen-Spike Emergence and Quadratic Equivalents for Conjugate Kernels on Nonlinearly Separable Data

arXiv:2605.29669v1 Announce Type: cross Abstract: Recent work in random matrix theory (RMT) has developed the notion of deterministic equivalents: typically linear surrogate models that approximate th

Enhancing LLM Training via Spectral Clipping

arXiv:2603.14315v2 Announce Type: replace Abstract: While spectral-based optimizers like Muon operate directly on the spectrum of updates, standard adaptive methods such as AdamW do not account for th

Ensemble Score Filtering for Real-Data Energy Consumption Forecast Correction

arXiv:2605.29072v1 Announce Type: new Abstract: Accurate estimation and forecasting of energy consumption are important for power-system operation, planning, and demand-side management. In practice, h

EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance

arXiv:2505.21876v2 Announce Type: replace-cross Abstract: Recent approaches for video generation with camera control often create anchor videos (i.e., rendered videos that approximate desired camera m

Genetically Aligned Patient Representations Improve Hematological Diagnosis

arXiv:2605.29980v1 Announce Type: cross Abstract: Multimodal alignment of histopathology encoders with transcriptomic and genomic data has been shown to significantly improve performance in downstream

Gradient Preconditioning for Efficient and Reliable Reward-Guided Generation

arXiv:2602.08646v2 Announce Type: replace Abstract: We propose a gradient preconditioning method that makes reward-guided generation with one-step generative models both efficient and reliable. Test-t

How's it going? Reinforcement learning in language models recruits a functional welfare axis

arXiv:2605.30232v1 Announce Type: cross Abstract: How does reinforcement learning shape a language model's internal representations? We present evidence that RL recruits a pre-existing representation

Inferring the Size of Large Language Models From Popular Text Memorization

model-releasesarxiv-cs-lg

arXiv:2605.29223v1 Announce Type: new Abstract: The parameter counts of the most widely used large language models (LLMs) are often withheld by their developers, leaving model size -- a primary refere

Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules

model-releasesarxiv-cs-lg

arXiv:2605.29075v1 Announce Type: new Abstract: LLMs encode both general capabilities and domain-specific knowledge in a single set of parameters. We ask whether this capacity can be reorganized: keep

Label-Free Reinforcement Learning via Cross-Model Entropy

arXiv:2605.29009v1 Announce Type: cross Abstract: Post-training large language models with reinforcement learning is bottlenecked by the reward signal. Existing approaches require either ground-truth

Tutorials

Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning

arXiv:2605.28990v1 Announce Type: new Abstract: Functional magnetic resonance imaging (fMRI) is a powerful tool for investigating human brain function. However, the high cost of data acquisition and t

tutorialsarxiv-cs-lg

Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance

arXiv:2411.14279v2 Announce Type: replace-cross Abstract: Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. However, despite showing promising per

Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting

arXiv:2605.29498v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models to new domains, tasks, and u

Metric-Dependent Annotation Saturation for Learning from Label Distributions

arXiv:2605.29797v1 Announce Type: new Abstract: When annotators disagree on a label, the disagreement itself carries signal -- and the number of annotators needed to capture it depends on the evaluati

Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learning

arXiv:2605.29577v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have emerged as a promising framework that unifies perception, reasoning, and control for robot manipulation by adap

safetyarxiv-cs-cv

Mitigating Stethoscope-Induced Shortcuts in Respiratory Sound Classification under Federated Domain Generalization with Causality-Inspired Interventions

arXiv:2605.29862v1 Announce Type: cross Abstract: AI-driven respiratory sound classification (RSC) is promising for automated pulmonary disease detection, yet multi-site deployment is hindered by inte

Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions

arXiv:2605.29738v1 Announce Type: cross Abstract: Legal NLP benchmarks overwhelmingly evaluate a single language or aggregate tasks that differ fundamentally across jurisdictions, making cross-lingual

Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

arXiv:2605.30280v1 Announce Type: cross Abstract: Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented cap

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

model-releasesarxiv-cs-cl

arXiv:2605.28827v1 Announce Type: new Abstract: Open Arabic large language models split into two classes: sub-1B multilingual models that treat Arabic as an afterthought (Qwen2.5-0.5B, Falcon-H1-0.5B)

Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets

arXiv:2605.30289v1 Announce Type: new Abstract: Numeric tabular datasets are the dominant data format in scientific practice, yet large language models lack native mechanisms for representing numeric

SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation

arXiv:2605.29655v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) enable 3D generation but struggle to scale to high-resolution shapes due to inadequate 3D tokeni

Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods

model-releasesarxiv-cs-cv

arXiv:2601.12500v2 Announce Type: replace Abstract: Counting and tracking dense crowds in large-scale scenes is a highly practical yet challenging problem. Existing methods mostly rely on fixed-camera

VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2605.29605v1 Announce Type: new Abstract: Confidence estimation for Vision-Language-Action (VLA) models is essential for robots to perform manipulation tasks in the open world, providing crucial

Tutorials

Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention

arXiv:2605.29548v1 Announce Type: new Abstract: Larger models learn tasks smaller models do not. What drives this phenomenon? We develop a simple phenomenological argument that power-law scaling alrea

tutorialsarxiv-cs-lg

A Factory-Floor Deployment Case Study of VLA Pipelines for Industrial Packaging Task: Workflow, Failures, and Lessons

arXiv:2605.27461v1 Announce Type: new Abstract: Vision-Language-Action (VLA) policies have shown promising manipulation capabilities, yet their practical impact is often limited by the reliability dem

safetyarxiv-cs-ro

A Structural Theory of Position Bias in Transformers

arXiv:2602.16837v2 Announce Type: replace Abstract: Transformer models systematically favor certain token positions, yet the architectural origins of this position bias remain poorly understood. This

Local Ai

An Evolutionary Approach for Designing Stable and Highly Expressible Low-Immunogenicity Therapeutic mRNA Sequences

arXiv:2605.27986v1 Announce Type: new Abstract: Messenger RNA (mRNA) sequences as therapeutics require optimized design to ensure efficient translation, structural stability, and minimal immunogenicit

local-aiarxiv-cs-cl

Applications of temporal graph learning for predicting the dynamics of biological systems

arXiv:2605.28659v1 Announce Type: new Abstract: Biological foundation models have shown strong performance in single-cell representation learning by applying transformer architectures directly to gene

Agents

AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning

arXiv:2605.28287v1 Announce Type: new Abstract: Discovering novel stable molecules without training data remains a grand scientific challenge. Current molecular generative models are trained on large,

agentsarxiv-cs-lg

Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity

model-releasesarxiv-cs-lg

arXiv:2605.28640v1 Announce Type: new Abstract: Efficient inference is critical for long-context language models, where attention computation and KV-cache access dominate the cost. Recent work RAT+, i

Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?

arXiv:2605.27733v1 Announce Type: new Abstract: Training instabilities such as loss spikes are frequently the result of stochastic gradient noise. Because of rare expressions in language training data

COTTA: Context-Aware Transfer Adaptation for Trajectory Prediction in Autonomous Driving

arXiv:2604.00402v2 Announce Type: replace-cross Abstract: Developing robust models to accurately predict the trajectories of surrounding agents is fundamental to autonomous driving safety. However, mo

Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies

arXiv:2605.24302v2 Announce Type: replace Abstract: Egocentric action recognition is a challenging task due to erratic camera motion, frequent hand occlusion, and the difficulty of maintaining consist

DebFilter: Eradicating Biases Stashed in Value

arXiv:2605.28167v1 Announce Type: new Abstract: Text-to-image diffusion models, which are theoretically equivalent to score-based generative models, generate images through a multi-step denoising proc

safetyarxiv-cs-cv

Explicit Critic Guidance for Aligning Diffusion Models

arXiv:2605.27736v1 Announce Type: cross Abstract: Online reinforcement learning is becoming increasingly important for aligning diffusion models with non-differentiable objectives. However, existing m

Fine-Tuning Dynamics of In-Context Factual Recall in Transformers

arXiv:2605.27774v1 Announce Type: new Abstract: In-context learning -- performing tasks based on examples given in the prompt -- is an important capability that has emerged in large language models an

Imitation Learning for Robot Assistance in Open Surgery: A Multi-Policy Evaluation on Suture Following

model-releasesarxiv-cs-ro

arXiv:2605.28736v1 Announce Type: new Abstract: This study presents the first evaluation of general-purpose imitation learning for surgeon-robot collaborative assistance in open surgery, targeting sut

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

arXiv:2605.27583v1 Announce Type: new Abstract: Electrocardiograms (ECGs) are widely used non-invasive measurements of cardiac activity and play a central role in clinical diagnosis. Recent multimodal

Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration

arXiv:2605.28184v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as the standard paradigm for improving reasoning capability of large language models,

Tutorials

Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images

arXiv:2605.28693v1 Announce Type: cross Abstract: Backpropagation is the core learning mechanism underlying deep learning. However, whether and how this algorithm is implemented in the brain remains h

tutorialsarxiv-cs-ai

Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction

arXiv:2605.27878v1 Announce Type: new Abstract: Large language models produce fluent fiction, yet their creative output is widely seen as flat. We ask where this quality originates in the training and