Search: “pretraining”

Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning

arXiv:2606.03817v1 Announce Type: new Abstract: Idioms can be analysed in terms of their decomposability, the extent to which constituent meanings contribute to the figurative whole. Decomposability i

researcharxiv-cs-cl

3 Jun 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

TrAction: Action Recognition with Sparse Trajectories

DGX agent

arXiv:2606.03490v1 Announce Type: new Abstract: Modern action recognition models operate on memory- and compute-intensive dense RGB video volumes and frequently exploit appearance and background short

researcharxiv-cs-cv

3 Jun 2026

Tutorials

Your Autoregressive Model Already Reveals the Causal Graph

arXiv:2602.01135v3 Announce Type: replace Abstract: Autoregressive models trained via next-token prediction implicitly learn the conditional independence structure of their data-generating process. We

tutorialsarxiv-cs-lg

3 Jun 2026

Bridging Topology and Deep Representation Learning: A TDA-ViT Fusion Model for Four-Class Brain Tumor Classification

arXiv:2606.00927v1 Announce Type: new Abstract: Accurate brain tumor classification from magnetic resonance imaging (MRI) is a key requirement for early diagnosis and clinical decision-making. Vision

Coarse-to-Fine Compositional Diffusion for Long-Horizon Planning

arXiv:2606.00837v1 Announce Type: cross Abstract: Diffusion models provide strong priors for generating structured data, but many tasks require outputs beyond the scale on which these models are typic

local-aiarxiv-cs-lg

DerMAE: Improving skin lesion classification through conditioned latent diffusion and MAE distillation

arXiv:2602.19848v2 Announce Type: replace Abstract: Skin lesion classification datasets often suffer from severe class imbalance, with malignant cases significantly underrepresented, leading to biased

local-aiarxiv-cs-cv

Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies

arXiv:2508.20072v4 Announce Type: replace Abstract: Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions. However, prevailing VLA

DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs

arXiv:2606.01024v1 Announce Type: cross Abstract: Discrete Masked diffusion language models generate text by iterative parallel decoding, but few-step decoding suffers from a tradeoff between length a

local-aiarxiv-cs-ai

Efficient Weighted Sampling via Score-based Generative Models

arXiv:2502.04646v2 Announce Type: replace-cross Abstract: Weighted sampling -- sampling from a probability density function (PDF) proportional to the product of a base PDF and a weight function -- is

Feature to Dynamics: Feature-space to Autoregression strategy for Zero-shot Time Series Forecasting

arXiv:2606.01289v1 Announce Type: new Abstract: Zero-shot time series forecasting aims to predict future values for previously unseen series, requiring models to generalize temporal dynamics beyond th

Heterogeneous Decentralized Diffusion Models

arXiv:2603.06741v2 Announce Type: replace-cross Abstract: Training frontier-scale diffusion models often requires substantial computational resources concentrated in tightly-coupled clusters, limiting

local-aiarxiv-cs-ai

Improving Visual Representation Alignment Generation with GRPO

arXiv:2606.00583v1 Announce Type: cross Abstract: Recent diffusion transformers have demonstrated strong image synthesis capabilities but remain inefficient to train due to weak alignment between gene

safetyarxiv-cs-ai

IndoBias: A Dual Track Culturally Grounded Benchmark for LLMs Bias Evaluation in Indonesian Languages

arXiv:2606.01260v1 Announce Type: cross Abstract: Despite being home to more than 1300 ethnic groups and 700 indigenous languages, bias in Large Language Models has not been fully studied in Indonesia

MOSS-Audio Technical Report

arXiv:2606.01802v1 Announce Type: cross Abstract: MOSS-Audio is a unified audio-language model for speech, environmental sound, and music understanding, supporting audio captioning, time-aware questio

OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models

arXiv:2606.00815v1 Announce Type: new Abstract: Electroencephalography (EEG) supports a variety of brain-computer interface (BCI) tasks ranging from brain-state monitoring to human-LLM interactions. E

Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning

arXiv:2603.11653v2 Announce Type: replace Abstract: Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that c

Task Structure Reverses Layerwise State Encoding in Sequence Models

arXiv:2606.00926v1 Announce Type: cross Abstract: Mechanistic studies of sequence models often treat layerwise state encodings as architectural traits: recurrent models concentrate readable state, att

researcharxiv-cs-cl

Agents

Towards a General Intelligence and Interface for Wearable Health Data

arXiv:2605.22759v2 Announce Type: replace Abstract: While ubiquitous wearable sensors capture a wealth of behavioral and physiological information, effectively transforming these signals into personal

agentsarxiv-cs-ai

Chem-PerturBridge: a harmonized compendium of small molecule perturbation transcriptomic effects

arXiv:2605.31522v1 Announce Type: new Abstract: Large perturbation models require training data encompassing chemical, cellular, and assay diversity. Current transcriptomic resources for small-molecul

researcharxiv-cs-lg

Learning Transferable Predictability Representations

arXiv:2605.30592v1 Announce Type: new Abstract: We study the problem of assigning a scalar score to a short trajectory window that reflects its position on an ordered continuum of predictability regim

local-aiarxiv-cs-lg

LLMs Lean on Priors, Not Programming Language Semantics

arXiv:2510.03415v3 Announce Type: replace-cross Abstract: Recent work asks whether large language models (LLMs) condition their reasoning on explicit rules rather than statistical regularities from pr

UniRTL: Unifying Code and Graph for Robust RTL Representation Learning

arXiv:2605.31040v1 Announce Type: new Abstract: Developing effective representations for register transfer level (RTL) designs is crucial for accelerating the hardware design workflow. Existing approa

Archon: A Unified Multimodal Model for Holistic Digital Human Generation

arXiv:2605.30311v1 Announce Type: cross Abstract: Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visu

Beyond Recall: Behavioral Specification as an Interpretive Layer for AI Personalization

arXiv:2605.28969v1 Announce Type: cross Abstract: If an AI agent makes decisions on a person's behalf, those decisions must align with its user. We introduce representational accuracy to measure how f

CLUBench: A Clustering Benchmark

arXiv:2605.29933v1 Announce Type: new Abstract: Clustering is a fundamental problem in data science with a long-standing research history, yielding numerous insightful algorithms. Despite this progres

ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression

arXiv:2605.29350v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models reduce per-token computation but still require storing and serving all experts, making deployment memory-intens

Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning

arXiv:2506.05985v3 Announce Type: replace Abstract: A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing catastrophic for

IP-Adapter Is All You Need: Towards Fine-Tuning-Free Diffusion-Based Talking Face Generation

arXiv:2605.30230v1 Announce Type: new Abstract: With the rapid advancement of diffusion models, talking face generation has made remarkable progress. However, existing diffusion-based methods still re

Agents

KairosAgent: Agentic Time Series Forecasting with Fused Semantic Reasoning

arXiv:2605.30002v1 Announce Type: new Abstract: Cross-domain multimodal time series forecasting is a challenging task, requiring models to integrate precise numerical comprehension, cross-domain seman

agentsarxiv-cs-ai

Large language models reorganize representational geometry during in-context learning

model-releasesarxiv-cs-cl

arXiv:2605.28854v1 Announce Type: new Abstract: Large language models (LLMs) exhibit remarkable flexibility: they can adapt to novel tasks from in-context examples without any parameter updates, a cap

MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection

arXiv:2605.30288v1 Announce Type: new Abstract: Mid-training has become an important stage in modern LLM development, using large-scale curated mixtures to strengthen capabilities before final post-tr

Plan, Don't Pose: Long Composite Motion Generation with Text-Aligned BFM

arXiv:2605.29906v1 Announce Type: new Abstract: Text-to-motion (T2M) generation has broad applications in character animation, virtual avatars, and human-robot interaction. Existing methods typically

Tutorials

Robust Cross-Domain Generalization Using Unlabeled Target Data with Source-Domain Supervision

arXiv:2605.29122v1 Announce Type: new Abstract: It is often desirable to generalize medical imaging AI models trained with dense annotations to data acquired from different ultrasound scanners or clin

tutorialsarxiv-cs-cv

The Impact of Semantic Pairs on Self-Supervised Representation Learning

arXiv:2510.08722v3 Announce Type: replace-cross Abstract: Instance discrimination learns visual representations by treating different augmented views of the same image as positive pairs. While this en

Turbulence-Robust Dynamic Object Segmentation with Multi-Signal Priors and SAM2 Refinement

arXiv:2605.29292v1 Announce Type: new Abstract: This technical report presents our solution for the CVPR 2026 UG2+ Challenge Track 3: Dynamic Object Segmentation in Turbulence (DOST). We design a trai

Hardware

VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion

arXiv:2605.30351v1 Announce Type: cross Abstract: Long-rollout causal video diffusion has converged on a fixed-size sliding-window KV cache, with recent progress innovating within this layout by chang

hardwarearxiv-cs-ai

Applications

A self-supervised learning approach to deep filter banks for texture recognition

arXiv:2605.27843v1 Announce Type: new Abstract: An important challenge in texture recognition is the limited amount of data for training frequently found in real-world applications. In computer vision

applicationsarxiv-cs-cv

NL-MambaXCT: Self-Supervised Nested-Learning Mamba for Nomex Honeycomb X-ray CT Defect Classification

model-releasesarxiv-cs-cv

arXiv:2605.27454v1 Announce Type: cross Abstract: X-ray computed tomography (XCT) is widely used for non-destructive testing of Nomex honeycomb structures in aerospace manufacturing, but industrial in

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective

model-releasesarxiv-cs-cl

arXiv:2605.28819v1 Announce Type: cross Abstract: Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstrea

VLMs May Not Globally Enhance Human Alignment over LLMs During Natural Reading

arXiv:2605.28818v1 Announce Type: new Abstract: Large language models (LLMs) have become increasingly useful computational models of human language processing, but it remains unclear whether vision-la

safetyarxiv-cs-cl

Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization

arXiv:2605.26282v1 Announce Type: new Abstract: Model-based reinforcement learning (RL) can be effectively supported at scale through the use of world models. However, in practice, scaling such approa

27 May 2026

The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context

arXiv:2605.26778v1 Announce Type: new Abstract: Retrieval-augmented generation promises to ground language model outputs in external evidence, yet the field has no reliable way to verify whether retri

27 May 2026

BandVQ: Band-Wise Vector-Quantized EEG Foundation Model

arXiv:2605.24921v1 Announce Type: new Abstract: A central challenge in electroencephalography (EEG) foundation modeling is learning transferable representations across recordings with diverse tasks, m

researcharxiv-cs-lg

Causal methods for LLM development and evaluation

arXiv:2605.25998v1 Announce Type: new Abstract: Large language model (LLM) development is currently driven by large-scale empirical iteration over data mixtures, reward models, routing strategies, and

ECo-MoE: Embodiment-Conditioned Mixture of Experts Increases the Evolvability of Robots

arXiv:2605.24225v1 Announce Type: new Abstract: In this paper, we introduce a model of evolution and learning in robots that co-optimizes a distribution of latent design vectors (genotypes) and a mixt

safetyarxiv-cs-ro

From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks

arXiv:2605.24771v1 Announce Type: cross Abstract: Classical noisy-label theory predicts that downstream performance under weak supervision is bounded above by the labeler's accuracy, implying a sharp

Applications

LWM-CDE: A Representation Space for Wireless Data Reasoning and Transferability

arXiv:2605.24077v1 Announce Type: cross Abstract: Machine learning deployments in real-world wireless communication tasks face significant generalization challenges due to location and environment-spe

applicationsarxiv-cs-lg

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs

arXiv:2605.24681v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine