Search: “pretraining”

Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance

arXiv:2605.31304v1 Announce Type: cross Abstract: Deep neural networks (DNNs) are widely used, but interpreting what they actually learn remains difficult. A major obstacle is that individual neurons

tutorialsarxiv-cs-cv

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Inverting Data Transformations via Diffusion Sampling

DGX agent

arXiv:2602.08267v2 Announce Type: replace-cross Abstract: We study the problem of transformation inversion on general Lie groups: a datum is transformed by an unknown group element, and the goal is to

researcharxiv-cs-ai

1 Jun 2026

Knowledge Graph-Enhanced Zero-Shot Topic Classification: A Multi-Strategy Comparative Study

arXiv:2605.30465v1 Announce Type: new Abstract: Multi-label topic classification without labeled training data is a challenging task, specially when documents contain complex relational information. W

researcharxiv-cs-cl

Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions

arXiv:2605.31586v1 Announce Type: cross Abstract: Grasping the semantics of rare constructions (form-meaning pairings) has been shown to be a challenging problem that has currently only been solved by

Linear Scaling Video VLMs for Long Video Understanding

arXiv:2605.31598v1 Announce Type: new Abstract: Video vision-language models (VLMs) are increasingly used in long-horizon and streaming settings, yet most video encoders still rely on spatiotemporal s

Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Models

arXiv:2605.31603v1 Announce Type: cross Abstract: Connector-based video unified models have demonstrated strong capability in instruction-grounded video synthesis, but integrating a large high-fidelit

tutorialsarxiv-cs-ai

Memory by Design: Probabilistic Sequence Layers

arXiv:2605.31163v1 Announce Type: cross Abstract: We introduce the design-model framework: a way to derive efficient recurrent sequence maps from explicit assumptions about memory. A design model writ

MultiAct: Text-to-Motion Generation from Composite Text via Tailored Attention Guidance

arXiv:2605.30925v1 Announce Type: new Abstract: Text-to-motion generation has progressed rapidly in recent years, offering an expressive interface for animation and human-computer interaction. However

Not All Synthetic Data Is Yours to Learn From

arXiv:2605.31126v1 Announce Type: cross Abstract: Can a language model improve from plain text sampled from itself, with no prompts, no teacher, no verifier, and no reward model? Yes, but only when th

On Revisiting Entropy for Identifying Mislabeled Images

arXiv:2605.31090v1 Announce Type: cross Abstract: Mislabeled samples in training datasets severely degrade the performance of deep networks, as overparameterized models tend to memorize erroneous labe

researcharxiv-cs-ai

Applications

Prior Availability in Industrial Visual Sim-to-Real: A Review of CAD-Guided and CAD-Unavailable Regimes

arXiv:2605.30581v1 Announce Type: cross Abstract: Industrial visual sim-to-real is often described as transferring from synthetic images to real images, but industrial deployment usually involves a br

applicationsarxiv-cs-ai

Quantifying the Uncertainty of Foundation Models with Singular Value Ensembles

arXiv:2601.22068v2 Announce Type: replace Abstract: Foundation models have become a dominant paradigm in machine learning, achieving remarkable performance across diverse tasks through large-scale pre

Representation Forcing for Bottleneck-Free Unified Multimodal Models

arXiv:2605.31604v1 Announce Type: new Abstract: Unified multimodal models (UMMs) aim to handle perception and generation in a single model. Yet existing UMMs still rely on a frozen, separately pretrai

tutorialsarxiv-cs-cv

Applications

SKETCH: Semantic Key-Point Conditioning for Long-Horizon Vessel Trajectory Prediction

arXiv:2601.18537v3 Announce Type: replace-cross Abstract: Accurate long-horizon vessel trajectory prediction remains challenging due to compounded uncertainty from complex navigation behaviors and env

applicationsarxiv-cs-ai

Softsign: Smooth Sign in Your Optimizer For Better Parameter Heterogeneity Handling

arXiv:2605.31371v1 Announce Type: new Abstract: Sign-based and LMO-inspired optimizers have recently attracted substantial attention in deep learning due to their strong performance and low memory foo

The Gaussian-Head OFL Family: One-Shot Federated Learning from Client Global Statistics

arXiv:2602.01186v2 Announce Type: replace-cross Abstract: Classical Federated Learning relies on a multi-round iterative process of model exchange and aggregation between server and clients, with high

researcharxiv-cs-ai

The Regularizing Power of Language-Training Deepfake Detectors

model-releasesarxiv-cs-cv

arXiv:2605.31192v1 Announce Type: new Abstract: Recently, thanks to the advent of Multimodal-LLMs, deepfake detectors are striving not only to be generalizable but also interpretable. We propose that

Unicorn: Scaling High-Dimensional Time Series Forecasting via Universal Correlation Modeling

arXiv:2605.30376v1 Announce Type: cross Abstract: Modern time series architectures face a fundamental trade-off: channel-independent models scale well with increasing data volume but ignore critical i

researcharxiv-cs-ai

Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification

arXiv:2509.24901v4 Announce Type: replace-cross Abstract: Although probing frozen models has become a standard evaluation paradigm, self-supervised learning in audio defaults to fine-tuning when pursu

When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework

arXiv:2605.31504v1 Announce Type: new Abstract: Multimodal models in oncology can produce accurate predictions, but accurate prediction does not reveal whether the model has learned biology that is sh

3DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding

arXiv:2605.29416v1 Announce Type: cross Abstract: Vision-Language-Action models have achieved remarkable progress in robotic manipulation, yet they suffer from a critical limitation: a lack of 3D scen

Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation

arXiv:2605.29992v1 Announce Type: new Abstract: Sentence embeddings are a foundational component for semantic search, clustering, classification, and retrieval-augmented generation. This paper present

BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models

arXiv:2605.30226v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for grounding visual-language understanding into real-world robotic manipulat

safetyarxiv-cs-ai

BrahmicTokenizer-131K: An Indic-Capable Drop-In Replacement for o200k_base

arXiv:2605.29379v1 Announce Type: new Abstract: We present BrahmicTokenizer-131K, a 131,072-vocabulary byte-level BPE tokenizer that closes the Brahmic compression gap at the 131K-vocabulary class whi

Agents

Code-QA-Bench: Separating Code Reasoning from Documentation Memorization in Repository-Level QA

arXiv:2605.29277v1 Announce Type: cross Abstract: We present Code-QA-Bench, a fully automated framework for synthesizing repository-level code understanding benchmarks that separates genuine code comp

agentsarxiv-cs-ai

CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective

arXiv:2502.03805v2 Announce Type: replace Abstract: Large language models have revolutionized natural language processing but face significant challenges of high storage and runtime costs, due to the

Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model

arXiv:2510.27607v3 Announce Type: replace Abstract: Augmenting vision-language-action models (VLAs) with world models is promising for robotic policy learning but faces challenges in jointly predictin

safetyarxiv-cs-cv

Eigen-Spike Emergence and Quadratic Equivalents for Conjugate Kernels on Nonlinearly Separable Data

arXiv:2605.29669v1 Announce Type: cross Abstract: Recent work in random matrix theory (RMT) has developed the notion of deterministic equivalents: typically linear surrogate models that approximate th

Enhancing LLM Training via Spectral Clipping

arXiv:2603.14315v2 Announce Type: replace Abstract: While spectral-based optimizers like Muon operate directly on the spectrum of updates, standard adaptive methods such as AdamW do not account for th

Ensemble Score Filtering for Real-Data Energy Consumption Forecast Correction

arXiv:2605.29072v1 Announce Type: new Abstract: Accurate estimation and forecasting of energy consumption are important for power-system operation, planning, and demand-side management. In practice, h

EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance

arXiv:2505.21876v2 Announce Type: replace-cross Abstract: Recent approaches for video generation with camera control often create anchor videos (i.e., rendered videos that approximate desired camera m

safetyarxiv-cs-ai

Genetically Aligned Patient Representations Improve Hematological Diagnosis

arXiv:2605.29980v1 Announce Type: cross Abstract: Multimodal alignment of histopathology encoders with transcriptomic and genomic data has been shown to significantly improve performance in downstream

safetyarxiv-cs-ai

Gradient Preconditioning for Efficient and Reliable Reward-Guided Generation

arXiv:2602.08646v2 Announce Type: replace Abstract: We propose a gradient preconditioning method that makes reward-guided generation with one-step generative models both efficient and reliable. Test-t

How's it going? Reinforcement learning in language models recruits a functional welfare axis

arXiv:2605.30232v1 Announce Type: cross Abstract: How does reinforcement learning shape a language model's internal representations? We present evidence that RL recruits a pre-existing representation

Inferring the Size of Large Language Models From Popular Text Memorization

arXiv:2605.29223v1 Announce Type: new Abstract: The parameter counts of the most widely used large language models (LLMs) are often withheld by their developers, leaving model size -- a primary refere

Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules

arXiv:2605.29075v1 Announce Type: new Abstract: LLMs encode both general capabilities and domain-specific knowledge in a single set of parameters. We ask whether this capacity can be reorganized: keep

Label-Free Reinforcement Learning via Cross-Model Entropy

arXiv:2605.29009v1 Announce Type: cross Abstract: Post-training large language models with reinforcement learning is bottlenecked by the reward signal. Existing approaches require either ground-truth

Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning

arXiv:2605.28990v1 Announce Type: new Abstract: Functional magnetic resonance imaging (fMRI) is a powerful tool for investigating human brain function. However, the high cost of data acquisition and t

tutorialsarxiv-cs-lg

Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance

arXiv:2411.14279v2 Announce Type: replace-cross Abstract: Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. However, despite showing promising per

Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting

arXiv:2605.29498v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models to new domains, tasks, and u

Metric-Dependent Annotation Saturation for Learning from Label Distributions

arXiv:2605.29797v1 Announce Type: new Abstract: When annotators disagree on a label, the disagreement itself carries signal -- and the number of annotators needed to capture it depends on the evaluati

Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learning

arXiv:2605.29577v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have emerged as a promising framework that unifies perception, reasoning, and control for robot manipulation by adap

safetyarxiv-cs-cv

Mitigating Stethoscope-Induced Shortcuts in Respiratory Sound Classification under Federated Domain Generalization with Causality-Inspired Interventions

arXiv:2605.29862v1 Announce Type: cross Abstract: AI-driven respiratory sound classification (RSC) is promising for automated pulmonary disease detection, yet multi-site deployment is hindered by inte

Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions

arXiv:2605.29738v1 Announce Type: cross Abstract: Legal NLP benchmarks overwhelmingly evaluate a single language or aggregate tasks that differ fundamentally across jurisdictions, making cross-lingual

Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

arXiv:2605.30280v1 Announce Type: cross Abstract: Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented cap

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

arXiv:2605.28827v1 Announce Type: new Abstract: Open Arabic large language models split into two classes: sub-1B multilingual models that treat Arabic as an afterthought (Qwen2.5-0.5B, Falcon-H1-0.5B)

Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets

arXiv:2605.30289v1 Announce Type: new Abstract: Numeric tabular datasets are the dominant data format in scientific practice, yet large language models lack native mechanisms for representing numeric

safetyarxiv-cs-lg

SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation

arXiv:2605.29655v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) enable 3D generation but struggle to scale to high-resolution shapes due to inadequate 3D tokeni