Search: “pretraining”

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective

model-releasesarxiv-cs-cl

arXiv:2605.28819v1 Announce Type: cross Abstract: Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstrea

28 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

VLMs May Not Globally Enhance Human Alignment over LLMs During Natural Reading

DGX agent

arXiv:2605.28818v1 Announce Type: new Abstract: Large language models (LLMs) have become increasingly useful computational models of human language processing, but it remains unclear whether vision-la

safetyarxiv-cs-cl

28 May 2026

Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization

arXiv:2605.26282v1 Announce Type: new Abstract: Model-based reinforcement learning (RL) can be effectively supported at scale through the use of world models. However, in practice, scaling such approa

27 May 2026

The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context

arXiv:2605.26778v1 Announce Type: new Abstract: Retrieval-augmented generation promises to ground language model outputs in external evidence, yet the field has no reliable way to verify whether retri

27 May 2026

BandVQ: Band-Wise Vector-Quantized EEG Foundation Model

arXiv:2605.24921v1 Announce Type: new Abstract: A central challenge in electroencephalography (EEG) foundation modeling is learning transferable representations across recordings with diverse tasks, m

Causal methods for LLM development and evaluation

arXiv:2605.25998v1 Announce Type: new Abstract: Large language model (LLM) development is currently driven by large-scale empirical iteration over data mixtures, reward models, routing strategies, and

ECo-MoE: Embodiment-Conditioned Mixture of Experts Increases the Evolvability of Robots

arXiv:2605.24225v1 Announce Type: new Abstract: In this paper, we introduce a model of evolution and learning in robots that co-optimizes a distribution of latent design vectors (genotypes) and a mixt

safetyarxiv-cs-ro

From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks

arXiv:2605.24771v1 Announce Type: cross Abstract: Classical noisy-label theory predicts that downstream performance under weak supervision is bounded above by the labeler's accuracy, implying a sharp

LWM-CDE: A Representation Space for Wireless Data Reasoning and Transferability

arXiv:2605.24077v1 Announce Type: cross Abstract: Machine learning deployments in real-world wireless communication tasks face significant generalization challenges due to location and environment-spe

applicationsarxiv-cs-lg

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs

arXiv:2605.24681v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine

Tutorials

MultiPUFFIN: A Multimodal Domain-Constrained Foundation Model for Molecular Property Prediction of Small Molecules

arXiv:2603.00857v2 Announce Type: replace-cross Abstract: MultiPUFFIN is a domain-informed multimodal foundation model for predicting thermophysical properties of small molecules, addressing a critica

tutorialsarxiv-cs-ai

PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models

arXiv:2512.20063v3 Announce Type: replace Abstract: We introduce exttt{PairFlow}, a lightweight preprocessing step for training Discrete Flow Models (DFMs) to achieve few-step sampling without requiri

Phonetic Modeling of Dialectal Variation in Vietnamese Speech

arXiv:2605.24451v1 Announce Type: new Abstract: Vietnamese exhibits substantial dialectal phonetic variation across Northern, Central, and Southern regions, where identical lexical items may be realiz

Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions

arXiv:2605.24452v1 Announce Type: cross Abstract: Legal NLP benchmarks evaluate models on randomly split data, implicitly assuming that legal language is stationary. We test this assumption by fine-tu

applicationsarxiv-cs-ai

ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning

arXiv:2605.23454v1 Announce Type: new Abstract: Rubric-based rewards offer a promising way to extend reinforcement learning (RL) for large language models beyond tasks with automatically verifiable an

applicationsarxiv-cs-cl

Exploring deep learning for Event-Based Saliency Prediction with a Transformer-based model

arXiv:2605.23790v1 Announce Type: new Abstract: Saliency prediction has been extensively studied in RGB images and videos as a computational model of human visual attention. In contrast, predicting sa

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction

arXiv:2603.10067v2 Announce Type: replace-cross Abstract: Muon has recently shown promising results in LLM training. In this work, we study how to further improve Muon. We argue that Muon's orthogonal

Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling

arXiv:2605.23198v1 Announce Type: new Abstract: Dataset pruning reduces the storage and training costs of deep learning by selecting an informative subset from a large dataset. However, most existing

Leveraging Foundation Models for Causal Generative Modeling

arXiv:2605.23861v1 Announce Type: cross Abstract: Causal generative modeling is essential for developing reliable and transparent AI systems capable of counterfactual reasoning. While existing approac

Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions

arXiv:2605.23885v1 Announce Type: new Abstract: Cross-lingual knowledge transfer is critical for building high-performing multilingual language models for languages with insufficient training data. Wh

Dynamic Mixture of Latent Memories for Self-Evolving Agents

arXiv:2605.21951v1 Announce Type: new Abstract: Achieving self-evolution in intelligent agents requires the continual accumulation of new knowledge across changing task sequences without forgetting pr

23 May 2026

An Open Multi-Center Whole-Body FDG PET/CT Foundation Model for Tumor Segmentation

arXiv:2605.21835v1 Announce Type: cross Abstract: The synergistic interpretation of anatomical information from computed tomography (CT) and metabolic information from positron emission tomography (PE

researcharxiv-cs-cv

22 May 2026

Bernini: Latent Semantic Planning for Video Diffusion

arXiv:2605.22344v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) and diffusion models have each reached remarkable maturity: MLLMs excel at reasoning over heterogeneous multimo

researcharxiv-cs-cv

22 May 2026

UniVL: Unified Vision-Language Embedding for Spatially Grounded Contextual Image Generation

arXiv:2605.21611v1 Announce Type: new Abstract: We introduce spatially grounded contextual image generation, a controllable image generation task that reframes the conditioning paradigm. Instead of su

22 May 2026

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

arXiv:2605.20756v1 Announce Type: new Abstract: Preconditioned optimizers are central to language model training, but their stochastic update rules are usually treated as direct approximations to popu

21 May 2026

DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation

model-releasesarxiv-cs-lg

arXiv:2605.20856v1 Announce Type: cross Abstract: Language-conditioned manipulation policies typically process instructions and observations through shared network parameters. This task-state entangle

21 May 2026

TERDNet: Transformer Encoder-Recurrent Decoder Network for Scene Change Detection

arXiv:2605.20822v1 Announce Type: new Abstract: In this work, we address the challenge of Scene Change Detection (SCD), where the goal is to identify variations between two images of the same location

applicationsarxiv-cs-cv

21 May 2026

LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition

arXiv:2605.19821v1 Announce Type: new Abstract: Facial Expression Recognition (FER) in the wild is still challenging due to uncontrolled variations in pose, occlusion, and illumination. Most existing

applicationsarxiv-cs-cv

20 May 2026

Learning ORDER-Aware Multimodal Representations for Composite Materials Design

arXiv:2602.02513v2 Announce Type: replace Abstract: Artificial intelligence has shown remarkable success in materials discovery and property prediction, particularly for crystalline and polymer system

20 May 2026

Stitched Value Model for Diffusion Alignment

arXiv:2605.19804v1 Announce Type: cross Abstract: For practical use, diffusion- or flow-based generative models must be aligned with task-specific rewards, such as prompt fidelity or aesthetic prefere

safetyarxiv-cs-ai

20 May 2026

CineMatte: Background Matting for Virtual Production and Beyond

arXiv:2605.18328v1 Announce Type: new Abstract: LED Virtual Production (VP) uses large LED volumes to render backgrounds in real time, enabling in-camera visual effects but making post-shot changes la

applicationsarxiv-cs-cv

Expectation and Acoustic Neural Network Representations Enhance Music Identification from Brain Activity

arXiv:2603.03190v3 Announce Type: replace Abstract: During music listening, cortical activity encodes both acoustic and expectation-related information. Prior work has shown that ANN representations r

Foundation Models for Credit Risk Prediction: A Game Changer?

arXiv:2605.18147v1 Announce Type: new Abstract: Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and los

Local Ai

SAMRI: Segment Any MRI

arXiv:2510.26635v3 Announce Type: replace-cross Abstract: Summary: SAMRI is an MRI-specialized adaptation of the Segment Anything Model achieving superior whole-body MRI segmentation, particularly for

local-aiarxiv-cs-cv

SAS: Semantic-aware Sampling for Generative Dataset Distillation

arXiv:2605.18012v1 Announce Type: cross Abstract: Deep neural networks have achieved impressive performance across a wide range of tasks, but this success often comes with substantial computational an

Spectral Progressive Diffusion for Efficient Image and Video Generation

arXiv:2605.18736v1 Announce Type: new Abstract: Diffusion models have been shown to implicitly generate visual content autoregressively in the frequency domain, where low-frequency components are gene

researcharxiv-cs-cv

Tutorials

Surface-Form Neural Sparse Retrieval: Robust Fuzzy Matching for Industrial Music Search

arXiv:2605.17762v1 Announce Type: new Abstract: Music search at the scale of Amazon Music presents a unique challenge: queries frequently deviate from indexed metadata due to misspellings, transpositi

tutorialsarxiv-cs-ai

GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding

arXiv:2605.15250v1 Announce Type: cross Abstract: Multi-head Latent Attention (MLA), the attention used in DeepSeek-V2/V3, jointly compresses keys and values into a low-rank latent and matches the H10

Mask-Morph Graph U-Net: A Generalisable Mesh-Based Surrogate for Crashworthiness Field Prediction under Large Geometric Variation

arXiv:2605.15231v1 Announce Type: cross Abstract: Nonlinear finite element crash simulations are accurate but computationally expensive, limiting their use in iterative design optimisation. Machine-le

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language

arXiv:2605.15607v1 Announce Type: new Abstract: Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from

Agents

Talking Trees: Reasoning-Assisted Induction of Decision Trees for Tabular Data

arXiv:2509.21465v3 Announce Type: replace Abstract: Tabular foundation models are becoming increasingly popular for low-resource tabular problems. These models make up for small training datasets by p

agentsarxiv-cs-lg

A cross-species neural foundation model for end-to-end speech decoding

arXiv:2511.21740v5 Announce Type: replace-cross Abstract: Speech brain-computer interfaces (BCIs) aim to restore communication for people with paralysis by translating neural activity into text. Most

safetyarxiv-cs-ai

A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring

arXiv:2605.14360v1 Announce Type: cross Abstract: Wearable devices capture physiological and behavioral data with increasing fidelity, but the psychological context shaping these outcomes is difficult

Energy-Regularized Sequential Model Editing on Hyperspheres

arXiv:2510.01172v3 Announce Type: replace Abstract: Large language models (LLMs) require constant updates to remain aligned with evolving real-world knowledge. Model editing offers a lightweight alter

applicationsarxiv-cs-cl

NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces

arXiv:2605.14698v1 Announce Type: cross Abstract: Foundation models (FMs) promise to extract unified representations that generalize across downstream tasks. They have emerged across fields, including

REALM: Retrospective Encoder Alignment for LFP Modeling

arXiv:2605.14867v1 Announce Type: cross Abstract: Spike activity has been the dominant neural signal for behavior decoding due to its high spatial and temporal resolution. However, as brain-computer i

Tokenizing Single-Channel EEG with Time-Frequency Motif Learning

arXiv:2502.16060v5 Announce Type: replace-cross Abstract: Foundation models are reshaping EEG analysis, yet an important problem of EEG tokenization remains a challenge. This paper presents TFM-Tokeni

Unified Pix Token And Word Token Generative Language Model

arXiv:2605.14028v1 Announce Type: new Abstract: Since the emergence of Vision Transformer (ViT), it has been widely used in generative language model and generative visual model. Especially in the cur