Search: “pretraining”

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs

arXiv:2605.24681v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Tutorials

MultiPUFFIN: A Multimodal Domain-Constrained Foundation Model for Molecular Property Prediction of Small Molecules

DGX agent

arXiv:2603.00857v2 Announce Type: replace-cross Abstract: MultiPUFFIN is a domain-informed multimodal foundation model for predicting thermophysical properties of small molecules, addressing a critica

tutorialsarxiv-cs-ai

26 May 2026

PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models

arXiv:2512.20063v3 Announce Type: replace Abstract: We introduce exttt{PairFlow}, a lightweight preprocessing step for training Discrete Flow Models (DFMs) to achieve few-step sampling without requiri

Phonetic Modeling of Dialectal Variation in Vietnamese Speech

arXiv:2605.24451v1 Announce Type: new Abstract: Vietnamese exhibits substantial dialectal phonetic variation across Northern, Central, and Southern regions, where identical lexical items may be realiz

Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions

arXiv:2605.24452v1 Announce Type: cross Abstract: Legal NLP benchmarks evaluate models on randomly split data, implicitly assuming that legal language is stationary. We test this assumption by fine-tu

applicationsarxiv-cs-ai

ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning

arXiv:2605.23454v1 Announce Type: new Abstract: Rubric-based rewards offer a promising way to extend reinforcement learning (RL) for large language models beyond tasks with automatically verifiable an

applicationsarxiv-cs-cl

Exploring deep learning for Event-Based Saliency Prediction with a Transformer-based model

arXiv:2605.23790v1 Announce Type: new Abstract: Saliency prediction has been extensively studied in RGB images and videos as a computational model of human visual attention. In contrast, predicting sa

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction

arXiv:2603.10067v2 Announce Type: replace-cross Abstract: Muon has recently shown promising results in LLM training. In this work, we study how to further improve Muon. We argue that Muon's orthogonal

Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling

arXiv:2605.23198v1 Announce Type: new Abstract: Dataset pruning reduces the storage and training costs of deep learning by selecting an informative subset from a large dataset. However, most existing

Leveraging Foundation Models for Causal Generative Modeling

arXiv:2605.23861v1 Announce Type: cross Abstract: Causal generative modeling is essential for developing reliable and transparent AI systems capable of counterfactual reasoning. While existing approac

Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions

arXiv:2605.23885v1 Announce Type: new Abstract: Cross-lingual knowledge transfer is critical for building high-performing multilingual language models for languages with insufficient training data. Wh

Dynamic Mixture of Latent Memories for Self-Evolving Agents

arXiv:2605.21951v1 Announce Type: new Abstract: Achieving self-evolution in intelligent agents requires the continual accumulation of new knowledge across changing task sequences without forgetting pr

23 May 2026

An Open Multi-Center Whole-Body FDG PET/CT Foundation Model for Tumor Segmentation

arXiv:2605.21835v1 Announce Type: cross Abstract: The synergistic interpretation of anatomical information from computed tomography (CT) and metabolic information from positron emission tomography (PE

researcharxiv-cs-cv

22 May 2026

Bernini: Latent Semantic Planning for Video Diffusion

arXiv:2605.22344v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) and diffusion models have each reached remarkable maturity: MLLMs excel at reasoning over heterogeneous multimo

researcharxiv-cs-cv

22 May 2026

UniVL: Unified Vision-Language Embedding for Spatially Grounded Contextual Image Generation

arXiv:2605.21611v1 Announce Type: new Abstract: We introduce spatially grounded contextual image generation, a controllable image generation task that reframes the conditioning paradigm. Instead of su

22 May 2026

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

arXiv:2605.20756v1 Announce Type: new Abstract: Preconditioned optimizers are central to language model training, but their stochastic update rules are usually treated as direct approximations to popu

safetyarxiv-cs-lg

21 May 2026

DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation

model-releasesarxiv-cs-lg

arXiv:2605.20856v1 Announce Type: cross Abstract: Language-conditioned manipulation policies typically process instructions and observations through shared network parameters. This task-state entangle

21 May 2026

TERDNet: Transformer Encoder-Recurrent Decoder Network for Scene Change Detection

arXiv:2605.20822v1 Announce Type: new Abstract: In this work, we address the challenge of Scene Change Detection (SCD), where the goal is to identify variations between two images of the same location

21 May 2026

LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition

arXiv:2605.19821v1 Announce Type: new Abstract: Facial Expression Recognition (FER) in the wild is still challenging due to uncontrolled variations in pose, occlusion, and illumination. Most existing

20 May 2026

Learning ORDER-Aware Multimodal Representations for Composite Materials Design

arXiv:2602.02513v2 Announce Type: replace Abstract: Artificial intelligence has shown remarkable success in materials discovery and property prediction, particularly for crystalline and polymer system

safetyarxiv-cs-lg

20 May 2026

Stitched Value Model for Diffusion Alignment

arXiv:2605.19804v1 Announce Type: cross Abstract: For practical use, diffusion- or flow-based generative models must be aligned with task-specific rewards, such as prompt fidelity or aesthetic prefere

safetyarxiv-cs-ai

20 May 2026

CineMatte: Background Matting for Virtual Production and Beyond

arXiv:2605.18328v1 Announce Type: new Abstract: LED Virtual Production (VP) uses large LED volumes to render backgrounds in real time, enabling in-camera visual effects but making post-shot changes la

Expectation and Acoustic Neural Network Representations Enhance Music Identification from Brain Activity

arXiv:2603.03190v3 Announce Type: replace Abstract: During music listening, cortical activity encodes both acoustic and expectation-related information. Prior work has shown that ANN representations r

Foundation Models for Credit Risk Prediction: A Game Changer?

arXiv:2605.18147v1 Announce Type: new Abstract: Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and los

Local Ai

SAMRI: Segment Any MRI

arXiv:2510.26635v3 Announce Type: replace-cross Abstract: Summary: SAMRI is an MRI-specialized adaptation of the Segment Anything Model achieving superior whole-body MRI segmentation, particularly for

local-aiarxiv-cs-cv

SAS: Semantic-aware Sampling for Generative Dataset Distillation

arXiv:2605.18012v1 Announce Type: cross Abstract: Deep neural networks have achieved impressive performance across a wide range of tasks, but this success often comes with substantial computational an

Spectral Progressive Diffusion for Efficient Image and Video Generation

arXiv:2605.18736v1 Announce Type: new Abstract: Diffusion models have been shown to implicitly generate visual content autoregressively in the frequency domain, where low-frequency components are gene

researcharxiv-cs-cv

Tutorials

Surface-Form Neural Sparse Retrieval: Robust Fuzzy Matching for Industrial Music Search

arXiv:2605.17762v1 Announce Type: new Abstract: Music search at the scale of Amazon Music presents a unique challenge: queries frequently deviate from indexed metadata due to misspellings, transpositi

tutorialsarxiv-cs-ai

GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding

arXiv:2605.15250v1 Announce Type: cross Abstract: Multi-head Latent Attention (MLA), the attention used in DeepSeek-V2/V3, jointly compresses keys and values into a low-rank latent and matches the H10

Mask-Morph Graph U-Net: A Generalisable Mesh-Based Surrogate for Crashworthiness Field Prediction under Large Geometric Variation

arXiv:2605.15231v1 Announce Type: cross Abstract: Nonlinear finite element crash simulations are accurate but computationally expensive, limiting their use in iterative design optimisation. Machine-le

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language

arXiv:2605.15607v1 Announce Type: new Abstract: Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from

Agents

Talking Trees: Reasoning-Assisted Induction of Decision Trees for Tabular Data

arXiv:2509.21465v3 Announce Type: replace Abstract: Tabular foundation models are becoming increasingly popular for low-resource tabular problems. These models make up for small training datasets by p

agentsarxiv-cs-lg

A cross-species neural foundation model for end-to-end speech decoding

arXiv:2511.21740v5 Announce Type: replace-cross Abstract: Speech brain-computer interfaces (BCIs) aim to restore communication for people with paralysis by translating neural activity into text. Most

safetyarxiv-cs-ai

A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring

arXiv:2605.14360v1 Announce Type: cross Abstract: Wearable devices capture physiological and behavioral data with increasing fidelity, but the psychological context shaping these outcomes is difficult

Energy-Regularized Sequential Model Editing on Hyperspheres

arXiv:2510.01172v3 Announce Type: replace Abstract: Large language models (LLMs) require constant updates to remain aligned with evolving real-world knowledge. Model editing offers a lightweight alter

applicationsarxiv-cs-cl

NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces

arXiv:2605.14698v1 Announce Type: cross Abstract: Foundation models (FMs) promise to extract unified representations that generalize across downstream tasks. They have emerged across fields, including

REALM: Retrospective Encoder Alignment for LFP Modeling

arXiv:2605.14867v1 Announce Type: cross Abstract: Spike activity has been the dominant neural signal for behavior decoding due to its high spatial and temporal resolution. However, as brain-computer i

Tokenizing Single-Channel EEG with Time-Frequency Motif Learning

arXiv:2502.16060v5 Announce Type: replace-cross Abstract: Foundation models are reshaping EEG analysis, yet an important problem of EEG tokenization remains a challenge. This paper presents TFM-Tokeni

Unified Pix Token And Word Token Generative Language Model

arXiv:2605.14028v1 Announce Type: new Abstract: Since the emergence of Vision Transformer (ViT), it has been widely used in generative language model and generative visual model. Especially in the cur

Unsupervised simulation of incompressible flows with physics- and equality- constrained artificial neural networks

arXiv:2511.18820v2 Announce Type: replace-cross Abstract: Physics-informed neural networks (PINNs) have shown promise for solving partial differential equations, yet their success in simulating incomp

Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement

arXiv:2605.14368v1 Announce Type: cross Abstract: Continuous diffusion language models lag behind autoregressive transformers, partly because diffusion is applied in spaces poorly suited to language d

Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer

arXiv:2605.12798v1 Announce Type: cross Abstract: Fine-tuning LLMs on narrow harmful datasets can induce Emergent Misalignment (EM), where models exhibit misaligned behavior far beyond the fine-tuning

safetyarxiv-cs-ai

Force-Aware Neural Tangent Kernels for Scalable and Robust Active Learning of MLIPs

model-releasesarxiv-cs-lg

arXiv:2605.13788v1 Announce Type: new Abstract: Active learning for machine-learning interatomic potentials (MLIPs) must address several challenges to be practical: scaling to large candidate pools, l

No One Knows the State of the Art in Geospatial Foundation Models

arXiv:2605.12678v1 Announce Type: new Abstract: Geospatial foundation models (GFMs) have been proposed as generalizable backbones for disaster response, land-cover mapping, food-security monitoring, a

Stylized Text-to-Motion Generation via Hypernetwork-Driven Low-Rank Adaptation

arXiv:2605.13333v1 Announce Type: cross Abstract: Text-driven motion diffusion models are capable of generating realistic human motions, but text alone often struggles to express fine-level nuances of

Fine-Tuning Large Language Models for Cooperative Tactical Deconfliction of Small Unmanned Aerial Systems

model-releasesarxiv-cs-ro

arXiv:2603.28561v2 Announce Type: replace Abstract: The growing deployment of small Unmanned Aerial Systems (sUASs) in low-altitude airspaces has increased the need for reliable tactical deconfliction

13 May 2026

GeneZip: Region-Aware Compression for Long Context DNA Modeling

model-releasesarxiv-cs-lg

arXiv:2602.17739v3 Announce Type: replace-cross Abstract: Long-context DNA models are limited by token-mixing cost and by how compression allocates representational budget across the genome. Existing

13 May 2026

Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment

arXiv:2605.11439v1 Announce Type: new Abstract: Rapid and accurate situational awareness is essential for effective response during natural disasters, where delays in analysis can significantly hinder