Search: “pretraining”

MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Models

model-releasesarxiv-cs-cl

arXiv:2511.03942v2 Announce Type: replace-cross Abstract: We present MIDI-LLM, a recipe that improves multitrack text-to-MIDI generation via adapting Large Language Models (LLMs). MIDI-LLM expands an

5 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Self-Supervised Representation-Guided Generative Dataset Distillation

DGX agent

arXiv:2608.03218v1 Announce Type: cross Abstract: Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target

safetyarxiv-cs-ai

5 Aug 2026

Beyond Token-Level Cross-Entropy: Frechet Distributional Post-Training for Autoregressive Image Generation

arXiv:2608.00562v1 Announce Type: new Abstract: Autoregressive image generators are commonly pretrained with token-level cross-entropy under teacher forcing, yet evaluated by the distributional qualit

Capability Provenance in Language Models: A Case Study in Social Reasoning

model-releasesarxiv-cs-cl

arXiv:2606.19625v2 Announce Type: replace Abstract: We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-r

LLM generation novelty through the lens of semantic similarity

arXiv:2510.27313v3 Announce Type: replace-cross Abstract: Generation novelty is a key indicator of an LLM's ability to generalize, yet measuring it against full pretraining corpora is computationally

researcharxiv-cs-cl

Hardware

MiniWorld: Democratizing the Training of Video World Models from Scratch

arXiv:2608.01127v1 Announce Type: new Abstract: Video world models predict future observations conditioned on historical observations and control signals, enabling long-horizon generation through auto

hardwarearxiv-cs-cv

Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenization

arXiv:2505.18819v2 Announce Type: replace Abstract: Vision-language models, such as CLIP, encode rich semantic knowledge through large-scale image-text pretraining. Reusing these models for 3D underst

Recursive Vision Language Models for General Symbolic Reasoning

arXiv:2608.01534v1 Announce Type: new Abstract: Hard symbolic-reasoning tasks such as Sudoku, maze pathfinding, and ARC remain challenging for LLMs due to their fixed-depth autoregressive reasoning, w

SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation

arXiv:2608.01184v1 Announce Type: new Abstract: Data-free continual model merging must incorporate a stream of specialized models while retaining both pretrained general knowledge and previously acqui

SingLEM: Single-Channel Large EEG Model

arXiv:2509.17920v2 Announce Type: replace Abstract: Current deep learning models for electroencephalography (EEG) are often task-specific and depend on large labeled datasets, limiting their adaptabil

local-aiarxiv-cs-lg

Understanding and Correcting Low-Frequency Bias in EEG Foundation Model

arXiv:2608.01898v1 Announce Type: new Abstract: Increasing EEG pretraining data scale or model capacity does not consistently improve downstream performance. We identify a persistent low-frequency bia

safetyarxiv-cs-lg

Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Mesh

arXiv:2608.00094v1 Announce Type: new Abstract: Pretrained video diffusion models can act as renderers when the desired scene state is already specified by an animated mesh, a camera trajectory, and a

Implicit Reasoning for Large Language Model-based Generative Recommendation

arXiv:2606.14142v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world kn

safetyarxiv-cs-ai

3 Aug 2026

Inference-time Trajectory Optimization for Structure-Preserving Manga Image Editing

arXiv:2603.27790v2 Announce Type: replace Abstract: We present a lightweight, training-free trajectory correction method that adapts a pretrained image editing model to each input manga image using on

3 Aug 2026

MoRoute: Dynamic Routing for In-Context Multimodal Video Generation

arXiv:2607.29545v1 Announce Type: new Abstract: Multimodal video generation aims to generate and edit videos conditioned on arbitrary combinations of text, images, and videos within a single model, al

3 Aug 2026

Continual Learning with Vision-Language Models via Semantic-Geometry Preservation

arXiv:2603.12055v3 Announce Type: replace Abstract: Continual learning of pretrained vision-language models (VLMs) is prone to catastrophic forgetting, yet current approaches adapt to new tasks withou

31 Jul 2026

I have trained a model to predict my blood sugar [P]

model-releasesr-machinelearning

It's an encoder-only transformer that consumes past(blood glucose + carbs + insulin) and future(carbs + insulin) and predicts future blood glucose for the next 2 hours. Announced meals and boluses/bas

31 Jul 2026

X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching

arXiv:2607.28560v1 Announce Type: new Abstract: Pretraining navigation diffusion policies rely on large-scale expert demonstrations. These data are typically generated by a fully-informed oracle plann

local-aiarxiv-cs-ro

31 Jul 2026

Foundation Models for Face Presentation Attack Detection: A Unified Linear-Probing Benchmark

arXiv:2607.26993v1 Announce Type: new Abstract: Face presentation attack detection (PAD) remains challenging under cross-dataset evaluation, where domain shift degrades models trained on a single data

30 Jul 2026

Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

arXiv:2607.26326v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance by integrating visual inputs with the rich priors of pretrained language models. How

30 Jul 2026

pimathbf{R}^2: Reactive Real-time Flow Policies

arXiv:2607.26055v1 Announce Type: cross Abstract: Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open

safetyarxiv-cs-ai

29 Jul 2026

{au}: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision

arXiv:2607.24485v1 Announce Type: new Abstract: Learning the informative tactile representation while effectively adapting it to pretrained Vision-Language-Action (VLA) models remains challenging at b

researcharxiv-cs-ro

CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

arXiv:2607.23258v1 Announce Type: new Abstract: Large-scale calcium imaging has created an opportunity to build foundation-style models for neural population dynamics, but a central question remains u

researcharxiv-cs-ai

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling

arXiv:2510.14717v2 Announce Type: replace-cross Abstract: Increasing the batch size during training -- a ''batch ramp'' -- is a promising strategy to accelerate large language model pretraining. While

Stabilizing Deep Reconstruction Operators with Contractive Anchoring

arXiv:2607.23341v1 Announce Type: cross Abstract: Pretrained deep denoisers can be used to solve a wide range of model-based image reconstruction tasks via Plug-and-Play (PnP) and Regularization-by-De

UNIFUSION: Adapting Autoregressive Language Models into Discrete Diffusion under a Unified Reverse-Rate Objective

arXiv:2607.24507v1 Announce Type: cross Abstract: Existing methods mainly adapt pretrained autoregressive (AR) language models to masked diffusion, whereas we directly adapt them to uniform-noise diff

WorldDiT: A Unified Diffusion Architecture for World and Action Modeling

arXiv:2607.23909v1 Announce Type: new Abstract: Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT

Autoregressive EHR Foundation Models with Multimodal Inputs

arXiv:2607.22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on st

safetyarxiv-cs-lg

27 Jul 2026

IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data

arXiv:2607.22351v1 Announce Type: new Abstract: The speed of sound in tissue is a prerequisite for well-focused imaging and has diagnostic value, but recovering it from raw pulse-echo channel data is

researcharxiv-cs-lg

27 Jul 2026

TextSLIP: Text Self-Supervised CLIP for Medical Report Generation

arXiv:2607.21970v1 Announce Type: new Abstract: Automating radiology report generation is important for improving reporting consistency and clinical workflows . While Contrastive Language--Image Pretr

safetyarxiv-cs-cv

27 Jul 2026

MSBraM: A Multi-scale Self-supervised Brain Foundation Model for Hierarchical EEG Dynamics Learning

arXiv:2607.21402v1 Announce Type: new Abstract: Self-supervised foundation models have recently shown strong potential for electroencephalogram (EEG)-based analysis. However, existing approaches strug

local-aiarxiv-cs-ai

Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning

arXiv:2607.20913v1 Announce Type: new Abstract: Fine-tuning large diffusion models for new domains or styles involves a trade-off: improving target-specific generation often degrades the pretrained mo

Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning

arXiv:2607.20914v1 Announce Type: new Abstract: Federated parameter-efficient fine-tuning (PEFT) enables communication-efficient adaptation of large pretrained models on decentralized edge data, but i

Zagreus-0.4B-por a small open source language model for Portuguese

model-releasesr-localllama

mii-llm, an open source AI lab, released Zagreus-0.4B-por, a compact bilingual Portuguese–English language model pretrained entirely from scratch. The model has approximately 400 million parameters an

Image Editing Models are Numerical Solvers

arXiv:2607.18787v1 Announce Type: new Abstract: We investigate whether a pretrained generative image-editing model can provide a common interface for numerical simulation. Physical inputs and solution

23 Jul 2026

LAVIFT: Latent-Action-Guided Vision Fine-Tuning for Surgical Interaction Recognition

arXiv:2607.19889v1 Announce Type: new Abstract: Understanding instrument-tissue interactions is essential for context-aware surgical AI and autonomous robotic surgery. Pretrained vision-language model

local-aiarxiv-cs-cv

23 Jul 2026

Applications

Semantic Anchoring for Robotic Action Representations

arXiv:2607.13597v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models inherit rich semantic representations from pretrained Vision-Language Models, yet fine-tuning on limited robot dem

applicationsarxiv-cs-ai

16 Jul 2026

Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

arXiv:2607.12831v1 Announce Type: new Abstract: Language models encode substantial factual knowledge in their parameters, which can lead to unreliable behavior when this knowledge is outdated, incompl

researcharxiv-cs-cl

15 Jul 2026

Tutorials

PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models

arXiv:2602.04029v2 Announce Type: replace-cross Abstract: Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the divers

tutorialsarxiv-cs-ai

15 Jul 2026

When Directional Accuracy Lies: A Base-Rate-Honest Benchmark for LoRA-Adapted TimesFM on Equity Forecasting

arXiv:2607.12248v1 Announce Type: cross Abstract: Large pretrained time-series models such as TimesFM are attractive for financial forecasting, but raw directional accuracy is a misleading scoreboard

15 Jul 2026

LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression

arXiv:2607.08221v1 Announce Type: new Abstract: Large language model (LLM)-based lossless image compression methods typically represent pixel data through the native text interface of a pretrained mod

10 Jul 2026

Scalable and Trustworthy Earth Observation Foundation Models

arXiv:2607.07758v1 Announce Type: new Abstract: Foundation models (FMs) have transformed machine learning from isolated task-specific model development toward general-purpose models pretrained on broa

10 Jul 2026

Activation Quantization of Vision Encoders Needs Prefixing Registers

arXiv:2510.04547v5 Announce Type: replace-cross Abstract: Large pretrained vision encoders are central to multimodal intelligence, powering applications from on-device vision processing to vision-lang

local-aiarxiv-cs-cv

9 Jul 2026

GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model

arXiv:2607.06882v1 Announce Type: cross Abstract: Visual navigation policies built on large pretrained models have so far followed a common recipe: a dedicated visual encoder, a bespoke action head, a

safetyarxiv-cs-ai

9 Jul 2026

CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

arXiv:2607.02576v1 Announce Type: cross Abstract: Parameter-Efficient Fine-Tuning (PEFT) commonly adapts pretrained weights through low-rank updates, and recent methods further exploit the singular va

7 Jul 2026

Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

arXiv:2607.03751v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragi

researcharxiv-cs-ro

7 Jul 2026

RetailSMV: Exocentric vs. Egocentric Adaptation of Foundation Video World Models in Retail

arXiv:2607.00310v1 Announce Type: cross Abstract: Foundation video diffusion models are increasingly viewed as world simulators for embodied agents, yet their pretraining on internet-scale generic vid

2 Jul 2026

Hardware

Hierarchical Global Attention (HGA)