Search: “pretraining”

Unsupervised simulation of incompressible flows with physics- and equality- constrained artificial neural networks

arXiv:2511.18820v2 Announce Type: replace-cross Abstract: Physics-informed neural networks (PINNs) have shown promise for solving partial differential equations, yet their success in simulating incomp

researcharxiv-cs-lg

15 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement

DGX agent

arXiv:2605.14368v1 Announce Type: cross Abstract: Continuous diffusion language models lag behind autoregressive transformers, partly because diffusion is applied in spaces poorly suited to language d

researcharxiv-cs-ai

15 May 2026

Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer

arXiv:2605.12798v1 Announce Type: cross Abstract: Fine-tuning LLMs on narrow harmful datasets can induce Emergent Misalignment (EM), where models exhibit misaligned behavior far beyond the fine-tuning

Force-Aware Neural Tangent Kernels for Scalable and Robust Active Learning of MLIPs

arXiv:2605.13788v1 Announce Type: new Abstract: Active learning for machine-learning interatomic potentials (MLIPs) must address several challenges to be practical: scaling to large candidate pools, l

No One Knows the State of the Art in Geospatial Foundation Models

arXiv:2605.12678v1 Announce Type: new Abstract: Geospatial foundation models (GFMs) have been proposed as generalizable backbones for disaster response, land-cover mapping, food-security monitoring, a

Stylized Text-to-Motion Generation via Hypernetwork-Driven Low-Rank Adaptation

arXiv:2605.13333v1 Announce Type: cross Abstract: Text-driven motion diffusion models are capable of generating realistic human motions, but text alone often struggles to express fine-level nuances of

researcharxiv-cs-ai

Fine-Tuning Large Language Models for Cooperative Tactical Deconfliction of Small Unmanned Aerial Systems

model-releasesarxiv-cs-ro

arXiv:2603.28561v2 Announce Type: replace Abstract: The growing deployment of small Unmanned Aerial Systems (sUASs) in low-altitude airspaces has increased the need for reliable tactical deconfliction

GeneZip: Region-Aware Compression for Long Context DNA Modeling

arXiv:2602.17739v3 Announce Type: replace-cross Abstract: Long-context DNA models are limited by token-mixing cost and by how compression allocates representational budget across the genome. Existing

Applications

Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment

arXiv:2605.11439v1 Announce Type: new Abstract: Rapid and accurate situational awareness is essential for effective response during natural disasters, where delays in analysis can significantly hinder

applicationsarxiv-cs-cv

Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction

arXiv:2605.11354v1 Announce Type: new Abstract: Transformer-based 3D reconstruction has emerged as a powerful paradigm for recovering geometry and appearance from multi-view observations, offering str

Training Transformers for KV Cache Compressibility

arXiv:2605.05971v2 Announce Type: replace Abstract: Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly wit

Bangla-WhisperDiar: Fine-Tuning Whisper and PyAnnote for Bangla Long-Form Speech Recognition and Speaker Diarization

arXiv:2605.08214v1 Announce Type: cross Abstract: Automatic Speech Recognition (ASR) and speaker diarization in Bangla remain challenging due to long form recordings, diverse acoustic conditions, and

researcharxiv-cs-ai

BenchHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition

arXiv:2605.08296v1 Announce Type: new Abstract: Human Activity Recognition (HAR) from wearable sensors supports broad healthcare and behavior science applications. However, data heterogeneity and the

Control-Augmented Autoregressive Diffusion for Data Assimilation

arXiv:2510.06637v3 Announce Type: replace-cross Abstract: Despite advances in test-time scaling and diffusion finetuning, guidance for Auto-Regressive Diffusion Models (ARDMs) remains underexplored. W

Cross-Modal Semantic-Enhanced Diffusion Framework for Diabetic Retinopathy Grading

arXiv:2605.09242v1 Announce Type: cross Abstract: Automated grading of diabetic retinopathy (DR) faces several critical challenges: subtle inter-grade visual distinctions in fine-grained lesion patter

DiffATS: Diffusion in Aligned Tensor Space

arXiv:2605.09275v1 Announce Type: new Abstract: Direct diffusion modeling of high-resolution spatiotemporal fields is computationally challenging. Parameter-efficient primitives address this by repres

EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecasting

arXiv:2602.01194v2 Announce Type: replace Abstract: Long-term weather forecasting is critical for socioeconomic planning and disaster preparedness. While recent approaches employ finetuning to extend

Inverse Design for Conditional Distribution Matching

arXiv:2605.09439v1 Announce Type: new Abstract: Generative models are powerful tools for sampling from a learned distribution P(Y mid X), and inverse-design methods invert this map to find an input x

researcharxiv-cs-lg

Large Language Models for Sequential Decision-Making: Improving In-Context Learning via Supervised Fine-Tuning

arXiv:2605.09009v1 Announce Type: cross Abstract: Large language models (LLMs) have shown remarkable in-context learning (ICL) capabilities, yet their potential for sequential decision-making remains

LLiMba: Sardinian on a Single GPU -- Adapting a 3B Language Model to a Vanishing Romance Language

model-releasesarxiv-cs-cl

arXiv:2605.09015v1 Announce Type: new Abstract: Sardinian, a Romance language with roughly one million speakers, has minimal presence in modern NLP. Commercial services do not support it, and current

MicroDiffuse3D: A Foundation Model for 3D Microscopy Imaging Restoration

arXiv:2605.08566v1 Announce Type: new Abstract: Chemical imaging enables label-free visualization of cells, tissues and living systems while providing direct biochemical information that is difficult

On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective

arXiv:2605.08368v1 Announce Type: new Abstract: Debates about large language model post-training often treat supervised fine-tuning (SFT) as imitation and reinforcement learning (RL) as discovery. But

local-aiarxiv-cs-ai

Positional Encoding via Token-Aware Phase Attention

arXiv:2509.12635v3 Announce Type: replace-cross Abstract: We prove under practical assumptions that Rotary Positional Embedding (RoPE) introduces an intrinsic distance-dependent bias in attention scor

Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift

arXiv:2505.19519v3 Announce Type: replace Abstract: Personalizing text-to-image diffusion models involves integrating novel visual concepts from a small set of reference images while retaining the mod

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design

arXiv:2605.10189v1 Announce Type: cross Abstract: Designing proteins with desired functions or properties represents a core goal in synthetic biology and drug discovery. Recent advances in protein lan

Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality

arXiv:2605.10142v1 Announce Type: cross Abstract: Artificial intelligence models are increasingly scaled to improve predictive accuracy, yet it remains unclear whether scale improves the quality of po

Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning

arXiv:2605.10247v1 Announce Type: new Abstract: Using Large Language Models (LLMs) to process graph-structured data is an active research area, yet current state-of-the-art approaches typically rely o

TFM-Retouche: A Lightweight Input-Space Adapter for Tabular Foundation Models

arXiv:2605.06047v2 Announce Type: replace-cross Abstract: Tabular foundation models (TFMs), such as TabPFN-2.6, TabICLv2, ConTextTab, Mitra, LimiX, and TabDPT, achieve strong zero-shot performance thr

Tracing Moral Foundations in Large Language Models

arXiv:2601.05437v2 Announce Type: replace-cross Abstract: Large language models often produce human-like moral judgments, but it is unclear whether this reflects an internal conceptual structure or su

WATCH: Wide-Area Archaeological Site Tracking for Change Detection

arXiv:2605.08160v1 Announce Type: cross Abstract: Monitoring archaeological sites at scale is vital for protecting cultural heritage, yet pinpointing when disturbances occur remains difficult because

WavesFM: Hierarchical Representation Learning for Longitudinal Wearable Sensor Waveforms

arXiv:2605.09173v1 Announce Type: cross Abstract: Wearable sensors enable the continuous acquisition of high-resolution physiological waveforms, such as photoplethysmography and accelerometry, under f

local-aiarxiv-cs-ai

When More Parameters Hurt: Foundation Model Priors Amplify Worst-Client Disparity Under Extreme Federated Heterogeneity

arXiv:2605.08992v1 Announce Type: new Abstract: Federated learning (FL) is increasingly used to fine-tune foundation models (FMs) on distributed private data. The community largely assumes that large-

Excluding the Target Domain Improves Extrapolation: Deconfounded Hierarchical Physics Constraints

arXiv:2605.07485v1 Announce Type: cross Abstract: Extrapolation to out-of-distribution conditions is a fundamental challenge for physics-constrained deep generative models. Existing methods apply phys

Inference-Time Attribute Distribution Alignment for Unconditional Diffusion

arXiv:2605.07456v1 Announce Type: new Abstract: Inference-time controllable generation is essential for real-world applications of unconditional diffusion models. However, most existing techniques foc

Applications

Large Video Planner Enables Generalizable Robot Control

arXiv:2512.15840v2 Announce Type: replace-cross Abstract: General-purpose robots require decision-making models that generalize across diverse tasks and environments. Recent works build robot foundati

applicationsarxiv-cs-cv

Pre-trained Tabular Foundation Models as Versatile Summary Networks for Neural Posterior Estimation

arXiv:2605.07765v1 Announce Type: new Abstract: In this work, we study TabPFN as a training-free, modular summary network for simulation-based Bayesian inference (SBI). Tabular foundation models such

researcharxiv-cs-lg

Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts

arXiv:2605.07307v1 Announce Type: new Abstract: Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must

researcharxiv-cs-cl

Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning

arXiv:2605.07106v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have made remarkable progress on vision-language reasoning, yet most methods still compress visual evidence int

researcharxiv-cs-cl

Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation

arXiv:2605.07950v1 Announce Type: new Abstract: We study Slowly Annealed Langevin Dynamics (SALD), a sampler for tracking a path of moving target distributions and approximating the terminal target th

TextLDM: Language Modeling with Continuous Latent Diffusion

arXiv:2605.07748v1 Announce Type: new Abstract: Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos. A natural next st

safetyarxiv-cs-cl

Tutorials

Transfer Learning Across Fast- and Full-Simulation Domains in High-Energy Physics

arXiv:2605.07471v1 Announce Type: new Abstract: Machine-learning models in high-energy physics are often trained on simulated data, where fully simulated samples are computationally expensive while fa

tutorialsarxiv-cs-lg

A large-scale heterogeneous 3D magnetic resonance brain imaging dataset for self-supervised learning

arXiv:2506.14432v3 Announce Type: replace-cross Abstract: We present FOMO260K, a large-scale, heterogeneous dataset of 260,927 brain Magnetic Resonance Imaging (MRI) scans from 77,589 MRI sessions and

FMI_SU_Yotkova_Kastreva at SemEval-2026 Task 13: Lightweight Detection of LLM-Generated Code via Stylometric Signals

arXiv:2605.04157v1 Announce Type: new Abstract: SemEval-2026 Task 13 investigates machine-generated code detection across multiple programming languages and application scenarios, asking participating

researcharxiv-cs-cl

POMA-3D: The Point Map Way to 3D Scene Understanding

arXiv:2511.16567v3 Announce Type: replace Abstract: In this paper, we introduce POMA-3D, the first self-supervised 3D representation model learned from point maps. Point maps encode explicit 3D coordi

safetyarxiv-cs-cv

Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training

arXiv:2605.04913v1 Announce Type: new Abstract: LLM post-training typically propagates task gradients through the full depth of the model. Although this end-to-end structure is simple and general, it

local-aiarxiv-cs-cl

TumorXAI: Self-Supervised Deep Learning Framework for Explainable Brain MRI Tumor Classification

arXiv:2605.01999v1 Announce Type: new Abstract: Classifying brain tumors using magnetic resonance imaging (MRI) is crucial for early diagnosis and treatment; however, tumor heterogeneity and a dearth

researcharxiv-cs-ai

6 May 2026

High-Fidelity Mobile Avatars with Pruned Local Blendshapes

arXiv:2605.01854v1 Announce Type: new Abstract: We propose a method to reconstruct high-fidelity human avatars from multi-view video that can run on mobile devices. Many works can model high-quality G

local-aiarxiv-cs-cv

5 May 2026

Linearizing Vision Transformer with Test-Time Training