Search: “pretraining”

Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training

arXiv:2605.26189v1 Announce Type: cross Abstract: Quantization-aware training (QAT) with low-bit floating-point formats enables efficient LLM deployment, yet introduces subtle failure modes invisible

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training

DGX agent

arXiv:2605.26842v1 Announce Type: cross Abstract: The Muon optimizer has recently offered a promising alternative to AdamW for large language model training, leveraging matrix orthogonalization to pro

local-aiarxiv-cs-cl

27 May 2026

Applications

No Data? No Problem: Robust Vision-Tabular Learning with Missing Values

arXiv:2512.19602v2 Announce Type: replace Abstract: Large-scale medical biobanks provide imaging data complemented by extensive tabular information, such as clinical measurements or demographics. Howe

applicationsarxiv-cs-cv

Olaf-World: Orienting Latent Actions for Video World Modeling

arXiv:2602.10104v2 Announce Type: replace-cross Abstract: Scaling action-controllable world models is limited by the scarcity of action labels. While latent action learning promises to extract control

Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks

arXiv:2605.26526v1 Announce Type: new Abstract: Recent defenses for safeguarding open-weight large language models (LLMs) are intended to prevent adversarial usage. Underlying these defenses is an ass

safetyarxiv-cs-lg

PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesis

arXiv:2605.27258v1 Announce Type: cross Abstract: Building state-of-the-art text-to-speech (TTS) systems typically demands millions of hours of proprietary data and complex multi-stage architectures,

Probing Minimalist Phase Structure in LLMs: What Universal Dependencies Cannot Represent

arXiv:2605.26431v1 Announce Type: new Abstract: Structural probes train on Universal Dependencies (UD), which does not encode formal-syntactic abstractions such as phase boundaries or phase-internal c

researcharxiv-cs-cl

Agents

Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction

arXiv:2602.00959v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can be seen as compressed knowledge bases, but it remains unclear what knowledge they truly contain and how far t

agentsarxiv-cs-cl

SCENT: Aligning Mass Spectra with Molecular Structure for Olfactory Perception

arXiv:2605.27009v1 Announce Type: new Abstract: Predicting human olfactory perception from molecular structure has seen remarkable progress, yet these approaches require explicit chemical structure at

safetyarxiv-cs-lg

SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens

model-releasesarxiv-cs-cl

arXiv:2508.05305v2 Announce Type: replace Abstract: The recently proposed Large Concept Model (LCM) generates text by predicting a sequence of sentence-level embeddings and training with either mean-s

Local Ai

Source-Free Domain Adaptation for Geospatial Point Cloud Semantic Segmentation

arXiv:2601.08375v2 Announce Type: replace Abstract: Semantic segmentation of 3D geospatial point clouds is fundamental to remote sensing applications, yet domain shifts caused by regional and acquisit

local-aiarxiv-cs-cv

Structured Relational Reasoning for Group Activity Assessment

model-releasesarxiv-cs-cv

arXiv:2508.07996v2 Announce Type: replace Abstract: Group Activity Detection (GAD) involves recognizing social groups and their collective behaviors in videos. Vision Foundation Models (VFMs), like DI

Temporal Simultaneity Predicts Annotation Quality in Sentiment Corpora

model-releasesarxiv-cs-cl

arXiv:2605.27239v1 Announce Type: new Abstract: Annotation quality is difficult to sustain when campaigns span weeks or months with small annotator pools. We present a Setswana sentiment dataset of 3,

Tutorials

Transformers Can Learn Posterior Predictive Distributions In-Context

arXiv:2605.26713v1 Announce Type: cross Abstract: Prior-data fitted networks (PFNs) have recently emerged as a powerful approach for Bayesian prediction tasks, approximating the posterior predictive d

tutorialsarxiv-cs-lg

Hardware

A universal vision transformer for fast calorimeter simulations

arXiv:2601.05289v2 Announce Type: replace-cross Abstract: The high-dimensional complex nature of detectors makes fast calorimeter simulations a prime application for modern generative machine learning

hardwarearxiv-cs-lg

A World Model of Radiologist Reading for Medical Image Representation Learning

arXiv:2605.23992v1 Announce Type: cross Abstract: Radiologist eye-tracking data provide a rich record of how experts search, compare, and accumulate evidence during image reading; yet, existing method

AnatomiX, an Anatomy-Aware Grounded Multimodal Large Language Model for Chest X-Ray Interpretation

arXiv:2601.03191v3 Announce Type: replace-cross Abstract: Multimodal medical large language models have shown substantial progress in chest X-ray interpretation but continue to face challenges in spat

Agents

Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundations

arXiv:2605.25620v1 Announce Type: new Abstract: World models enable agents to predict future dynamics conditioned on actions, making the choice of latent representation central to planning and control

agentsarxiv-cs-ai

Benchmarking Pathology Foundation Models for Spatial Domain Understanding

arXiv:2605.25764v1 Announce Type: cross Abstract: Pathology foundation models (PFMs) have emerged as a core approach for learning transferable representations from whole slide images (WSIs), and they

Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork

arXiv:2605.24423v1 Announce Type: new Abstract: In-Context Reinforcement Learning (ICRL) has enabled foundation agents to adapt instantaneously to novel tasks, yet its efficacy in Ad-Hoc Teamwork (AHT

D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentation

arXiv:2605.25022v1 Announce Type: cross Abstract: Dataset distillation (DD) aims to compress large-scale datasets into compact synthetic sets while preserving training efficacy. However, existing stud

Don't Retrain, Just Reuse: Recovering Dual-Target Molecules from Single-Target Diffusion Models

arXiv:2605.25681v1 Announce Type: cross Abstract: Designing a single molecule that modulates two targets is a promising strategy for polypharmacology, but it remains substantially harder than standard

DUEL: Adversarial Self-Play for Multimodal Reasoning

arXiv:2605.24794v1 Announce Type: cross Abstract: Reinforcement learning (RL) has emerged as an effective paradigm for improving the reasoning capability of vision-language models (VLMs). However, RL-

researcharxiv-cs-cl

E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving

arXiv:2512.04733v2 Announce Type: replace-cross Abstract: End-to-end autonomous driving (AD) systems increasingly adopt vision-language-action (VLA) models, yet they typically ignore the passenger's e

Emergent Analogical Reasoning in Transformers

arXiv:2602.01992v4 Announce Type: replace Abstract: Analogy is a central faculty of human intelligence, enabling abstract patterns discovered in one domain to be applied to another. Despite its centra

Empirical Analysis and Detection of Hallucinations in LLM-Generated Bug Report Summaries

arXiv:2605.24137v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used to generate summaries of software bug reports, including sections such as Steps-to-Reproduce (S2R),

Fine-Tuning Masked Diffusion for Provable Self-Correction

arXiv:2510.01384v4 Announce Type: replace Abstract: A natural desideratum for generative models is self-correction--detecting and revising low-quality tokens at inference. While Masked Diffusion Model

Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay

arXiv:2605.26097v1 Announce Type: new Abstract: Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required repla

GEESE: Genotype-aware End-to-End Spatio-temporal Embedding for Behavioral Phenotyping

arXiv:2605.24370v1 Announce Type: new Abstract: Behavioral phenotyping of genetic animal models currently requires labor-intensive manual feature engineering that limits reproducibility and scalabilit

Grouter: Decoupling Routing from Representation for Accelerated MoE Training

arXiv:2603.06626v2 Announce Type: replace-cross Abstract: Traditional Mixture-of-Experts (MoE) training typically proceeds without any structural priors, effectively requiring the model to simultaneou

Hidden-State Privacy Has an Empty Middle

arXiv:2605.24042v1 Announce Type: cross Abstract: Of 1{,}536 Gaussian release covariances we tested for single-layer hidden-state privacy, zero achieve both moderate utility and moderate privacy again

Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization

arXiv:2605.25203v1 Announce Type: cross Abstract: We apply the influence-adaptive Walsh geometry of a companion theory paper (arXiv:2605.01637) to extreme low-bit weight-only LLM quantization. The rec

Intrinsically Interpretable Attention via Sparse Post-Training

arXiv:2512.05865v5 Announce Type: replace-cross Abstract: We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsi

Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learning

arXiv:2605.25267v1 Announce Type: cross Abstract: Safe in-context reinforcement learning (ICRL) adapts online from interaction history without test-time parameter updates while controlling episode cos

Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learning

arXiv:2605.26012v1 Announce Type: cross Abstract: Deep reinforcement learning (RL) agents commonly rely on high-dimensional neural representations, despite growing evidence that task-relevant value an

Learning Protein Structure-Function Relationships through Knowledge-guided Representation Decomposition

arXiv:2605.23960v1 Announce Type: cross Abstract: Proteins encode diverse functions within complex three-dimensional structures, yet most deep learning representations remain highly entangled, obscuri

LLMTabBench: Evaluating LLMs on Binary Tabular Classification From Zero to Few Shots

model-releasesarxiv-cs-lg

arXiv:2605.24417v1 Announce Type: new Abstract: Supervised classification for tabular data remains a core machine learning task, yet its reliance on large labeled datasets limits applicability in data

Local Ai

Lngram: N-gram Conditional Memory in Latent Space

arXiv:2605.24869v1 Announce Type: new Abstract: Sequence modeling requires both compositional reasoning and local static knowledge retrieval, yet standard Transformers handle both through dense comput

local-aiarxiv-cs-cl

MAGIC: Multimodal Alignment & Grounding-aware Instruction Coreset for Vision-Language Models

arXiv:2605.26004v1 Announce Type: cross Abstract: Instruction tuning of large vision-language models (LVLMs) increasingly depends on massive multimodal corpora, yet these datasets contain samples with

safetyarxiv-cs-cl

Momentum Streams for Optimizer-Inspired Transformers

arXiv:2605.24425v1 Announce Type: cross Abstract: The residual update of a pre-norm Transformer layer admits an interpretation as one step of a first-order optimizer acting on a surrogate token energy

Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation

arXiv:2605.23961v1 Announce Type: cross Abstract: The design of RNA molecules that interact with specific proteins is a critical challenge in experimental and computational biology. Despite recent pro

Nano World Models: A Minimalist Implementation of Future Video Prediction

arXiv:2605.23993v1 Announce Type: cross Abstract: World models have become a central paradigm for learning predictive simulators that support generation, planning, and decision-making. Yet, despite ra

Peak-Then-Collapse and the Four Interface Channels of Knowledge-Graph Tool Use

arXiv:2605.26037v1 Announce Type: new Abstract: We test the standard RLVR tool-use recipe -- GRPO on Qwen2.5-7B-Instruct -- on a deliberately minimal knowledge-graph tool API: four Freebase navigation

safetyarxiv-cs-cl

Physen-Noise2Noise: Physics-Guided Self-Supervised Defocus Deblurring with Bias Correction under Low-Light Conditions

model-releasesarxiv-cs-lg

arXiv:2605.24590v1 Announce Type: cross Abstract: Low-light, long-exposure defocus deblurring remains a challenging problem due to the simultaneous presence of severe blur and complex biased noise. Ex

Representation-Guided Discrete Molecular Graph Retrosynthesis

arXiv:2605.24428v1 Announce Type: new Abstract: Stochastic process-based molecular graph generators have become the state of the art for template-free single-step retrosynthesis. However, these models

SliceWorld: A Predictive and Controllable World-State Model for CT Report Generation

arXiv:2605.24371v1 Announce Type: cross Abstract: CT report generation (CTRG) requires models to summarize three-dimensional anatomical context and pathological findings from hundreds of axial slices.

safetyarxiv-cs-cl

Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation

arXiv:2605.25488v1 Announce Type: cross Abstract: Audio-driven talking-head generation has achieved remarkable progress with recent models such as AniTalker, FLOAT, and Sonic. Despite their success, m

Thaka at KSAA-2026 Task 2: Regularized Fine-Tuning for Arabic Speech Diacritization

arXiv:2605.25928v1 Announce Type: new Abstract: We describe the winning system for Task 2 of the KSAA-2026 Shared Task on Arabic Speech Dictation with Automatic Diacritization. The task requires produ

researcharxiv-cs-cl