Search: “pretraining”

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

arXiv:2605.19723v1 Announce Type: cross Abstract: Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimization

DGX agent

arXiv:2605.19782v1 Announce Type: new Abstract: LLM discovery and optimization systems are increasingly applied across domains, implementing a common propose-evaluate-revise loop. Such optimization or

hardwarearxiv-cs-ai

20 May 2026

Applications

Prognostic Value of Lung Ultrasound Biomarkers for Readmission Risk in Congestive Heart Failure: A Pilot Data-Driven Analysis

arXiv:2605.18878v1 Announce Type: cross Abstract: Hospital readmission within 30 days of discharge is a leading driver of morbidity, mortality, and avoidable healthcare expenditure in congestive heart

applicationsarxiv-cs-cv

Rapid patient-specific neural networks for intraoperative X-ray to volume registration

arXiv:2503.16309v2 Announce Type: replace-cross Abstract: Advanced navigation techniques in image-guided interventions and surgical robotics require the rapid and precise alignment of 3D preoperative

safetyarxiv-cs-cv

Real-World On-Vehicle Evaluation of Embedding-Based Anomaly Detection

arXiv:2605.19744v1 Announce Type: new Abstract: Detecting anomalies in traffic scenes is crucial for ensuring safety in autonomous driving, yet collecting representative anomalous data remains challen

ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models

arXiv:2605.19095v1 Announce Type: cross Abstract: Schedule-Free Learning has shown promise as a practical anytime training method for machine learning, showing success across dozens of standard benchm

Shaping the Prior: How Synthetic Task Distributions Determine Tabular Foundation Model Quality

arXiv:2605.18971v1 Announce Type: cross Abstract: What determines the quality of a tabular foundation model? Unlike language or vision, tabular foundation models acquire their inductive biases almost

Tutorials

Smartphone-based Circular Plot Sampling for Forest Inventory

arXiv:2605.19213v1 Announce Type: new Abstract: Circular sample plots are a cornerstone of forest inventory, yet accurate measurement of tree diameter at breast height (DBH) and spatial location withi

tutorialsarxiv-cs-cv

Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction

arXiv:2605.06270v2 Announce Type: replace Abstract: Feed-forward 3D reconstruction models based on Vision Transformers can directly estimate scene geometry and camera poses from a small set of input i

Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlock

arXiv:2605.19378v1 Announce Type: new Abstract: This paper systematically diagnoses the training failure modes of Token-Choice sparse Mixture-of-Experts (MoE) on video Diffusion Transformers. Starting

Applications

Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence

arXiv:2505.23747v2 Announce Type: replace-cross Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced performance on 2D visual tasks. However, improving

applicationsarxiv-cs-ai

SpecX: A Large-Scale Benchmark for Multi-Modal Spectroscopy and Cross-Paradigm Evaluation

arXiv:2605.18791v1 Announce Type: cross Abstract: Existing spectral benchmarks are limited in scale, modality alignment, and evaluation scope, and typically focus on either specialized models or multi

Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence

arXiv:2605.19727v1 Announce Type: new Abstract: Existing 3D foundation models typically align point clouds to frozen vision-language spaces like CLIP, which achieve strong cross-modal retrieval by com

The future of biology shouldn’t stay behind black-box APIs. Especially when it touches personal health. Whether you’re @bryan_johnson measur…

model-releasesclem-delangue--x

The future of biology shouldn’t stay behind black-box APIs. Especially when it touches personal health. Whether you’re @bryan_johnson measuring every biomarker, or @sytses openly sharing and analyzing

The Routing and Filtering Structure of Attention

arXiv:2605.18826v1 Announce Type: cross Abstract: The attention interaction matrix QK^{op} contains two entangled computations: a skew-symmetric component that redistributes information between positi

Applications

VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals

arXiv:2605.18837v1 Announce Type: cross Abstract: Wearable devices enable continuous health monitoring from multimodal signals, but real-world deployment is hindered by limited labeled data and pervas

applicationsarxiv-cs-ai

What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code

arXiv:2605.19762v1 Announce Type: new Abstract: Code has become a standard component of modern foundation language model (LM) training, yet its role beyond programming remains unclear. We revisit the

1GC-7RC: One Graphic Card -- Seven Research Challenges! How Good Are AI Agents at Doing Your Job?

arXiv:2605.17046v1 Announce Type: cross Abstract: Autonomous AI coding agents are becoming a core tool for ML practitioners in industry and research alike. Despite this growing adoption, no standardiz

A Large-Scale Study on the Accuracy vs Cost Trade-offs of Training and Evaluation Settings in Fine-Grained Image Recognition

arXiv:2605.18700v1 Announce Type: new Abstract: Prior work on fine-grained image recognition (FGIR) has established the importance of the backbone selection, but has neglected the accuracy-vs-cost tra

Adaptive Fused Prior Transfer for Controllable Generative Image Compression

arXiv:2605.16817v1 Announce Type: cross Abstract: Learned image compression has achieved competitive rate-distortion performance, but very-low-bitrate reconstruction remains difficult because the tran

AtlasVid: Efficient Ultra-High-Resolution Long Video Generation via Decoupled Global-Local Modeling

arXiv:2605.16649v1 Announce Type: new Abstract: Recent diffusion-based video generators have achieved remarkable visual fidelity and prompt controllability, yet scaling them to ultra-high-resolution (

Atoms as Language: VQ-Atom: Semantic Discretization for Molecular Representation Learning

arXiv:2605.16823v1 Announce Type: new Abstract: Molecular representation learning has become a central approach in AI-driven drug discovery, yet existing molecular tokenizations such as SMILES remain

researcharxiv-cs-lg

Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion

arXiv:2605.16579v1 Announce Type: new Abstract: Autoregressive (AR) video diffusion is a powerful paradigm for streaming and interactive video generation. However, its reliance on softmax self-attenti

Automatic Generation of High-Performance RL Environments

arXiv:2603.12145v2 Announce Type: replace-cross Abstract: Translating complex reinforcement learning (RL) environments into high-performance implementations has traditionally required months of specia

safetyarxiv-cs-ai

BELIEF: Structured Evidence Modeling and Uncertainty-Aware Fusion for Biomedical Question Answering

arXiv:2605.17435v1 Announce Type: new Abstract: Biomedical question answering often requires decisions from retrieved literature whose relevance, quality, and support for candidate answers are uneven.

researcharxiv-cs-cl

Beyond Patches: Global-aware Autoregressive Model for Multimodal Few-Shot Font Generation

arXiv:2601.01593v2 Announce Type: replace Abstract: Manual font design is an intricate process that transforms a stylistic visual concept into a coherent glyph set. This challenge persists in automate

CAB: Accelerating Flow and Diffusion Sampling via Rectification and Corrected Adams-Bashforth

arXiv:2605.16736v1 Announce Type: new Abstract: Flow and diffusion models achieve high-fidelity, high-resolution image synthesis, but often require many function evaluations (NFEs) at sampling time. E

Can Heterogeneous Language Models Be Fused?

arXiv:2604.01674v2 Announce Type: replace Abstract: Model merging aims to integrate multiple expert models into a single model that inherits their complementary strengths without incurring the inferen

Canonical Regularisation of Wide Feature-Learning Neural Networks

arXiv:2605.18180v1 Announce Type: cross Abstract: Wide neural networks in the feature-learning regime drive modern deep learning, and yet they remain far less studied than their kernel-regime counterp

safetyarxiv-cs-lg

Contrastive-SDXL: Annotation-Preserving Night-Time Augmentation for Pedestrian Detection

arXiv:2605.16406v1 Announce Type: new Abstract: Night-time pedestrian detection remains challenging because labelled night-time data are limited and large illumination differences make daytime-only tr

safetyarxiv-cs-cv

DepthPolyp: Pseudo-Depth Guided Lightweight Segmentation for Real-Time Colonoscopy

arXiv:2605.16519v1 Announce Type: new Abstract: Accurate polyp segmentation in colonoscopy is essential for early colorectal cancer detection, yet real-world clinical environments pose persistent chal

Face inpainting with Identity Preserving Latent Diffusion Models

arXiv:2605.16696v1 Announce Type: new Abstract: Face inpainting techniques recover missing or occluded facial regions in a visually realistic manner, but preserving the identity in the final output re

safetyarxiv-cs-cv

Factorized Latent Dynamics for Video JEPA: An Empirical Study of Auxiliary Objectives

arXiv:2605.17165v1 Announce Type: new Abstract: Joint-Embedding Predictive Architectures (JEPA) are a promising framework for self-supervised video representation learning, yet the behavior of auxilia

Fine-tuning Pocket-Aware Diffusion Models via Denoising Policy Optimization

arXiv:2605.17693v1 Announce Type: cross Abstract: Structure-based drug design has been accelerated by pocket-aware 3D generative models, yet most methods primarily fit the training distribution and ma

From BERT to T5: A Study of Named Entity Recognition

arXiv:2605.18462v1 Announce Type: new Abstract: Named entity recognition (NER) has been one of the essential preliminary steps in modern NLP applications. This report focuses on implementing the NER t

researcharxiv-cs-cl

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

arXiv:2605.16928v1 Announce Type: cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention. Existing efficient alternatives often rely ei

Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap

model-releasesarxiv-cs-lg

arXiv:2605.16349v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures achieve scalable capacity through sparse routing, yet the geometric structure of expert specialization remains po

Hardware

Guard: Scalable Straggler Detection and Node Health Management for Large-Scale Training

arXiv:2605.17879v1 Announce Type: cross Abstract: Training frontier-scale foundation models involves coordinating tens of thousands of GPUs over multi-month runs, where even minor performance degradat

hardwarearxiv-cs-ai

Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models

arXiv:2605.18601v1 Announce Type: new Abstract: Modern interactive video world models have achieved impressive visual fidelity, yet lack fine-grained multi-entity control and cross-entity, cross-world

Inducing Spatial Locality in Vision Transformers through the Training Protocol

arXiv:2605.16390v1 Announce Type: new Abstract: We investigate whether the training protocol can induce spatial locality in the early layers of a Vision Transformer (ViT) trained from scratch, without

KVCapsule: Efficient Sequential KV Cache Compression for Vision-Language Models with Asymmetric Redundancy

arXiv:2605.16439v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have emerged as a critical and fast-growing extension of Large Language Models (LLMs) that enable multimodal reasoning t

LACE: Latent Visual Representation for Cross-Embodiment Learning

arXiv:2605.16743v1 Announce Type: new Abstract: Cross-embodiment learning from human demonstrations is hindered by the visual gap between human and robot embodiments. While self-supervised learning (S

safetyarxiv-cs-ro

LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift

arXiv:2605.18541v1 Announce Type: new Abstract: Modeling hyperspectral imagery (HSI) across different sensors presents a fundamental challenge due to variations in wavelength coverage, band sampling,

LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation

arXiv:2410.13846v3 Announce Type: replace-cross Abstract: Scaling language models to handle longer contexts introduces substantial memory challenges due to the growing cost of key-value (KV) caches. M

MaskAttn-SDXL: Controllable Region-Level Text-To-Image Generation

arXiv:2509.15357v2 Announce Type: replace Abstract: Diffusion models have achieved strong results in text-to-image generation, but important limitations remain as prompts become more structured and mu

Neural equilibria for long-term prediction of nonlinear conservation laws

arXiv:2501.06933v3 Announce Type: replace Abstract: Nonlinear conservation laws govern a broad class of important physical systems in science and industry and are central to scientific machine learnin

safetyarxiv-cs-lg

NGM: A Plug-and-Play Training-Free Memory Module for LLMs

arXiv:2605.16893v1 Announce Type: new Abstract: Recent studies introduce conditional memory modules that decouple knowledge storage from neural computation, enabling more direct knowledge access. Comp

Tutorials

Observation-Aligned Mask Priors for Learning Physical Dynamics from Authentic Occlusions

arXiv:2605.16818v1 Announce Type: cross Abstract: Learning physical dynamics directly from incomplete observations is challenging because authentic occlusions are structured, sample-dependent, and oft

tutorialsarxiv-cs-ai