Search: “pretraining”

Thoughts About Scaling Law - Z.ai

model-releasesr-localllama

Thoughts About Scaling Law Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn't a question that can be answered on its own. Parameter

19 Aug 2026

Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.17268v1 Announce Type: cross Abstract: Curriculum learning has been widely adopted in the post-training of large language models by organizing training data from easy to hard. However, its

researcharxiv-cs-ai

19 Aug 2026

What Tokens are Learned when Tokenization is Optimized Jointly with Language Modeling?

arXiv:2608.17325v1 Announce Type: new Abstract: Tokenization is a fundamental component of language modeling pipelines. Despite its importance, it is often fixed, even though it significantly impacts

researcharxiv-cs-cl

19 Aug 2026

A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models

arXiv:2608.15984v1 Announce Type: new Abstract: Motion Language Models (MoLMs) typically understand human motions by tokenizing 3D motion and processing the resulting tokens using a language model. Ho

applicationsarxiv-cs-cv

AeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning

arXiv:2608.14721v1 Announce Type: new Abstract: Vision-language models (VLMs) have been widely employed in understanding and reasoning tasks for unmanned aerial vehicles (UAVs). Existing UAV benchmark

AlignJEPA: Predictive Vision-Language Alignment for Remote Sensing Foundation Models

arXiv:2608.15456v1 Announce Type: new Abstract: Remote sensing (RS) foundation models provide transferable Earth observation representations across sensors, resolutions, and geographies, yet most rema

Anatomical and Physical Supervision for CT-less PET Attenuation Correction: BIC-MAC 2026 Challenge

arXiv:2608.15721v1 Announce Type: new Abstract: This report describes our submission to the Big Cross-Modal Attenuation Correction (BIC-MAC) 2026 Challenge for CT-less PET attenuation correction throu

Arm-Aware Guided Dexterous Grasp Generation with Arm-Agnostic Grasp Models

arXiv:2608.16351v1 Announce Type: new Abstract: Dexterous grasp generation that considers arm-related constraints is crucial in real-world scenarios involving arm environment collision avoidance, work

applicationsarxiv-cs-ro

Beyond Similarity Matching: Structured Reasoning for Open-Vocabulary Referring Segmentation in 3DGS

arXiv:2608.16103v1 Announce Type: new Abstract: Open-vocabulary referring segmentation in 3D Gaussian Splatting (3DGS) requires a neural model to select Gaussian primitives according to free-form lang

safetyarxiv-cs-cv

Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustness

arXiv:2506.15700v2 Announce Type: replace-cross Abstract: Control contraction metrics (CCMs)-defined by Riemannian metrics under which a closed-loop system is incrementally exponentially stable-offer

safetyarxiv-cs-ai

CrevasseSeg: A Label-Efficient UAV Crevasse Segmentation Framework

model-releasesarxiv-cs-lg

arXiv:2608.15790v1 Announce Type: new Abstract: Crevasse mapping from uncrewed aerial vehicle (UAV) imagery matters for glaciological research and for field safety in glaciated terrain. Yet, pixel-lev

Cross-View Urban Sensing: Mapping Subjective Streetscape Perception via AlphaEarth Embeddings and Urban Context

arXiv:2608.16310v1 Announce Type: new Abstract: Residents' perception of the urban streetscape is an important factor in public health, active mobility, and social wellbeing. Street view imagery (SVI)

CytoFormer: A Molecularly Supervised Cell Foundation Model for Histopathology Cell Classification

arXiv:2608.16718v1 Announce Type: new Abstract: Identifying cell types directly from routine haematoxylin and eosin (H&E) histology would enable single-cell analysis at scale, but training such models

Distinguishing AI-Generated Music from Edited Audio as a Hard-Negative Robustness Task

arXiv:2608.14916v1 Announce Type: cross Abstract: AI-generated music detectors are commonly evaluated against original songs, but real-world uploads are often remixed, re-encoded, pitch-shifted, or ot

applicationsarxiv-cs-ai

Domain-Specific Text Embedding Models for Entity Resolution

arXiv:2608.16161v1 Announce Type: cross Abstract: General-purpose text embedding models are designed to capture semantic similarity but are not optimised for distinguishing entity records that represe

applicationsarxiv-cs-ai

DRAFE: Domain-Robust Asymmetric Fusion of Heterogeneous Detection Transformers for Cross-City Fine-Grained Traffic Object Detection

arXiv:2608.16632v1 Announce Type: new Abstract: Deep learning-based object detectors are fundamental to intelligent transportation systems, enabling traffic monitoring, vehicle analytics, and infrastr

Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference

arXiv:2608.15282v1 Announce Type: cross Abstract: Earth observation foundation models (EOFMs) are emerging as reusable representation frameworks for data-driven retrieval, prediction and process model

Local Ai

ES3D: Embedding Semantics into 3D Space for Component-Aware Editing

arXiv:2608.15749v1 Announce Type: new Abstract: Existing 3D editing methods have made notable progress in controllability, yet they remain limited in several important ways. Most approaches rely on te

local-aiarxiv-cs-cv

Evolving Executable Pipeline Programs for AutoML with Language Models

model-releasesarxiv-cs-lg

arXiv:2608.16416v1 Announce Type: new Abstract: Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in

FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learning

arXiv:2608.16523v1 Announce Type: new Abstract: Event cameras generate asynchronous, high-frequency data streams offering spatially sparse information at lower latency than traditional cameras.In prin

FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences

arXiv:2608.15296v1 Announce Type: new Abstract: Audio-driven 3D facial animation is essential for advancing immersion and interactivity in virtual experiences. Although recent advances have shown prom

safetyarxiv-cs-cv

Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats

arXiv:2608.15651v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) represents 3D content with anisotropic primitives that jointly encode geometry and appearance. Fixed-budget encoders consum

HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

arXiv:2608.16837v1 Announce Type: cross Abstract: Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation mo

safetyarxiv-cs-ai

Tutorials

Human Pose Estimation in Trampoline Gymnastics: How to Improve Performance on Extreme Poses

arXiv:2604.01322v2 Announce Type: replace Abstract: Trampoline gymnastics involves extreme human poses and uncommon viewpoints, on which state-of-the art pose estimation models tend to under-perform.

tutorialsarxiv-cs-cv

MAPLE: MoE Adaptive Plug-and-play Layer-wise Expert allocation

arXiv:2608.15299v1 Announce Type: cross Abstract: Sparsely-activated Mixture-of-Experts (MoE) Transformers universally fix the same number of routed experts across all layers, a convention that ignore

Measuring Structured Predictability in Neural Training Dynamics: A Cross-Regime Study

model-releasesarxiv-cs-lg

arXiv:2608.15483v1 Announce Type: new Abstract: Modern deep networks are trained through long update trajectories, yet their temporal organization remains less systematically characterized than archit

mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR

arXiv:2602.22431v2 Announce Type: replace-cross Abstract: Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech.

researcharxiv-cs-lg

MoE Router-Guided Clustering for Heterogeneous Federated Instruction Tuning

arXiv:2608.15311v1 Announce Type: new Abstract: Federated instruction fine-tuning enables Large Language Models (LLMs) to adapt to decentralized, privacy-sensitive data without requiring data sharing.

OvDSGG: End-to-End Open-Vocabulary Dynamic Scene Graph Generation

arXiv:2608.14835v1 Announce Type: new Abstract: Dynamic scene graphs (DSGs) capture spatio-temporal interactions across videos as langlesubject, predicate, objectrangle triplets, and underpin downstre

P2E-VQ: ECG-linked representation augmentation for PPG via discrete patch retrieval

arXiv:2608.14656v1 Announce Type: cross Abstract: Photoplethysmography (PPG) is widely used in consumer wearables because of its low cost and ease of acquisition. However, unlike electrocardiography (

researcharxiv-cs-ai

Local Ai

PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulation

arXiv:2608.15026v1 Announce Type: new Abstract: Post-training of vision-language-action (VLA) models typically relies on expert demonstrations and policy interaction trajectories. However, in long-hor

local-aiarxiv-cs-ro

PertMind: Eliciting Emergent Biological Reasoning in LLM via Reinforcement Learning on Cellular Perturbation Data

arXiv:2608.16419v1 Announce Type: cross Abstract: Large language models can describe mechanisms, yet scalable post-training still depends on costly, manually curated biological reasoning traces. Here

researcharxiv-cs-ai

Picking the Right Image to Classify: Reliable-Input Selection in Teledermatology

arXiv:2608.16198v1 Announce Type: cross Abstract: Dermatology models face distribution shifts in teledermatology settings, where submitted images differ from the training data in lighting, angle, dist

PickStyle: Video-to-Video Style Transfer with Context-Style Adapters

arXiv:2510.07546v2 Announce Type: replace Abstract: We address the task of video style transfer with diffusion models, where the goal is to preserve the context of an input video while rendering it in

safetyarxiv-cs-cv

PosterText: Towards Unified Visual Text Generation and Editing for E-commerce Poster

arXiv:2608.16289v1 Announce Type: new Abstract: Automated e-commerce poster design requires both high-quality poster generation and flexible editing of existing designs. However, most existing methods

Quantifying Depth Sufficiency in Residual Neural Networks: A First-Order Criterion

arXiv:2608.14664v1 Announce Type: new Abstract: How can we determine whether a trained neural network is already deep enough? We study this under a fixed function-preserving residual-growth protocol s

researcharxiv-cs-lg

Tutorials

RETRACE: Resilience-Guided Trait-Conditioned Craving Estimation from Wearable Physiology in Opioid Use Disorder

arXiv:2608.14947v1 Announce Type: new Abstract: Detecting opioid craving from wearable physiological signals is critical yet difficult, with the potential to support proactive interventions for indivi

tutorialsarxiv-cs-ai

Robo-Dopamine 2.0: History-Conditioned and OOD-Aware Process Reward Modeling for Robotic Manipulation

arXiv:2608.15680v1 Announce Type: cross Abstract: Vision-language-action (VLA) models improve robotic manipulation but remain vulnerable to compounding errors, scene changes, and off-trajectory states

Agents

RoboStriker: Latent-Space Strategic Games for Autonomous Humanoid Boxing

arXiv:2608.16195v1 Announce Type: new Abstract: Achieving human-level competitive intelligence and physical agility in humanoid robots remains a profound challenge, particularly in contact-rich and hi

agentsarxiv-cs-ro

SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry

arXiv:2608.16287v1 Announce Type: new Abstract: Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representatio

safetyarxiv-cs-lg

Local Ai

SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable

arXiv:2608.14922v1 Announce Type: cross Abstract: Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools

local-aiarxiv-cs-ai

SQuad: Sub-Quadratic Attention Distillation for Efficient Video Generation

arXiv:2608.16585v1 Announce Type: new Abstract: Video Diffusion Transformers (DiTs) spend most of their compute inside the Self-Attention operation, whose cost grows quadratically, O(n^2), with the nu

The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese

model-releasesarxiv-cs-cl

arXiv:2607.10745v2 Announce Type: replace Abstract: This paper presents the first ChineseBabyLM Challenge, organized as part of NLPCC 2026. The challenge asked participants to train language models fr

The Trade-off Between Covariate Dependence and Latent Structure in Representation Learning

arXiv:2608.16245v1 Announce Type: new Abstract: Disentangled representation learning seeks latent representations whose indicidual dimensions each align with a distinct covariate. Unsupervised approac

safetyarxiv-cs-lg

Tutorials

Train Yourself as an LLM: Exploring Effects of AI Literacy on Persuasion via Role-playing LLM Training

arXiv:2604.02637v2 Announce Type: replace Abstract: As large language models (LLMs) become increasingly persuasive, there is concern that people's opinions and decisions may be influenced across vario

tutorialsarxiv-cs-cl

ViTaR: Visuo-Tactile Residual Adaptation for Foundation VLA Manipulation

model-releasesarxiv-cs-ro

arXiv:2608.15816v1 Announce Type: new Abstract: As Vision-Language-Action (VLA) models scale toward real-world deployment, contact-rich manipulation exposes a critical blind spot: these policies encod

Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data

arXiv:2608.14712v1 Announce Type: cross Abstract: Each row of a transformer's attention matrix is a probability distribution over tokens, and in trained models most of that probability lands on a sing

researcharxiv-cs-ai

Wolff-Parkinson-White Detection at 471:1 Class Imbalance: A Leakage-Controlled Study of the Data Bottleneck

arXiv:2608.14633v1 Announce Type: cross Abstract: Wolff-Parkinson-White (WPW) syndrome is a congenital cardiac pre-excitation, clinically important and often missed on the resting 12-lead ECG. Detecti

researcharxiv-cs-lg