Search: “pretraining”

Scaling Online Complex Event Detection with Synthetic Supervision and Mamba-Based Neural Algorithmic Reasoning

model-releasesarxiv-cs-lg

arXiv:2502.07250v3 Announce Type: replace Abstract: Modern machine learning models excel at detecting individual actions, sounds, or scene attributes from short, localized observations. However, many

14 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation

DGX agent

arXiv:2512.08897v2 Announce Type: replace Abstract: Content-aware layout generation is a critical task in graphic design automation, focused on creating visually appealing arrangements of elements tha

applicationsarxiv-cs-cv

14 Sept 2026

Local Ai

internlm/Intern-S2 · Hugging Face

from internlm: We introduce Intern-S2-397B, our most capable multimodal foundation model for scientific intelligence and long-horizon agents. Intern-S2-397B scales along three critical dimensions: pre

local-air-localllama

13 Sept 2026

Is there still strong interest in a dense 9b model?

model-releasesr-localllama

I have a full model, it's ready to train. It's ~9b parameters. 9.4b to be more exact. That includes a 1/2/3 Engram table, Moonshot's AttnRes modeling, and RoPE / NoPE layering at 3:1 as more or less v

13 Sept 2026

Looped transformers are a popular architecture topic right now. This new technical report extends the loop across tokens. Recurrent Looped T…

Looped transformers are a popular architecture topic right now. This new technical report extends the loop across tokens. Recurrent Looped Transformer (RLT) makes the decoder recurrent over every toke

researchdair-ai--x

13 Sept 2026

Hardware

Characterizing Job Power Elasticity for Power-Flexible AI Training

arXiv:2609.11542v1 Announce Type: new Abstract: Large language model (LLM) training is among the fastest-growing sources of electricity demand in modern data centers, and power availability is a prima

hardwarearxiv-cs-ai

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

model-releasesarxiv-cs-ai

arXiv:2609.11859v1 Announce Type: new Abstract: How does a language model's dependence on query-routing information and target knowledge change as it answers a question? We study this question through

Applications

The Coxon psyop was a decade and more than a billion dollars in the making. Here's the story. Coefficient Giving (formerly Open Philanthropy…

applicationsyann-lecun--x

The Coxon psyop was a decade and more than a billion dollars in the making. Here's the story. Coefficient Giving (formerly Open Philanthropy) is the grant-providing vehicle of Dustin Moskowitz. CG is

“We can’t slow down because China will beat us.” We can slow down. Chinese AI is waterskiing behind the US. Their AI capabilities come at a …

“We can’t slow down because China will beat us.” We can slow down. Chinese AI is waterskiing behind the US. Their AI capabilities come at a delay since they mainly come from distilling US AIs. If we s

safetydan-hendrycks--x

Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks

arXiv:2609.10608v1 Announce Type: cross Abstract: Diffusion models have achieved remarkable success in generative tasks across various areas, however their training process raises significant privacy

safetyarxiv-cs-lg

Anthropic: Most hypocritical company in history? 👉 Screams about how AI will kill everyone, while racing ahead. 👉 Moans about distillation…

Anthropic: Most hypocritical company in history? 👉 Screams about how AI will kill everyone, while racing ahead. 👉 Moans about distillation while distilling the world’s books, articles, and other writi

safetygary-marcus--x

Local Ai

BruNet: A Cross-Domain Transfer Framework for Bruise Segmentation

arXiv:2609.11463v1 Announce Type: new Abstract: Segmenting bruises is a challenging task in medical imaging due to limited data and annotations, diffuse boundaries, and highly variable appearance. In

local-aiarxiv-cs-cv

Applications

Combining Synthetic and Real Data for Low-Resource Historical OCR: A Manchu Case Study

arXiv:2609.11495v1 Announce Type: new Abstract: Manchu, now critically endangered, was one of the principal languages of the Qing empire (1636-1912), and its extensive archival record is increasingly

applicationsarxiv-cs-lg

Detectable Only Where It Is Confounded: What Verified Duplication Counts Say About Membership Evidence in Language Models

arXiv:2609.10830v1 Announce Type: new Abstract: When a language model finds a sentence unusually cheap to predict, it is tempting to conclude that the sentence was in its training data. Almost every p

researcharxiv-cs-cl

E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets

model-releasesarxiv-cs-cl

arXiv:2609.11334v1 Announce Type: new Abstract: Natural Language Inference processes pairs of sentences to extract their semantic relations. NLI has been a hot research topic, integrated as a main com

Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting

arXiv:2609.11872v1 Announce Type: cross Abstract: Continuous glucose monitoring (CGM) provides high-frequency measurements of glucose dynamics and enables short-term glucose forecasting for diabetes m

Agents

ExpTest: Loss-Curve Hypothesis Testing for Autonomous Learning-Rate Selection in Deep Neural Networks

arXiv:2411.16975v2 Announce Type: replace Abstract: Hyperparameter tuning remains a significant challenge in the training of deep neural networks (DNNs), requiring manual search or time-intensive grid

agentsarxiv-cs-lg

FARM: Reading Failure Signals from the Internal Predictive States of a Frozen Robotic World Model

model-releasesarxiv-cs-ro

arXiv:2609.11445v1 Announce Type: new Abstract: Reliable robot deployment requires online failure monitoring, yet existing monitors mainly derive risk from proxy signals or train dedicated monitoring

GRADE: Single-Frame Generative Radar Depth Estimation Under Visual Degradation

arXiv:2609.10756v1 Announce Type: new Abstract: Dense 3D depth perception fails under smoke, fog, and darkness because optical sensors cannot penetrate airborne particulates. mmWave radar remains usab

HiPerViT: A Hierarchical Perceiver-Vision Transformer Architecture for Multi-Scale Texture Recognition

arXiv:2609.10917v1 Announce Type: new Abstract: Texture recognition remains challenging for modern vision models because discriminative evidence is often carried by higher-order spatial statistics rat

InstantHDR: Single-forward Gaussian Splatting Initialization for HDR 3D Reconstruction

arXiv:2603.11298v3 Announce Type: replace Abstract: High dynamic range (HDR) novel view synthesis (NVS) aims to reconstruct HDR scenes from multi-exposure low dynamic range (LDR) images. Existing HDR

Learning/RSI through ngrams?

model-releasesr-localllama

Hey gang, im wondering if you in theory could use ngrams as seen with Qwen 3.8 Flash or DS4.1 in order to dynamically train the model? Normally the ngram embeddings behave similar to a lookup table of

Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling

arXiv:2609.11804v1 Announce Type: new Abstract: Visual Autoregressive Models (VAR) generate images through next-scale prediction, producing all tokens within each scale in parallel. We show that this

Measuring the Value of World-Model Updates: A Counterfactual Utility Protocol for Continual Adaptation

arXiv:2609.10954v1 Announce Type: new Abstract: Continual world models must decide whether new data justify changing the model. Fixed replay schedules and prediction-error triggers specify when to upd

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

arXiv:2609.11842v1 Announce Type: new Abstract: Diffusion and flow-matching schedules control the signal and noise coefficients that mix data and noise along affine probability paths. Minimizing a kin

Overpainting: Localized Context-aware Diffusion Image Editing

arXiv:2609.10811v1 Announce Type: new Abstract: We present 'overpainting', an image editing operation which offers both control over the location of the edit and awareness of the previous content in t

RDDMPI: Residual Denoising Diffusion Model for Probabilistic Multivariate Time Series Imputation

model-releasesarxiv-cs-lg

arXiv:2609.11648v1 Announce Type: new Abstract: Multivariate time series imputation (MTSI) aims to recover missing values in temporal data composed of multiple interdependent variables. This problem i

Revisiting Avatar-As-Image: High-Fidelity Registration is All You Need

arXiv:2609.11722v1 Announce Type: new Abstract: The representation of 3D clothed humans as standardized 2D UV texture and displacement maps over an underlying body model has long been studied. This co

RIDE: Relocalization-Informed Depth Estimation with 3D Gaussian Splatting

arXiv:2609.11079v1 Announce Type: new Abstract: Render--match--PnP relocalization establishes correspondences between query image pixels and 3D map points for camera pose recovery, but their potential

researcharxiv-cs-ro

SG-Blend: Learning an Interpolation Between Improved Swish and GELU for Robust Neural Representations

arXiv:2505.23942v2 Announce Type: replace Abstract: Prevailing activation functions such as Swish and GELU tend toward domain-specific optima, Swish was discovered via neural architecture search on vi

safetyarxiv-cs-lg

SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control

arXiv:2609.11752v1 Announce Type: cross Abstract: For industrial content risk control, the real deployment constraint is not average accuracy but how much risk can be auto-handled under high precision

safetyarxiv-cs-cl

Tutorials

The information geometry of large language models is shared, learned, and controllable

arXiv:2609.11063v1 Announce Type: cross Abstract: Large language models learn similar behaviours, yet it remains unclear what structure they share or how to change one behaviour without disturbing oth

tutorialsarxiv-cs-cl

The Platonic brain bridge hypothesis: human brain networks as an architectural prior for omni models

model-releasesarxiv-cs-lg

arXiv:2609.10947v1 Announce Type: cross Abstract: We propose the Platonic brain bridge hypothesis: omni models, which process video, audio and text jointly like the brain, converge on brain-like repre

Agents

Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration

arXiv:2609.09418v1 Announce Type: new Abstract: World Action Models (WAMs) couple predictive world modeling with action generation, allowing anticipated future states to guide agent behavior. Although

agentsarxiv-cs-ai

Vision Transformer-Based Multi-Level Feature Fusion for Multi-Label Sewer Defect Classification

arXiv:2609.11375v1 Announce Type: new Abstract: Automated classification of sewer defects is essential for infrastructure condition assessment and maintenance decision-making, but existing deep learni

Zero-shot rib design: merging training-free generative prior with topology optimization

arXiv:2609.10643v1 Announce Type: new Abstract: Natural load-bearing patterns such as leaf venation, trabecular bone, and spider webs achieve high stiffness per unit mass, yet classical topology optim

Local Ai

A radiographic world model for clinical reasoning and evidence generation

arXiv:2609.07719v1 Announce Type: new Abstract: Medical imaging artificial intelligence (AI) is commonly developed as separate mappings from radiographs to diagnostic outputs or from clinical descript

local-aiarxiv-cs-ai

AgroVisNet: A lightweight Convolutional Network and the BD-PlantDX Expert-Validated Benchmark for Radish, Potato and Pointed Gourd Disease Classification

arXiv:2609.10469v1 Announce Type: new Abstract: Automated plant disease diagnosis is increasingly deployed on farmer-held devices in regions where agronomic expertise is scarce and network connectivit

Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

model-releasesarxiv-cs-ai

arXiv:2603.20957v4 Announce Type: replace-cross Abstract: Frontier LLM companies have repeatedly assured courts and regulators that their models do not store copies of training data. They further rely

Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition

arXiv:2609.10394v1 Announce Type: cross Abstract: Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the c

Decoupled Self-Forcing Distillation for Streaming Talking Head Generation

arXiv:2609.10317v1 Announce Type: new Abstract: Streaming talking-head generation produces each frame as its driving audio arrives, yet fidelity and efficiency have so far pulled in opposite direction

safetyarxiv-cs-cv

Deep and shallow biases in language models

arXiv:2609.09901v1 Announce Type: new Abstract: Large language models often repeatedly select the same answer even when many alternatives are plausible. Prior work treats this concentration as bias, b

safetyarxiv-cs-cl

Tutorials

DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivity

arXiv:2609.09254v1 Announce Type: new Abstract: Field-programmable gate arrays (FPGAs) enable efficient neural-network inference, but most deployment flows either accelerate multiply-accumulate operat

tutorialsarxiv-cs-lg

Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Images

arXiv:2511.18031v1 Announce Type: cross Abstract: Few-shot object detection (FSOD) aims to detect novel instances with only a limited number of labeled training samples, presenting a challenge that is

Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference

model-releasesarxiv-cs-ai

arXiv:2609.08189v1 Announce Type: new Abstract: Dynamic layer routing reduces the inference cost of Large Language Models (LLMs) by learning to skip layers for individual tokens. Existing methods, how

Edu-QuRating: Multi-Dimensional Educational Data Curation with Distilled Pairwise Judgements

model-releasesarxiv-cs-cl

arXiv:2609.09425v1 Announce Type: new Abstract: Educational data filters have become a practical way to improve language-model pre-training, but most filters treat educational value as a single scalar

// human-values.pseudo | ARTIFACT 0.1.0-experimental.1 | BASE 1.2.0-rc1 | EDITION ai-compact | NARRATOR System_Synthesis_Node_01 | PROVENANC…

safetyyohei-nakajima--x

Hybrid Quantum-Classical NLP Classification with Compact Semantic Representations: An Experimental Analysis of Representation Compression

arXiv:2609.10089v1 Announce Type: new Abstract: Large language and sentence-embedding models provide rich semantic representations, but their high dimensionality poses a challenge for near-term quantu