Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer

The QAD (Quantization‑Aware Distillation) pipeline for Nemotron 3.5 Lightning leverages NVIDIA Model Optimizer to aggressively compress the model with W4A16/NVFP4 quantization while keeping accuracy close to a BF16 baseline, reducing size from 66 GB to 22 GB and enabling up to 4× higher throughput. QAD consists of post‑training quantization followed by distillation where a low‑precision student is trained against a frozen full‑precision teacher using KL loss and simulated quantization, restoring nearly all accuracy lost by aggressive quantization. NVIDIA Model Optimizer and Megatron‑Bridge supply end‑to‑end workflows—including recipe selection, training configuration, and checkpoint export—to reproduce QAD and facilitate memory‑ and compute‑efficient deployment of Nemotron models.

Source: NVIDIA Developer | 2026-08-17