Hardware
Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson
NVIDIA Jetson can run compact open‑source reasoning models such as Nemotron 3.5 Lightning (a mixture‑of‑experts architecture with 30 B total parameters but only ~3 B active per token) and dense Qwen 3
NVIDIA Jetson can run compact open‑source reasoning models such as Nemotron 3.5 Lightning (a mixture‑of‑experts architecture with 30 B total parameters but only ~3 B active per token) and dense Qwen 3.8‑27B (all 27 B parameters activated), giving edge devices capabilities that previously required data‑center GPUs. Applying NVFP4 quantization together with speculative decoding—DSpark for Nemotron and DFlash2 for Qwen—reduces memory, computation, and can deliver up to a 6.28× decode throughput speedup over BF16 on Jetson; however, optimal settings must be validated against workload‑specific prompts. Developers should consult the Jetson AI Lab Model page, benchmark results, recommended recipes, and tutorials on deployment and speculative decoding for guidance when integrating these models into edge applications.
Related
- Deploy an Open Model from Checkpoint to Inference in Two Commands with NVIDIA TensorRT Model Connect
- NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents
- Deploy Agentic-Ready AI at the Edge with Memory Efficiency in NVIDIA JetPack 7.2
Source: NVIDIA Developer | 2026-09-04