Model Releases
TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor
TensorRT Edge‑LLM ran Qwen 3.6‑27B on a single NVIDIA Jetson AGX Thor and completed the MLPerf Edge Agentic benchmark in 24 min 36 s—52.33 tokens/s—which is 6.4× faster than the llama.cpp reference (2
TensorRT Edge‑LLM ran Qwen 3.6‑27B on a single NVIDIA Jetson AGX Thor and completed the MLPerf Edge Agentic benchmark in 24 min 36 s—52.33 tokens/s—which is 6.4× faster than the llama.cpp reference (2 h 37 m). The acceleration comes from NVFP4 weight/activation quantization, FP8 KV‑cache, tree‑based multi‑token prediction, and cache reuse across agent turns, allowing ~96 % of prompt tokens to be served from a hot cache and delivering an additional ~40 % decoding speedup for function‑calling workloads. For deployment, the community can consult the TensorRT Edge‑LLM 0.9.1‑mlpinf branch, download the calibrated Qwen 3.6‑27B checkpoint, and review the MLCommons Edge Agentic example for detailed dataset and client configuration.
Related
- Local agentic coding Benchmark : Qwen 3.8 27B (in many weights quants / cache quants / engine / reasoning effort) vs others.
- pay attention anon. this is what local ai actually feels like in 2026. qwen 3.6 27b dense just knocked down the second test in my single fil…
- Qwen 35B-A3B MoE vs 27B dense in local coding tests: ~4× faster, much smaller quality gap than I expected
Source: NVIDIA Developer | 2026-09-16