Tutorials
Self-host a validated coding assistant with NeMo Guardrails
NVIDIA tutorial: StarCoder2-7B NIM behind NeMo Guardrails, plus CI checks and commit trailers so generated patches stay auditable.
Lovina Dmello (NVIDIA, 29 July 2026) walks through a self-hosted coding assistant for networks that cannot send source to a vendor API.
Stack
- StarCoder2-7B as an NVIDIA NIM with an OpenAI-compatible completions endpoint; source stays on local GPUs
- NeMo Guardrails in front, refusing paths marked human-only (auth, payments, cryptography) before the model runs
- CI gate (
ai-assisted): unit tests, SAST, secret scanning, hallucinated-dependency detection, license checks - Commit trailers record assistant name and scope; Prometheus/Grafana watch defect escape rate against a human baseline
Pilot hardware in the post: a supported NVIDIA GPU with at least 24 GB (A10, L4, L40S, or A100). H100/H200 are the high-throughput profile, not required to start.
"The model is not the control plane."