Local Ai
graphsignal-top: a `top` for inference GPUs in the open-source Graphsignal profiler, kernels ranked by time, live, with readable names
A few weeks ago we posted the open-source Graphsignal profiler here: wrap vllm serve / sglang serve / whatever in graphsignal-run and it serves everything it measures as JSON, mainly so an agent can r
A few weeks ago we posted the open-source Graphsignal profiler here: wrap vllm serve / sglang serve / whatever in graphsignal-run and it serves everything it measures as JSON, mainly so an agent can read it in a loop. Turns out we kept opening the JSON ourselves to see what the GPU was doing. So now there is graphsignal-top: graphsignal-run vllm serve --port 8000 # one terminal graphsignal-top # another What you get: per-GPU util over time, memory, power, throttling; the engine's running/waiting requests, tok/s and TTFT/TPOT p50/p95; a GPU time split (kernels / graphs / memcpy / sync); and every kernel ranked by time over the last 5 s with a readable name and category, not just the mangled symbol. Tabs for CUDA graph replays, transfers and syncs. Screenshot is Qwen3-32B on 2x RTX PRO 6000, tp 2, over PCIe. NCCL all-reduce is a quarter of GPU time and half the wall time is host sync waiting on it. You see that in one glance. Works remotely too: graphsignal-top --host gpu-box against a profiler started with --listen-host, or port-forward over ssh. --once prints a single frame if you just want it in a log. Repo: https://github.com/graphsignal/graphsignal submitted by /u/l0g1cs [link] [comments]
Related
- I compared all specs of the major GPUs/machines that are being used here, because bandwidth is not everything. Some of ya'll need a reality check.
- GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference
- I pushed Qwen3.8-27B to 2.000 prefill per second and 132 decode per second on A RTX 3090.
- Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.
Source: r/LocalLLaMA | 2026-10-12