Hardware
Given how common RTX 3090 use is for LLMs, why don't we see more INT8 W8A8 models ?
Based on https://huggingface.co/hardware, the RTX 3090 is the second most used GPU by LLM enthusiasts. Because RTX 3090 has native INT8 tensors cores, it can provide better performance with INT8 W8A8.
Based on https://huggingface.co/hardware, the RTX 3090 is the second most used GPU by LLM enthusiasts. Because RTX 3090 has native INT8 tensors cores, it can provide better performance with INT8 W8A8. However people seems to default to FP8 or smaller quants anyway. I suppose I am missing information that explains why ? submitted by /u/TheOnlyBen2 [link] [comments]
Related
- What do you guys do for GPU Kernels?
- Qwen3.8 27B C8 at 972 TG / 5,680 PP on 4x MI100 rig ($6.5k) using my new INT8 vLLM fork
- The official GLM-5.2 NVFP4 from NVIDIA is now available. Curious how it compares to other quantizations. https://huggingface.co/nvidia/GLM-5…
Source: r/LocalLLaMA | 2026-09-01