[D] 60% MatMul Performance Bug in cuBLAS on RTX 5090 [D]

A bug was identified in NVIDIA's cuBLAS library where cublasSgemmStridedBatched dispatches the same suboptimal cutlass_80_simt_sgemm_128x32_8x5 kernel for every batched FP32 workload from 256×256 to 8192×8192×8, running at only ~40% FMA pipe utilization. The dispatcher fails to escalate to a larger tile at any threshold, despite the same cuBLAS binary correctly escalating on other sm_120 and sm_90 GPUs such as the RTX PRO 6000 Blackwell. During benchmarking, a custom TMA kernel achieved batched-mode numbers on the 5090 that were 50–60% higher than cuBLAS at sizes from 1024 to 8192 , revealing the extent of the performance loss for local LLM inference and other FP32 deep learning workloads on the RTX 5090.

AI Systems Performance Engineering by Chris Fregly - is it worth it? [D]
Anyone have an S3-compatible store that actually saturates H100s without the AWS egress tax? [R]
What if your HNSW index stored 3-bit embeddings instead of float32? [R]
[[p-pca-before-truncation-makes-non-matryoshka-embeddings-comp|[P] PCA before truncation makes non-Matryoshka embeddings compressible: results on BGE-M3 [P]]]

Source: research

Related