Hardware
TurboOCR: 270–1200 img/s OCR with Paddle + TensorRT (C++/CUDA, FP16) [P]
TurboOCR is a high-performance OCR project that combines PaddleOCR with NVIDIA TensorRT, implemented in C++ and CUDA, achieving throughput of 270–1,200 images per second using FP16 half-precision infe
TurboOCR is a high-performance OCR project that combines PaddleOCR with NVIDIA TensorRT, implemented in C++ and CUDA, achieving throughput of 270–1,200 images per second using FP16 half-precision inference. TensorRT acts as an optional acceleration layer on top of GPU inference, replacing supported subgraphs with optimized TRT engines at runtime, and FP16 operations offer 2× reduced memory bandwidth and storage, while GEMMs and convolutions with FP16 inputs can leverage Tensor Cores for significantly higher computational throughput over FP32. The project is shared on r/MachineLearning as a practical demonstration of pushing OCR inference speeds to production-scale levels through low-level GPU optimization rather than Python-based pipelines.
Related
- Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
- Running Large-Scale GPU Workloads on Kubernetes with Slurm
- $200/month is enough to buy an H100 GPU for 6 hours every workday
- Is an nvidia DGK Spark or similar worth it?
- AI Systems Performance Engineering by Chris Fregly - is it worth it? [D]
Source: r/MachineLearning | 2026-04-13