Hardware
Are inference providers able to make any margins?
Spoke with many providers who lurk in this sub, plus met folks who work in inference. For a 10k monthly revenue, a provider was able to retain only 200 dollars in profit due to GPU costs. Their custom
Spoke with many providers who lurk in this sub, plus met folks who work in inference. For a 10k monthly revenue, a provider was able to retain only 200 dollars in profit due to GPU costs. Their customers were negotiating the prices down to what other players cost for same, and it feels like major players are running a distribution game at low or negative margins. Even though this seems like a billion dollar market, the unavailablity of compute, plus cost and competition, makes the business seems not sexy enough to start with. However, the software layers around it such as optimisations for SLAs continue to enjoy good margins. Any thoughts? submitted by /u/metalvendetta [link] [comments]
Related
- GPU prices haven't stopped climbing for 3 weeks straight across the EU, here's the data
- Most AI teams treat compute as a commodity. It's not.
- Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2
Source: r/LocalLLaMA | 2026-09-10