Local Ai
Has anyone here fiddled with TPUs for inference ?
I discovered recently that Google uses their own TPUs, like tiny ASIC cards like the toy ones that existed for bitcoin. And while it sounds inefficient the fact they use thousands of them because...th
I discovered recently that Google uses their own TPUs, like tiny ASIC cards like the toy ones that existed for bitcoin. And while it sounds inefficient the fact they use thousands of them because...they can...means at scale they aren't so bad. Has no one here given them a try? I see web search results of tiny ones that can be purchased and look like nvme adapted where I search them for 58 euros. Not sure what 40 TOPS translates to compared to my Nvidia 5060. But not just that, but the user experience with them, are they a nightmare to use ? submitted by /u/misanthrophiccunt [link] [comments]
Related
- Do people building local LLM rigs track RTX Ada/workstation card prices, or just consumer cards like the 5090?
- Has anyone bought a 3080 20GB mod recently?
- I compared all specs of the major GPUs/machines that are being used here, because bandwidth is not everything. Some of ya'll need a reality check.
- Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.
Source: r/LocalLLaMA | 2026-08-08