Local Ai
LFM 2.5 230M running at 1440 tok/s in-browser through a custom backend
Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for the specific hardwa
Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for the specific hardware of the device. The Nvidia kernels are aggressively fused into a multi-pass architecture, while the Apple Silicon kernels are created as a fused mega-kernel to minimize the Tile Based Deferred Rendering (TBDR) overhead on WebGPU. Demo: https://warp.sipp.sh RTX 3090 (webgpu) M4 (webgpu) LFM 2.5 230M 1400-1500 tok/s 400-500 tok/s Bonsai 1.7B 500-600 tok/s 100-150 tok/s This is still in active development, and I'll be folding this into the Sipp library in the coming weeks. submitted by /u/lordhiggsboson [link] [comments]
Related
- Bonsai 27B: 1-bit dense LLM running locally in your browser using custom WebGPU kernels
- I bundled a fully local LLM inside my Unity game. No internet, no cloud, no API key. The conversation is the gameplay.
- Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.
Source: r/LocalLLaMA | 2026-07-25