XeBoostLM: Native C++ Local LLMs for Intel NPUs & iGPUs

https://reddit.com/link/1wi2zuc/video/rs1a4sx92xph1/player Hey everyone, I’ve been experimenting with local LLMs on Intel hardware, and I wanted a hyper-optimized, standalone CLI tool that didn't rely on a heavy Python backend. So, I built XeBoostLM entirely in C++ using the OpenVINO GenAI library. It specifically targets Intel Core Ultra NPUs (AI Boost), Arc iGPUs, and CPUs with zero Python overhead during generation. [Link to GitHub: github.com/balaragavan2007/XeBoostLM] Here is what it currently does: Native C++ Execution: Compiled via MSVC. Extremely lightweight footprint. Smart Hardware Routing: Manually target NPU, GPU, or CPU, or let it run in HYBRID mode to balance throughput and host responsiveness. OpenAI-Compatible Server: Run xeboost serve to instantly spin up an HTTP server with Server-Sent Events (SSE) streaming. Plugs directly into Open WebUI and LibreChat out of the box. Dynamic Catalog: xeboost pull automatically fetches and verifies optimized INT4/INT8 models (Qwen 2.5, Phi-3.5, Llama 3.2, DeepSeek R1) using a cached remote catalog. Terminal REPL: Run xeboost chat for multi-turn conversations straight in the CLI. This is a v0.1.2 release. I just got the automated GitHub catalog and the REST API stabilized. I would love for anyone with an Intel chip to test it out, try the CLI, or connect it to their frontends and let me know how it performs! Feedback, feature requests, or PRs are highly welcome. submitted by /u/Spiritual-Ad-5916 [link] [comments]

Source: r/LocalLLaMA | 2026-09-16

Related