Knowledge catalogue
Meta released Muse Glimmer 30B: multimodal model for your Claw/Pi setups 🔥 we tested and fine-tuned the model for you, and shipped day-0 support in transformers and llama.cpp, including DFlash for 2-4
Zed editor has released a new version with integrated support for llama.cpp as a model provider, enabling users to run local language models within the editor. This update allows developers to use lla
This post acknowledges a pull request contribution (#59964) to the Zed code editor repository by user hanouticelina, as shared by Georgi Gerganov on X (formerly Twitter). The specific details of what
llama.cpp recently added DFlash support to its speculative decoding arsenal. Along with MTP, Eagle3 and various ngram-based techniques, the local model performance takes another step up. Special thank
Speculative decoding is a technique implemented in llama.cpp that speeds up inference by using a smaller, faster model to predict multiple tokens ahead, which a larger model then verifies in parallel,
In HF GGUF section of models, we are emphasizing MTP heads with its own sign 𝗠𝗧𝗣 llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance j
Gemma 4 MTP (Multi-Token Prediction) has been officially integrated into llama.cpp, enabling developers to build faster AI experiences by using the model with this inference framework. This merge allo
Highlighting recent advances in multi-GPU and tensor parallel support in llama.cpp Over the last few months llama.cpp maintainers and engineers from NVIDIA collaborated to improve the multi-GPU perfor
Computex 2024 is expected to feature significant announcements around local AI deployment, with major industry players including NVIDIA and Microsoft actively promoting and discussing on-device AI wor
These are some of my LLM assisted contributions from the past month. Nothing amazing, but I'm slowly getting better at it. Atm, using Qwen3.6 27B exclusively. For hardware - switching between M2 Ultra
Highlighting the new WebGPU backend in llama.cpp/ggml The work to bring full-fledged WebGPU support in llama.cpp started about an year and a half ago. It has been lead by @reeselevine and team at USCS
We have an arxiv paper up describing the work in more detail here: https://arxiv.org/abs/2605.20706. Also want to call out that there is even more room for improvement, some recent updates to wllama b
What hardware actually powers open-source AI? Not benchmarks. Not vendor marketing. Real-world community usage. We’re launching @huggingface Hardware: → trending GPUs & CPUs → VRAM distribution → infe
Llama-eval is a proposed evaluation tool for llama.cpp designed to standardize and improve comparability of community-run evaluations. The tool aims to address inconsistencies in how different users b
@BereznevKi20669 @ggerganov Yes I believe the real llama.cpp revolution is yet to happen at its full scale. As computers will have more RAM and models will improve, and *if* China will continue shippi
This post documents a technical demonstration of running the large Qwen3.5-397B-A17B model across distributed hardware using llama.cpp with advanced networking protocols. The approach leverages 4-bit
llama.cpp now supports various small OCR models that can run on low-end devices. These models are small enough to run on GPU with 4GB VRAM, and some of them can even run on CPU with decent performance
sam3.cpp - Meta's SAM 3 in pure C++ with @ggerganov's ggml - Supports SAM 3.1, 3, 2.1, 2 and EdgeTAM - FP16, 4-bit quant (EdgeTAM in 15 MB) - Apple Metal GPU, CUDA, CPU - Text-prompted: 'peach' → ever