CompanyAnthropic8 recent entries17 Aug 2026Qwen 3.8 27B Overthinking, It has to be done, it has to be overthinking to punch Opus 4.6Yes, it sucks to waste time waiting on 16K+ reasoning tokens alone. But here's the thing, this is only a 27B model trying to perform on par with 1T+ parameter models. Something has to be sacrificed, a→26 Aug 2026Underrated Muse GlimmerBenchmarked qwen3.8 xhigh, medium and muse glimmer. Xhigh effort mode with qwen3.8 took almost 30hrs. (And still failed on 16 cases because of the 32K output token limit) Medium effort mode and muse g→
CompanyOpenAI8 recent entries17 Aug 2026Qwen 3.8 27B Overthinking, It has to be done, it has to be overthinking to punch Opus 4.6Yes, it sucks to waste time waiting on 16K+ reasoning tokens alone. But here's the thing, this is only a 27B model trying to perform on par with 1T+ parameter models. Something has to be sacrificed, a→28 Aug 2026Self-hosted agent + BI platform that runs fully local, Ollama or vLLM for both chat and embeddingsBeen building AgentSwarms for a while. Agent platform with RAG, multi-agent workflows and a BI layer over your own data. Posting here because the whole thing runs without a single hosted API if you wa→25 Sept 2026Running an AI SaaS against local Ollama models — what actually breaks when you swap the providerI maintain a starter kit where the LLM provider is a runtime-swappable DB row, not a hardcoded constant — OpenAI, Anthropic, or a local Ollama instance. Mostly so I can demo without burning API credit→1 Oct 2026Built an SQLite-based memory system for Ollama that doesn't eat all your VRAMWanted to share a project I've been working on to give local Ollama models persistent document memory without needing heavy vector databases. It's called Hillock. The main problem I had with standard →3 Oct 2026Replacing vector databases with SQLite and SIMD hypervectors in under 1.2GB VRAM (Hillock)Disclosure: I am the creator of this project. After days of lurking and building up enough karma, I can finally post here. Every time I tried running local RAG on my own machine, I hit the exact same →7 Oct 2026Built an SQLite memory engine for Ollama that does not eat all your VRAMWanted to share a local memory engine I built called Hillock. The main problem I had with standard local RAG was that running document parsing and vector search chewed up so much VRAM that my actual O→8 Oct 2026I built a local server that speaks the Ollama API - sessions survive crashes, and your chat doesn't wait behind your agentsDisclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is →12 Oct 2026A 'free' multi-modal 'embedding' model (already) included with your LLM: vLLM plugin + upstream PRDisclaimer: I work for NuMind, the company behind this plugin. TLDR: Text/image (simili-)embeddings from the LLM you already have loaded :-) I've been working on making it easier to obtain embeddings
CompanyGoogle5 recent entries23 Jul 2026I built an open-source RAG chatbot starter that runs fully locally with Ollama (FastAPI + ChromaDB)I kept re-wiring the same RAG plumbing on every project, so I turned it into a clean starter and open-sourced it. Upload a PDF, ask questions, and get answers with page-level source citations. It runs→23 Sept 2026Which small model works best with Ollama on a Raspberry Pi 5?I've been comparing and working on Gemma 4, Phi-4-Mini, and Qwen3-4B for local LLM use on an 8 GB Raspberry Pi 5. The results were indeed close, but each model has a different strength: Gemma 4 E2B: a→1 Oct 2026We benchmarked 18 RAG pipelines against an agent loop on Google's FRAMES. The best pipeline hit 78.9%. The agent loop hit 92.7%.Hybrid search, reranking, query decomposition, and query expansion are often treated as must-haves for good RAG. We wanted to see how much each actually helped, so we tested them. Same model, same emb→6 Oct 2026Introducing EmbeddingGemma 2: A best-in-class open model for natively multimodal embeddings | Googlehttps://huggingface.co/google/embeddinggemma-2 EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combina→6 Oct 2026google/embeddinggemma-2 · Hugging FaceEmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combinations thereof—into a single, unified 768-dimens
CompanyMeta8 recent entries9 Aug 2026Best Embedding + Reranking ModelWhat Local Embedding + Reranking Models are you guys running for RAG? I went down this rabbit hole because I wanted a Embedding Model + Reranker for a Translation Memory Server. Essentially, given X p→14 Aug 2026Built a Mac app that actually uses Ollama for work, not just chat — with a calibration step for models with flaky tool callingMost Ollama frontends I've tried are chat UIs with tool calling bolted on as an afterthought, which is fine until you actually try to make a small local model do multi-step work and it silently emits →17 Aug 2026Qwen 3.8 27B Overthinking, It has to be done, it has to be overthinking to punch Opus 4.6Yes, it sucks to waste time waiting on 16K+ reasoning tokens alone. But here's the thing, this is only a 27B model trying to perform on par with 1T+ parameter models. Something has to be sacrificed, a→21 Aug 2026Llama.cpp DSpark PC Tree Fork (up to 3%-29.5% faster!)Hello gang, I made an implementation of DSpark PC Tree (Parent conditioned drafting tree). This is an implementation of this research paper: https://arxiv.org/abs/2608.02123 Unaffiliated, just found i→23 Aug 2026Trying to optimize LLM inference latency — Ollama + OpenWebUI/AnythingLLMHi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo→26 Aug 2026little tool for offline wikipedia RAGI was bored and handwrote a tiny 100-line bash script to let an agent search for and read articles from an offline wikipedia archive during a regular chat. It's not particularly useful, but it's defin→25 Sept 2026Running an AI SaaS against local Ollama models — what actually breaks when you swap the providerI maintain a starter kit where the LLM provider is a runtime-swappable DB row, not a hardcoded constant — OpenAI, Anthropic, or a local Ollama instance. Mostly so I can demo without burning API credit→1 Oct 2026Built an SQLite-based memory system for Ollama that doesn't eat all your VRAMWanted to share a project I've been working on to give local Ollama models persistent document memory without needing heavy vector databases. It's called Hillock. The main problem I had with standard
CompanyMistral2 recent entries6 Aug 2026nvidia/NVIDIA-Nemotron-Parse-2.0 · Hugging FaceNVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blu→26 Aug 2026How to Fine-Tune an LLM: An End-to-End GuideI ended up fine tuning a mistral 7b to outperform our costly foundational model and saved $300k. I previously thought that fine tuning was pointless (it's definitely not) and that all these problems c
CompanyxAI1 recent entries14 Oct 2026Discover AI: A Learning Guide - 51 hands-on labs for learning how AI actually works.This is an offline app with 51 hands-on labs for learning how AI actually works, from neurons to agents and more... **A bit of insight** - 51 labs in 6 groups, from the basics (what a neuron is, gradi
CompanyDeepSeek4 recent entries1 Aug 2026A collection of small domain-specific benchmarks for local models (30+ and growing)Hello fellow local AI people! I took 'you must create your own benchmarks' literally, and built a website for this. How does the end result look like Let's say I want to know which model has most comm→15 Aug 2026Agentic harness for small modelsHello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty s→30 Sept 2026peculiar-ragdoll's Dirk-Qwen 3.8-27B vs. UkisAI Swift-1.5 Qwen3.8-27BI have a long list of my own domain specific eval questions that I run to validate which models I can rely on: coding, coding (numpy/pandas), data analytics decision making, local RAG, and voice assis→1 Oct 2026We benchmarked 18 RAG pipelines against an agent loop on Google's FRAMES. The best pipeline hit 78.9%. The agent loop hit 92.7%.Hybrid search, reranking, query decomposition, and query expansion are often treated as must-haves for good RAG. We wanted to see how much each actually helped, so we tested them. Same model, same emb
CompanyNVIDIA7 recent entries29 Apr 2026What are people using for low-latency autocomplete in production? [P]Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr→6 Jun 2026Built a fully-local paper-RAG across 2× 1080 Ti + a 3090. Three Ollama gotchas that each cost me a day.A developer documented their experience building a fully-local paper Retrieval-Augmented Generation (RAG) system using two NVIDIA GTX 1080 Ti GPUs and one RTX 3090, sharing three significant challenge→6 Aug 2026nvidia/NVIDIA-Nemotron-Parse-2.0 · Hugging FaceNVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blu→11 Aug 2026Introducing Unsloth Desktop appHi LocalLlama, we're super excited to release Unsloth Desktop today! 🦥 It's the first desktop app that enables you to run and train models locally. Open-source. Available on Mac, Windows, and Linux Su→23 Aug 2026Trying to optimize LLM inference latency — Ollama + OpenWebUI/AnythingLLMHi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo→28 Aug 2026Computer Hardware to run on premisesWe are considering to buy computer, server to run decent model on premises. I would like to run big open source model with more that 70b parameters roughly. I read the people used to run on Apple Stud→8 Oct 2026New challenge: consolidated 2 cards in a single system but one card hogs all the models even if I run 2 different instances of Ollama on 2 different portsIn my attempt to combat powerty; as electricity cost a ton these days and it is hot as hell, I consolidated 2 machines into one, so my 2 crappy machines now are one, and that means a bit less power an