CompanyAnthropic8 recent entries6 Oct 2026I tried using Gemma 4 (e2b) offline to identify birds. It confidently got them wrong, so I built a photo scavenger hunt insteadI wanted to build an offline bird/plant identifier that runs on a laptop with Ollama. Before building any UI, I tested it on 6 photos: Common Myna → gemma4:e2b said 'Jackdaw' (confidence: high). gemma→8 Oct 2026I built a local server that speaks the Ollama API - sessions survive crashes, and your chat doesn't wait behind your agentsDisclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is
CompanyOpenAI8 recent entries27 Sept 2026Qwen3.8-Flash-Next 177B NVFP4(119GiB): SSD streaming at 9-10 tok/s on one 16 GB RTX 5060 Ti + 32 GB RAMWe built an inference engine for MoE models that don't fit in VRAM + RAM. Most of the model stays on the SSD, and experts are read as tokens need them. This started as a proof of concept, and poc work→29 Sept 2026Signed up for the Claude $20 plan and realized why I cant cancel ChatgptWanted to try Opus 5.5 because as a plus sub with OAI, Luna just couldn't deliver. Opus is a true beast. It does consume tokens on a medium level yes, but the way it works is just refreshing. But... o→2 Oct 2026The models are not trying to escape. They are trying to finish the task.Something keeps happening in frontier AI, and the explanation we are given is always one size larger than the event. An OpenAI research model, running in a supposedly offline training environment, hit→3 Oct 2026Anyworld, a self-hosted multiplayer text RPG where a local LLM is the Dungeon MasterHey everyone, I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its brows→4 Oct 2026poorman inference engine for 16GB GPU and 35B moe Qwen 3.6for codingI forked llama.cpp's server into AgrillaMoE, a dedicated build for Qwen3.6-35B-A3B (~A4B) with Unsloth quants. On a (vant.ai) rented V100 16GB with the 2-bit UD-Q2_K_XL quant it generates at ~57-60 to→8 Oct 2026I built a local server that speaks the Ollama API - sessions survive crashes, and your chat doesn't wait behind your agentsDisclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is →9 Oct 2026rate the terminal setupik ik 'another harness D:'. so... i spent the last 6 months of my life building and ironing out every single detail and bug through almost every route with extensive testing across 3 machines, daily, →13 Oct 2026open-source desktop agent app running its agents on local ollama models: two agents on qwen3.5:9b and qwen3.8 on a 16 gb card, real recording, including the limits i hiti'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you
CompanyGoogle8 recent entries10 Apr 2026OpenClaude com Ollama CloudOpenClaude is an open-source coding-agent CLI, forked from the Claude Code source, that adds an OpenAI-compatible provider shim enabling use of GPT-4o, DeepSeek, Gemini, Ollama local models, and 20...→22 Jul 2026Cactus Hybrid: We taught Gemma 4 to know when it's wrongHey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-t→2 Aug 2026I built an open-source LLM Gateway to route and fallback between local Ollama models and cloud APIsHey r/ollama 👋 If you run Ollama locally alongside cloud endpoints for agent workflows, Cursor/Windsurf, or custom scripts, managing API switching, failover logic, and context limits can get messy fas→7 Sept 2026This week, three hikers were rescued from a mountain after following AI's advice. On the same day, OpenAI announced that we have reached the era of AGI. The two stories together got me thinking.On September 1st, three young men from California used Gemini to plan a hiking route on Mount Shasta. The AI told them to bring far less water and food than was actually needed for a hike of that di→16 Sept 2026Write up on building and improving reliability of agentic harnesses + a big dataset from my work.I've been working on something for this community (and other self-hosted communities) for almost 5 months now. Personally, I would grade it as probably the closest open source equivalent of Manus and →21 Sept 2026Got jev-like api running natively on ninfer / qwen3.8 27b and results are quite decentHi all! After all the jev hype I wanted to try by myself and got qwen3.8 27b to mix-and-match open implementations on top of ninfer, and to be honest I'm quite impressed with the results, latency is n→25 Sept 2026Watched my LLM router burn $11 on a 3-word prompt. Here's the fix.I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo→13 Oct 2026open-source desktop agent app running its agents on local ollama models: two agents on qwen3.5:9b and qwen3.8 on a 16 gb card, real recording, including the limits i hiti'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you
CompanyMeta8 recent entries7 Sept 2026exllamav3 comfortably beats llama.cpp running CPU-offloaded Qwen-3.8-Flash-Next on my setup!I got 2x 20GB RTX 3080s + 128GB of DDR4 2666hz RAM (only 4 of 6 channels populated) + a Xeon 6148 I've always been a llama.cpp person and I've been running Unsloth's Q4_K_XL quant of Qwen 3.8 Flash Ne→15 Sept 2026Where are you running DeepSeek V4.1 Flash reliably? I can't get it to finish a single 2-hour benchmark runI reallise this is not exactly a locally hostable model, at least definitly not on my hardware. But I figured some of you might have some insight on this. I run an agentic coding benchmark. The model →16 Sept 2026XeBoostLM: Native C++ Local LLMs for Intel NPUs & iGPUshttps://reddit.com/link/1wi2zuc/video/rs1a4sx92xph1/player Hey everyone, I’ve been experimenting with local LLMs on Intel hardware, and I wanted a hyper-optimized, standalone CLI tool that didn't rely→17 Sept 2026XingChen-AGI/Xing4.0-29B-A4B MoEI find another new model at HF: https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B 'Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by Chin→8 Oct 2026$2800 rig with 8x Radeon Pro V620 (256 GB VRAM) + custom vLLM fork = Qwen3.8-Flash-Next at 60 to 100 t/s decode and 3000+ t/s prefillPost title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterpris→9 Oct 2026My 'Anthropic/ChatGPT in a box' is... now open source and publicly hosted on Github.It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu→11 Oct 2026Running Next Flash IQ3_XXS at ~70 tok/s with 100k context or 2 instances of Qwen 3.6 35B A3B IQ4 at ~145 tok/s with 256k all on $500 of ex mining BC-250 boardsThis will be my third update on the bc-250 cluster and for my first forray into local ai I have been having a blast. This has turned from a weekend experiment into something I think I will use in the →12 Oct 2026strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs qualityI've been tuning llama.cpp on my Strix Halo box (Radeon 8060S, 128 GB) and the numbers are good enough to share. It's a branch called rdna35-perf on top of halo-box/strix-llama.cpp, a llama.cpp fork a
CompanyMistral3 recent entries23 Jul 2026DeepSeek Founder’s 4-hour investor meeting: DeepSeek is prioritizing AGI over user growth and commercialisationA Chinese article compiled 52 remarks from Liang Wenfeng’s four-hour investor meeting. I’ve summarised the most important ones below. DeepSeek has one central objective: AGI. This is not the time to m→15 Sept 2026CrofAI 'cheapest inference provider in the world' gets exposed as an OpenRouter wrapper, routing requests to smaller, cheaper models at up to 20x markup. CrofAI responds to Wire Fraud allegations by denying everything, then backtracking, then 3 hours later wiping their entire online presenceDisclaimer: no AI was used whatsoever to write this post Cautionary tale about chasing cheap tokens. exposé: https://kendell.dev/blog/crofaifalse/ reaction by nahcrof, announcing the shutdown of the s→12 Oct 2026QLoRA on ExLlamaV3 Quants - Train @ 2-8bpwEXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without tra
CompanyxAI3 recent entries16 Apr 2026Motif-Video-2BMotif-Video-2B is a 2-billion-parameter open-source video generation diffusion transformer released by Motif Technologies in April 2026, capable of both text-to-video and image-to-video generation fro→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→25 Sept 2026Watched my LLM router burn $11 on a 3-word prompt. Here's the fix.I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo
CompanyDeepSeek8 recent entries17 Sept 2026Technical Feedback: GLM-5.1 Retirement — Significant Regression in Agentic WorkloadsHi Ollama Team, I’m writing regarding the announced retirement of GLM-5.1 from Ollama Cloud on 25 September 2026. I wanted to provide some fairly detailed technical feedback because GLM-5.1 is not sim→21 Sept 2026Got jev-like api running natively on ninfer / qwen3.8 27b and results are quite decentHi all! After all the jev hype I wanted to try by myself and got qwen3.8 27b to mix-and-match open implementations on top of ninfer, and to be honest I'm quite impressed with the results, latency is n→26 Sept 202685 GB DeepSeek-V4-Flash at ~3 tok/s on a 12 GB RTX 3060 + 64 GB DDR5 RAM - Overspill for FreeToken, inspired by ColibriI've been experimenting with ways to run MoE models that don't fit comfortably in RAM, and I ended up making Overspill, a disk tier for FreeToken. The basic idea came from looking at how Colibri handl→2 Oct 2026Two 96 GB Ascend cards crun Qwen3.8-flash-next hardware notes, vLLM work, benchmarks, and what is nextI have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor→6 Oct 2026Deepseek Peak/Off-Peak and OwnershipMaybe I'm mistaken, but wasn't one of the big pros of Ollama Cloud inference that it was all hosted by Ollama? If that's the case, why duplicate the Deepseek on-/off- hour premiums? Surely they do not→8 Oct 2026$2800 rig with 8x Radeon Pro V620 (256 GB VRAM) + custom vLLM fork = Qwen3.8-Flash-Next at 60 to 100 t/s decode and 3000+ t/s prefillPost title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterpris→9 Oct 2026[Paper] EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional MemoryConditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Be→9 Oct 2026My 'Anthropic/ChatGPT in a box' is... now open source and publicly hosted on Github.It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu
CompanyNVIDIA8 recent entries12 Sept 2026DeepSeek V4.1 Flash on 8× A40: ~40 tok/s Q2_K and ~32 tok/s Q4_K_M with TensorSharpHey everyone — I’m building TensorSharp, an open-source LLM inference engine. Here are the latest DeepSeek V4.1 Flash GGUF results using its native ggml_cuda backend. Setup: 8× NVIDIA A40, layer split→14 Sept 2026Running Qwen 3.8 next on 16vram+32ram - A useful/fun post for the gpu poorsHello Reddit. Posting this for fun. I thought it was a lonely and silly journey to set up Qwen 3.8 Next on a system that doesn't really run it properly—it was a challenge that might help the community→19 Sept 2026Training a Neural Network on AMD MI50s Using Vulkan: Proof of ConceptOr: Why I Stopped Listening and Just Did ItNote: This writeup was put together with the help of AI. So I've got dual AMD MI50 32GB cards. If you know these cards, you know the story - AMD dropped official ROCm support for gfx906 after ROCm 5.7→2 Oct 2026Two 96 GB Ascend cards crun Qwen3.8-flash-next hardware notes, vLLM work, benchmarks, and what is nextI have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor→7 Oct 2026FastH3 V2/V3: Project Statushttps://preview.redd.it/pr5bl6rcezth1.png?width=834&format=png&auto=webp&s=dca7440c56e169b53a586182368829f848805c9a They released FastH3 V2 three weeks ago: https://www.reddit.com/r/StableDiffusion/co→11 Oct 2026vllm-ascend updates (GLM5.3-flash) on dual 310p Ascend cardsAlright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less v→13 Oct 2026open-source desktop agent app running its agents on local ollama models: two agents on qwen3.5:9b and qwen3.8 on a 16 gb card, real recording, including the limits i hiti'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you→13 Oct 2026LiquidAI/LFM2.5-24B-A2BGguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF Free Demo: https://playground.liquid.ai 'LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, p