CompanyAnthropic8 recent entries5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→8 Sept 2026How many agents can 2×4090 actually run at once? Three weeks of llama.cpp concurrency data — soft cap 5 @ 64k, hard cap 9, and why.I'm the CTO of a mid-size nonprofit, and I run a local Qwen stack as a coding subagent for that work — partly on principle (a fair amount of what I handle shouldn't leave the building), partly because
CompanyOpenAI8 recent entries27 Jul 2026Give any Ollama-compatible client session memory + a shared knowledge wiki by swapping the chat URLHey folks — I built ContextMemory, an open-source agentic context gateway for apps that already talk to LLMs. The idea is simple: keep your existing POST /api/chat client (Ollama wire format), point i→5 Aug 2026Qwen Developers' responses from their recent Twitter/X AMAQuestions & Responses(in BOLD) below. Favorite question(s) moved to end of the thread with combined responses(removed duplicates). Be optimistic folks. I'm sure we're getting other models too apart fr→22 Aug 2026Single RTX 5090: Qwen3.8-27B NVFP4 at a real 262K context in vLLM — 77 tok/s short-context, 64.7 tok/s at 128KThis is the Qwen3.8-27B setup I actually use every day on one RTX 5090. I wanted to write it down with enough detail that another 5090 owner can reproduce it instead of guessing which memory knobs I u→11 Sept 2026Thinking that we’ll get safety by CoT traces is wishful thinking. Safety lives in the harness, not the chain of thoughtAstra's launch has produced a strange discourse. The reporting that broke the story framed the model's use of recurrent depth primarily as a safety regression, because it means the model reveals less →21 Sept 2026The Digital MindI've been thinking a lot about the idea of how a human mind is similar to a digital neural network. Seeing videos of the fruit fly brain scan, and the crazy things people have done with it probably co→22 Sept 2026I asked 4 LLMs to invent words for things only an LLM could understand. Got a dictionary of their own bad habits instead.You know the thing where you ask a chatbot what's wrong with your plan and it goes 'there are three problems here' and problem three is clearly the runt of the litter? There's a word for that now. An →1 Oct 2026Qwen3.8-27B Q4_K_M on one RTX 3090 + OpenCode: throughput, four coding tasks, and a reasoning-budget failureI put an old RTX 3090 to work as a local coding agent with Qwen3.8-27B, llama.cpp, and OpenCode. Here are the setup and results, including what failed. This is a summary of my own blog post, linked be→4 Oct 2026poorman inference engine for 16GB GPU and 35B moe Qwen 3.6for codingI forked llama.cpp's server into AgrillaMoE, a dedicated build for Qwen3.6-35B-A3B (~A4B) with Unsloth quants. On a (vant.ai) rented V100 16GB with the 2-bit UD-Q2_K_XL quant it generates at ~57-60 to
CompanyGoogle7 recent entries10 Apr 2026AI pricing tiers in a nutshellThe specific Reddit post (r/ChatGPT/comments/1shzwii) was not directly retrievable, but based on the closely related search results about AI pricing tiers discussed in that community, here is a fac...→23 Jul 2026I built an open-source RAG chatbot starter that runs fully locally with Ollama (FastAPI + ChromaDB)I kept re-wiring the same RAG plumbing on every project, so I turned it into a clean starter and open-sourced it. Upload a PDF, ask questions, and get answers with page-level source citations. It runs→24 Jul 2026I built an open-source multi-agent SDLC harness that beats a cold Claude Code run on large repos, by learning the repo once. Real benchmarks (incl. where it loses) inside. [P]Built an open-source AI coding agent that was 7%–75% cheaper than a cold 'claude -p' run on 6/6 well-localized tasks across repositories up to ~82k LOC. The biggest difference: Cold agent: 6.83, 207 t→1 Oct 2026We benchmarked 18 RAG pipelines against an agent loop on Google's FRAMES. The best pipeline hit 78.9%. The agent loop hit 92.7%.Hybrid search, reranking, query decomposition, and query expansion are often treated as must-haves for good RAG. We wanted to see how much each actually helped, so we tested them. Same model, same emb→6 Oct 2026Introducing EmbeddingGemma 2: A best-in-class open model for natively multimodal embeddings | Googlehttps://huggingface.co/google/embeddinggemma-2 EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combina→6 Oct 2026google/embeddinggemma-2 · Hugging FaceEmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combinations thereof—into a single, unified 768-dimens→13 Oct 2026Where do LLMs actually earn their place in a recommender system? Kim Falk’s new book + 5 ebook giveaway [D]Hi r/MachineLearning, Stjepan from Manning here. The mods approved this post, and I appreciate being welcomed back to share another book with you. We’ve just launched AI Recommender Systems: Transform
CompanyMeta8 recent entries12 Aug 2026I ran DeepSeek V4 Flash 284B + DSpark on one RTX PRO 6000. The drafter was faster in RAM than VRAM.Hey guys, Just finished benchmarking DeepSeek V4 Flash 284B + DSpark on a single RTX PRO 6000 96GB. Short version: DSpark: ~15–17% faster generation on my coding workload On this setup, the DSpark dra→19 Aug 2026Thoughts About Scaling Law - Z.aiThoughts About Scaling Law Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn't a question that can be answered on its own. Parameter→19 Aug 2026Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant.I have been searching for suitable model to run on my 8GB RAM toy, NVIDIA Orin Nano Super 8GB. This little toy was priced at 249 earlier this year (not any more), and pulls very little power when idle→31 Aug 2026The Chrono Trigger plot challenge - Crono awakens in his modest bedroom of 2095...A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I al→15 Sept 2026I trained a 44M parameter quantized LLM from scratch on 45B tokens. It ships in 19.8 MB and runs at ~1,900 tok/s on CPU. [P]Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you. That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from →1 Oct 2026Qwen3.8-27B Q4_K_M on one RTX 3090 + OpenCode: throughput, four coding tasks, and a reasoning-budget failureI put an old RTX 3090 to work as a local coding agent with Qwen3.8-27B, llama.cpp, and OpenCode. Here are the setup and results, including what failed. This is a summary of my own blog post, linked be→2 Oct 2026Adding memory to search instead of sampling in reward maximization tasks [R]I am one of the authors of FLEET - an algorithm that enhances Best-of-N generation by attributing external rewards to particular tokens and then uses MCTS to adjust logits during the next run. I find →12 Oct 2026U-Space: Uncovering When and Why Uncertainty Arises in Language Models [R][D]If you build on open-weight reasoning models, you have probably wanted a 'how sure is the model about this answer?' signal that does not cost 5x sampling, a judge model, or a labeled dataset. Hence, w
CompanyMistral2 recent entries6 Aug 2026nvidia/NVIDIA-Nemotron-Parse-2.0 · Hugging FaceNVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blu→31 Aug 2026The Chrono Trigger plot challenge - Crono awakens in his modest bedroom of 2095...A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I al
CompanyxAI7 recent entries10 Apr 2026Lighter joke??I was unable to retrieve the specific Reddit post at the provided URL (reddit.com/r/ChatGPT/comments/1sh9hma/lighter_joke/), as it did not appear in the search results. The search only returned gen...→10 Apr 2026Grok for youThe specific Reddit post (r/ChatGPT, post ID `1shi3fn`, titled 'Grok for you') was not directly retrievable or indexed in search results. Based on the available context from surrounding community d...→10 Apr 2026Got early access to a real-time interactive video model, here's what I foundI was unable to retrieve the specific Reddit post at the provided URL through my search. The post (reddit.com/r/StableDiffusion/comments/1shxmfk) did not surface in the search results, and I cannot...→10 Apr 2026Err... what's going on?I was unable to retrieve the specific Reddit post from the URL provided, and my search did not return relevant results for that thread. Reddit content is often not indexed in real time, and I canno...→10 Apr 2026AI pricing tiers in a nutshellThe specific Reddit post (r/ChatGPT/comments/1shzwii) was not directly retrievable, but based on the closely related search results about AI pricing tiers discussed in that community, here is a fac...→24 Jul 2026I built an open-source multi-agent SDLC harness that beats a cold Claude Code run on large repos, by learning the repo once. Real benchmarks (incl. where it loses) inside. [P]Built an open-source AI coding agent that was 7%–75% cheaper than a cold 'claude -p' run on 6/6 well-localized tasks across repositories up to ~82k LOC. The biggest difference: Cold agent: 6.83, 207 t→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa
CompanyDeepSeek8 recent entries12 Aug 2026I ran DeepSeek V4 Flash 284B + DSpark on one RTX PRO 6000. The drafter was faster in RAM than VRAM.Hey guys, Just finished benchmarking DeepSeek V4 Flash 284B + DSpark on a single RTX PRO 6000 96GB. Short version: DSpark: ~15–17% faster generation on my coding workload On this setup, the DSpark dra→15 Aug 2026club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27BQuick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but a→22 Aug 2026Single RTX 5090: Qwen3.8-27B NVFP4 at a real 262K context in vLLM — 77 tok/s short-context, 64.7 tok/s at 128KThis is the Qwen3.8-27B setup I actually use every day on one RTX 5090. I wanted to write it down with enough detail that another 5090 owner can reproduce it instead of guessing which memory knobs I u→31 Aug 2026The Chrono Trigger plot challenge - Crono awakens in his modest bedroom of 2095...A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I al→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→1 Oct 2026We benchmarked 18 RAG pipelines against an agent loop on Google's FRAMES. The best pipeline hit 78.9%. The agent loop hit 92.7%.Hybrid search, reranking, query decomposition, and query expansion are often treated as must-haves for good RAG. We wanted to see how much each actually helped, so we tested them. Same model, same emb→2 Oct 2026Two 96 GB Ascend cards crun Qwen3.8-flash-next hardware notes, vLLM work, benchmarks, and what is nextI have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor→9 Oct 2026Just another purely open-weight models benchmarkLive results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area
CompanyNVIDIA8 recent entries22 Aug 2026Single RTX 5090: Qwen3.8-27B NVFP4 at a real 262K context in vLLM — 77 tok/s short-context, 64.7 tok/s at 128KThis is the Qwen3.8-27B setup I actually use every day on one RTX 5090. I wanted to write it down with enough detail that another 5090 owner can reproduce it instead of guessing which memory knobs I u→28 Aug 2026Qwen3.8-27B on a 24GB RTX PRO 4000 Blackwell: 128K real context, 785 tok/s prefill, 67 tok/s MTP3 decodeI wanted to see whether NInfer could run Qwen3.8-27B on an RTX PRO 4000 Blackwell 24GB. Hardware/software: NVIDIA RTX PRO 4000 Blackwell, 24GB 145W power limit CUDA 13.3 NVIDIA driver 610.x Ubuntu LXC→31 Aug 2026Qwen3.8-Flash-Next in llama.cpp from CPU-only to 96GB VRAM: 8.5 to 109 tok/s, max context and parameters test. My findings on RTX 6000 PRO.Hey guys, I tested Qwen3.8 Flash with llama.cpp from CPU-only to the full 96GB of my RTX PRO 6000. Short version: CPU-only reached 8.34 tok/s at a 2K prompt Full 96GB reached 109.07 tok/s At 245K cont→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→2 Oct 2026Two 96 GB Ascend cards crun Qwen3.8-flash-next hardware notes, vLLM work, benchmarks, and what is nextI have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor→9 Oct 2026Just another purely open-weight models benchmarkLive results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area→15 Oct 2026nvidia/Nemotron-3.5-Embed-1B-BF16 · Hugging FaceNemotron-3.5-Embed-1B-BF16 is a versatile embedding model developed by NVIDIA for multimodal and multilingual question-answering retrieval. The model can embed a document page in the form of an image,→17 Oct 2026model : add Kolibri-1 (Aleph-Alpha) support by riteme · Pull Request #30673 · ggml-org/llama.cppEuropean 78B A3.4B model is now supported by llama.cpp, I am waiting for some GGUFs https://huggingface.co/Aleph-Alpha/Kolibri-1 Kolibri is Aleph Alpha's mixture-of-experts (MoE) reasoning model, with