CompanyAnthropic8 recent entries2 Oct 2026The models are not trying to escape. They are trying to finish the task.Something keeps happening in frontier AI, and the explanation we are given is always one size larger than the event. An OpenAI research model, running in a supposedly offline training environment, hit→2 Oct 2026Inference Engineering for DummiesHi all! I am a former SWE who has recently transitioned into inference engineering. I launched a side hustle a few months back and I've just taken it full time due to excessive demand. Its been such a
CompanyOpenAI8 recent entries2 Oct 2026Inference Engineering for DummiesHi all! I am a former SWE who has recently transitioned into inference engineering. I launched a side hustle a few months back and I've just taken it full time due to excessive demand. Its been such a→3 Oct 2026Replacing vector databases with SQLite and SIMD hypervectors in under 1.2GB VRAM (Hillock)Disclosure: I am the creator of this project. After days of lurking and building up enough karma, I can finally post here. Every time I tried running local RAG on my own machine, I hit the exact same →5 Oct 2026When Redditors come in here and ask why we run LLMs, this is why: Big AI is watching.Anthropic Reports Florida Woman's Claude 'Diary' Threat to Law Enforcement And this time it wasn't the AI model that made the LEO referral. It was the 'human review team'. The frontier AI companies ar→5 Oct 2026PewDiePie getting banned twice by OpenAI while making a local model is top-tier comedy 💀So PewDiePie decides to fine-tune a local AI model called Ajax on his own computer. Pretty normal stuff for local model fans. To make his dataset, he uses OpenAI's API. OpenAI catches him using their →7 Oct 2026Looking for 10–20 people for a coordinated LLM testThursday, October 8 — 15:00–19:00 UTC Access is free and the API is OpenAI-compatible. During those 4 hours, everyone will be asked to use it actively at the same time—coding, chatting, agents, long c→8 Oct 2026Involution Without Export Is Wasted EffortA week in Beijing and Shanghai with the people building AI in China some quotes from the article: ByteDance/Doubao is the only major lab keeping its flagship closed, because it is the only one that al→8 Oct 20262 months (meme-d) research, Do People Notice the Difference Between AI Models?Preamble and disclaimer, Sample size: 8, at any size of form this research is just screwing around being writing it down. This was just a little,(well... big), curiosity test I wanted to run to see wh→12 Oct 2026A 'free' multi-modal 'embedding' model (already) included with your LLM: vLLM plugin + upstream PRDisclaimer: I work for NuMind, the company behind this plugin. TLDR: Text/image (simili-)embeddings from the LLM you already have loaded :-) I've been working on making it easier to obtain embeddings
CompanyGoogle8 recent entries26 Sept 2026How accessible is local AI actually, and what happens if affordable access to frontier models doesn’t last?Sometimes it’s easy to forget that this sub and others like it are probably the extreme minority when it comes to this hobby. Most people, I would think, don’t use or can’t afford one good GPU, let al→27 Sept 2026Qwen, where's the small stuff? (1B/2B/4B)I know Qwen is a key player in the local LLM space and has consistently introduced truly impactful technologies—like n-gram in Qwen-Next and the recent Qwen 3.8 27B, which is an amazing local model. H→28 Sept 2026Sonnet 5.5 is by far the best free model available right nowI know most people here probably don't care as much about sonnet 5.5 as they did opus 5.5, because their 100 and 200 plans already give them plenty of weekly usage for opus. But keep in mind: free use→2 Oct 2026Inference Engineering for DummiesHi all! I am a former SWE who has recently transitioned into inference engineering. I launched a side hustle a few months back and I've just taken it full time due to excessive demand. Its been such a→3 Oct 2026ai slop game based on batlle video in real time with only open modelCheck out Chimera Arena (https://chimeraarena.com/static/showcase/index.html?lang=en)! Here’s my slightly scrappy AI-generated teaser for Chimera Arena, my card game combining AI-generated video battl→4 Oct 2026Poor People Vulkan GPUs listHelp with this list. Give me your recommendation on 'not supported anymore' GPUs. Looking for budget and Vulkan friendly options. Most of the GPU are not supported by latest CUDA / ROCm. Often with so→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→12 Oct 2026What are you doing for a reliable, well designed local model based AI search?Ive been using OpenWebUI's inbuilt web search and I can't put my finger on it, but it's not quite as polished, reliable, seamless as using Claude or Gemini. And its often slow beyond a limit that make
CompanyMeta8 recent entries28 Sept 202695+ TPS through 100K generated for qwen3.8 27b, 262K ctx, on a single 3090Hello everyone! A little while back I posted about LlamAmpere, a fork of Llama.cpp with Ampere-specific improvements (though it is caught up to main and will support other hardware, too). Thank you to→29 Sept 2026Reflection 70B was released two years ago (September 2024)You may think that jev, OpenClaw or TurboQuant are super cool, but actually the coolest LLM invention happened two years ago As we all know, the best source of reliable information about LLMs is YouTu→29 Sept 2026Inference Engines will become a series of one-offsninfer, dwarfstar, Splash, llamAmpere, gufo, etc. We've all seen them popping up, great tok/s, people loving them. Forks of llama.cpp or another engine, or made from scratch. For better or worse, the →30 Sept 2026Penalties of PCIe generations? (2x R9700)I just bought the GPUs after deliberating and debating for over two years. With costs not coming down any time soon and me just wanting to get this massive todo-box ticked, I decided to just YOLO it; →5 Oct 2026k_llama.cpp MoE Optimizations: Expert Residency, Hybrid CPU/GPU Execution, Q2_0 SupportFinally finished my fork: https://github.com/IceFog72/ik_llama.cpp Nothing else I wanted to add/try currently works In short, it now has: Bandwidth-adaptive CPU/GPU execution Shared expert-residency i→6 Oct 2026Overclocking DDR5 For Faster MoE Prefill and DecodeWith llama.cpp using a customized SYCL backend on Intel B70 (32GB), overclocking my DDR5 memory gave modest gains for MoE models which do not fit in VRAM. Both PP and TG increased after overclocking D→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→12 Oct 2026Running unsloth/Qwen3.8-27B-UD-IQ4_XS 262k q4_0 KV + vision on 16GB VRAM @ 80t/s and 1500ppI get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card. People seem to think I am lying or m
CompanyMistral8 recent entries23 Jul 2026Arcee AI has spoken out against the ban on open Chinese models in USThis is rather counterintuitive, since banning Chinese models would benefit them the most. Jensen Huang is also against the ban, although the interests here are more obvious. Do you think that if Arce→29 Jul 2026'Uncensored' LLMs are measurably more optimistic than their base modelsHi. Many people think uncensored models are basically the same model that just doesn't refuse, but... I was recently checking whether uncensored models would give me better answers for stock market pr→19 Aug 2026GLM-5.3 is out on AA, and I'm fed up with their Intelligence/cost plotI think AA's intelligence/cost plot is seriously misleading, so I decided to make my own. Their plot is in the second image. All points are at max thinking. All intelligence index scores are from AA. →26 Aug 2026How to Fine-Tune an LLM: An End-to-End GuideI ended up fine tuning a mistral 7b to outperform our costly foundational model and saved $300k. I previously thought that fine tuning was pointless (it's definitely not) and that all these problems c→31 Aug 2026The Chrono Trigger plot challenge - Crono awakens in his modest bedroom of 2095...A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I al→14 Sept 2026DeepSeek engineer relections on RSI - burying my talent to yesterdayNote - This is translated from the actual blog link right at the bottom. A few days ago, DeepSeek v4.1 was released. It raised the ability of small models to a new level. AI is improving much faster t→30 Sept 2026Follow-up: my native Rust + Vulkan Transformer training backend — 14 days later, now 14 parity-verified architectures and full PEFTFollow-up to my post from about two weeks ago. A lot has changed since then, so I wanted to post an update on where the backend is now. Where the green architectures stand When I posted last time, 7 a→2 Oct 2026[16GB AMD] Smartest local model that also does not refuse? 7950X + RX 6950 XTI want the smartest local model I can run that also does not refuse. Not a tiny 'uncensored' 8B. Not a censored model with a jailbreak prompt. Tried dolphin-llama3:8b with a custom SYSTEM prompt. It c
CompanyxAI8 recent entries10 Apr 2026Bad news on Happy Horse from twitterHappyHorse-1.0 is a pseudonymous AI video generation model that appeared on April 7, 2026, topping the Artificial Analysis Video Arena leaderboard in both text-to-video and image-to-video (no audio...→25 Jul 2026Old Coder Needs help with New AI Development and wants to get up to speed to understand it all.Hi Guys, I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing→2 Aug 2026Vacuum 16Thttps://huggingface.co/tsfrm/vacuum-16t A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that 'haha I have the biggest model out t→14 Sept 2026Animated transition from AA Intelligence Index v4.1 to v4.3I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison. All intelligence and price per task are sam→16 Sept 2026Can current MiniCPM5-2B/any best SOTA under 10B + modern harness beat pre-March 2025 frontier models like Grok 3/GPT-4o?Hey experts! I genuinely have this question and would love a general consensus from people actually using these models. LLMs have advanced a lot in benchmarks and in practical usage. Even GPT-4o and G→19 Sept 2026What is the best way to use LLMS's in 2026? I feel like a caveman with the way I use them?I'm still using LLM's the 2022/23 way of doing everything manually from the chat box. I copy and paste everything manually into the box and ask the AI to explain or do something, such as pasting code →25 Sept 2026Watched my LLM router burn $11 on a 3-word prompt. Here's the fix.I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo→29 Sept 2026Reflection 70B was released two years ago (September 2024)You may think that jev, OpenClaw or TurboQuant are super cool, but actually the coolest LLM invention happened two years ago As we all know, the best source of reliable information about LLMs is YouTu
CompanyDeepSeek8 recent entries30 Sept 2026Follow-up: my native Rust + Vulkan Transformer training backend — 14 days later, now 14 parity-verified architectures and full PEFTFollow-up to my post from about two weeks ago. A lot has changed since then, so I wanted to post an update on where the backend is now. Where the green architectures stand When I posted last time, 7 a→1 Oct 2026Update #2: Post training yandex/AliceAI-80B-A3B [instruct!] from scratchLast update: https://www.reddit.com/r/LocalLLaMA/comments/1wu9ksu/update_yandexaliceai_80ba3b_fine_tune_progress/ - basically, an instruct fine tune on the base model using a synthetic distilled data →1 Oct 2026Jeff-Qwen3.5-0.8B v1.2 + 9 LoRA adapters: put it in front of Qwen3.8-27B for 38× faster decisions and +8.7 points accuracy, for under 2 GB extra memoryA few days ago I released Jeff-Qwen3.5-0.8B, a small 'System 1' model that picks between options you define and returns a calibrated probability for each, in one forward pass. Speed was great on my M4→2 Oct 2026Inference Engineering for DummiesHi all! I am a former SWE who has recently transitioned into inference engineering. I launched a side hustle a few months back and I've just taken it full time due to excessive demand. Its been such a→4 Oct 2026Update #4: Post training yandex/AliceAI-80B-A3B [instruct!] from scratchLast update for those following: https://www.reddit.com/r/LocalLLaMA/comments/1wvyc3e/update_3_post_training_yandexaliceai80ba3b/ Project in a sentence: An instruct finetune of ALiceAI-80B-A3B-Base ca→4 Oct 2026For dual DGX spark users; GLM 5.3 flash got a 50%+ performance boostFor the last few months, I ran DeepSeek v4.0 flash (NVFP4). First 0731, then visionexp because it was a free improvement. I got around 65 tps decode and almost 2k prefill, and ran 4-5 agents in parall→6 Oct 2026Local AI World Model Part 2 - Deep NN to turn Images into Playable Characters, with prompt switching mid rolloutLast time when I posted on this subReddit to share my work, the response almost made me cry because a tiny demo got so many people talking about this In the past 2 months Ive been training a new model→8 Oct 2026Open sourcing Game Summoner, my prompt to game sitetldr: Open sourced my prompt to game suite: https://github.com/ndamiano/ai-agent-test, it's MIT licensed, and this runs well on my 5090 with 64gb ram, but any model that can handle tool calls can mana
CompanyNVIDIA8 recent entries26 Sept 2026Are there still any hidden gem gpu's left?you read about gpu's like the Tesla P40, V100, AMD M150 etc that people pick up for cheap. probably many others as well. Most are either server gpu's being phased out or mining discards, right? The pr→30 Sept 2026Qwen3.8 flash next ISTA-DASLab GGUF 50t/s TG and 1500t/s PP with 12GB VRAM and 64GB RAM Laptop on 'Strata' engineI think most people are sleeping on this inference engine. I tried multiple llama.cpp forks and none of them comes close to the inference speed of Strata. Initial version had some bugs with kv cache, →30 Sept 2026Follow-up: my native Rust + Vulkan Transformer training backend — 14 days later, now 14 parity-verified architectures and full PEFTFollow-up to my post from about two weeks ago. A lot has changed since then, so I wanted to post an update on where the backend is now. Where the green architectures stand When I posted last time, 7 a→3 Oct 2026mlsubgen — subtitles in 45 languages for your videos, entirely on your own machineFull disclaimer: I've leaned heavily on Fable to develop this, but I've tested it thoroughly on my own library for a couple of months before putting it on GitHub. I live in Thailand, and it started as→4 Oct 2026Poor People Vulkan GPUs listHelp with this list. Give me your recommendation on 'not supported anymore' GPUs. Looking for budget and Vulkan friendly options. Most of the GPU are not supported by latest CUDA / ROCm. Often with so→5 Oct 2026Infermeld: a Linux kit for running one GGUF across AMD + NVIDIA GPUs with llama.cppFollowing on from club-5060ti and club-rdna16, I’ve put together Infermeld: a small, open-source Linux companion kit for running one GGUF across an AMD GPU and an NVIDIA GPU, powered by llama.cpp. I’m→5 Oct 2026How is it possible that qwen 27b is so good? When GPT 4o had a trillion parameters and was worse?Picture from a post in r/amodei . People were praising qwen and I'm just wondering, what kind of new technologies are at play here? Does qwen just have 'better' pre training data? That's more high qua→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res