Knowledge catalogue
I am considering dumping my ChatGPT Plus subscription and go full local, but to do so I would first need to reach a decent result for quality (and reasonable speed). My 4090 fried itself out of nowher
I finished my home inference server. First I tried Lenovo p620 workstation and while it’s a good value overall it pissed me off with a ton of proprietary Lenovo shit to deal with and I return it in th
Not sure if anyone out there is working on this, don't see any on huggingface to try out. It might be too new at the moment, but it would be cool to see the difference between a tiny model trained wit
The first generation of our 150M model has just been released Its performance is similar to that of GPT2-Small The benchmarks: PIQA: 62.24% Hellaswag: 32.20% Arc-Easy: 44.91% Arc-Challenge: 25.00% Ari
A few people here mentioned interest in a way to test their custom Pi setups, so I figured I’d drop this here: RoastMyHarness The basic idea is a small engine that sets up an environment to run DeepSW
Don't upvote, just looking for CPU ram options for running qwen flash next, and maybe larger models (so maybe between 128 and 256gb total ram) with an existing cmp 170hx. Use case is agentic coding so
VLLM Benchmark: Prefill, Prompt processing - avg, 871.93 tok/s (3 hours constant running xhigh) - 10K prompt, 1000.26 tok/s (16 runs) - 90K prompt, 743,59 tok/s (16 runs) Decode, tok gen - avg, 38.39
I gave DS V4.1 Flash an HLE problem with a bash tool + 2 hours. Hour 1: it wrote three MILP solvers. (225,200) Hour 2: it downloaded the HLE dataset from Hugging Face, found the question, read the ans
Want to help me with a tiny ChatGPT experiment? I stumbled onto something interesting after seeing how differently ChatGPT users got responses to the exact same underspecified prompt, and now I’m curi
It's still a dream for many folks to run medium size(30B range) models @ Q8 with Unquantized KVCache (256K Context) on their GPUs. It would be awesome to have DeepSeek-V4.1-Flash's KVCache + Engram fo
Hi 👋 long time lurker It’s been a few years since the last post about Horse Racing within this sub. A few years ago I was reading a popular science book and came across the story Bill Benter and the s
Since we know Qwen 3.8 27B thinks quite long, but gives at least a good one-shot result where you can leave it to do everything on it own, how does it compare to the older series of models for very si
It’s always bothered me that after fine-tuning a model for a project, there isn’t a particularly easy way to host it without either running it locally and keeping a GPU on 24/7 or paying for an entire
One is from Sam Altman, the other Claude suggested and it works extremely well. The first question I ask: What are you least confident about right now. The AI will list like 6 to 7 things that it didn
Using the RTX3090 on a linux machine I built for it and running Qwen 3.8 27B getting average 100t/s compared to my MacBook 20t/s I think I can finally get rid of my Claude subscription, this is good e
Hey guy, does anyone know how to replicate this artstyle, i have tried finding lora on civitai, took help from claude and chatgpt but i am unable to recreate this. submitted by /u/Negative-Rise-9250 [
from internlm: We introduce Intern-S2-397B, our most capable multimodal foundation model for scientific intelligence and long-horizon agents. Intern-S2-397B scales along three critical dimensions: pre
I have a full model, it's ready to train. It's ~9b parameters. 9.4b to be more exact. That includes a 1/2/3 Engram table, Moonshot's AttnRes modeling, and RoPE / NoPE layering at 3:1 as more or less v
my question about LLM performance We see a lot of posts about token prediction, token generation per second, etc. But is it really the metric? I can see that DeepSeek V4 Flash 0731 (with DSPark; mac s
I'll start by saying I'm not talking about the models themselves, I'm aware that I can't come close to something like Fable's intelligence locally. Just wanted to get that out of the way. Basically. O
As a SWE I have ultimate belief that having a irreplaceable subscription on OpenAI and Anthropic models is actually against my beliefs (I'm not fond of becoming a digital slave and being unable to do
This is an evolution on top of Raymond's KV cache streaming fork - all credits to what enabled this goes to him. The basic idea behind what he enabled was a pool of memory in VRAM that is used differe
Hi All. Just tried to down the NVFP4 version from Ollama but it’s the MLX!? Anyone else been able to access this version? Only have 1 DGX so wanted the smaller footprint for user with Hermes. Any sugg
> 'make an svg of a frog playing on a chello on the back of a whale with carribean island in the back.' interestingly the svg looks different in the OpenWebUi preview then when looked at in preview (o
I'm documenting a reproducible setup for running current llama.cpp on a mixed AMD Vega system with both: Radeon RX Vega 64 / Vega10 / gfx900 (8GB HBM2) Radeon Instinct MI50 (Vega20) / gfx906 (16GB HBM
Just stumbled upon this. Seems to be a new 3B TTS model mainly focused on Indian languages (22 Indic languages + English). Apparently supports code-switching, romanized text, emotion/delivery control
Style transfer capabilities of different open-source methods 2026 Update This is the updated version of the study published at 2025.09.12. Link: https://www.reddit.com/r/StableDiffusion/comments/1nfoz
I’m developing this https://swarm.dafonet.com If someone want to collaborate? I’m looking for people with capabilities to test it in different infraestructures. Best regards Gabriel Giani submitted by
For anyone using MiniMax H3 in ComfyUI I made the official TaoMate H3 3 Step LoRA available in a ComfyUI compatible safetensors format. No retraining, no merge, no extra fine tuning. The model itself
Lately because of the current hardware shortage, unfortunately or fortunately, we can’t just throw infinite cloud compute at our problems, but we’re forced to actually care about what’s happening unde
The entire page of the NY Times today above the fold absent one article is AI (the models are just too strong/too dangerous, must be regulated). They forgot to include 'Sponsored by OpenAI' at the end
EDIT: Thanks from the Waymo AI Team! A huge thank you to the r/MachineLearning community for all the thoughtful, deep-dive questions today. If you want to dive deeper into our technical research, chec
Hey everyone, Giving AI agents access to MCP tools or shell commands is awesome, but prompt injection is still a huge headache. If an agent reads an untrusted document containing `ignore previous inst
Sept 9, 2026 hits an all time daily high of 447 new machine learning papers uploaded to cs.LG (https://arxiv.org/list/cs.LG/recent?skip=0&show=500). This is many times more papers than what a human be
What I have: - CPU: EPYC 7551 (32c/64T, Zen 1) - Board: Supermicro H11SSL-i (SP3), Rev 2.0 - RAM: 128 GB DDR4-2133 (all 8 channels full) - GPU: 2x RTX 3090 (48 GB total, PCIe 3.0) - 1500 W PSU What I
Applied science work, from workflow design, data pipeline, results analysis, article/reports writing and data publishing online. 5 projects I did in the past replicated from start to finish. 3x to 4x
I’ve been building Catalyst around a capability I want engineering agents to have: An AI agent can read a program and make a good guess about what matters. Catalyst lets it measure what actually matte
I find this new model at HF: 'Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding. Architecture Agnes-3.0-Flash
Q2 is there and Q4 is uploading as I type. Has his github been updated yet? How do you run this? https://huggingface.co/antirez/deepseek-v4.1-flash-gguf/tree/main submitted by /u/Queasy_Asparagus69 [l
Hey everyone! I'm curious to hear from people that use a combination of cloud-based frontier models and local ones for development. I'm planning to set something similar up and wanted to hear about ac
Blog Post : Per-tensor layout maps for GGUF quantization Reddit thread : New tensor type layouts for my GGUF uploads EDIT : Model card has updated things such as Graph, table, text, etc., submitted by
I’ve been researching which models to sub to and I’m currently not subbed to anything. I’ve definitely searched here and other places for opinions but a lot of what I was reading wasn’t current or 6mo
January 2025 I made the switch from OpenAI to Claude. It was awesome. No more annoying GPT Chats, I dove into the wonderful world of Claude Code, learned vibe coding with Github, Vercel on Next.js. Wa
I have a full M5 Pro Macbook Pro with 48GB of RAM, I'm just getting into this local space. Like many of you, the costs of using frontier/cloud models is getting kinda silly. I don't have any insane ne
I tried using deepseek v4.1 flash max subagents with astra extra high as an orchestrator. It was working perfectly with 1 until it suddenly stopped responding or doing work after 3 hours. Then trying
So, uh... the popularity of so-called humanlike Qwen (currently on top in this sub) made me realize just how clueless the general public is about the models they have. You'd be shocked but you don't n
Kimi routed some PLA requests to Claude for distillation purposes without warning the PLA users. There is rumor that 16 Moonshot employees were arrested for this leak. submitted by /u/Ok_Warning2146 [
Hey everyone — I’m building TensorSharp, an open-source LLM inference engine. Here are the latest DeepSeek V4.1 Flash GGUF results using its native ggml_cuda backend. Setup: 8× NVIDIA A40, layer split
Can someone explain to me why Ollama Cloud are billing with same peak/off peak windows as Deepseek since they are hosting the model across US/EU infrastructure? submitted by /u/veganmaister [link] [co
I'm a teacher looking into running an LLM locally. I've been using ChatGPT for a lot of this work, so I started looking at Ollama because I understand it's designed to run LLMs locally. That said, I'm
First off, I know that GLM, Qwen, and DeepSeek absolutely dominate in terms of SOTA Open Source models, and that’s what I use in my personal projects and for school, however, I’m also responsible for
For many, Vram is precious, I see many people recommend using --no-mmproj-offload to save gpu vram but it is painfully slow. Especially if you are using it with agentic coding. If possible, add that s
I have enabled the vision for the CIRU Strix UL4 quant of Qwen 3.8 flash next (others quants likely perform very similar) and tried it on a few things, then wanted to show my partner how great it work
I recently moved to Japan and ran into an issue where a lot of anime on Netflix only has Japanese subtitles here. I wanted something similar to the subtitle translation extensions that already exist,
https://github.com/Sayitobar/chat_llm_cookbook This is my personal project that took several months. I wanted to see whether a 2B small local model could simulate a six-person group chat trained & ran
I've been keeping journals on and off for more than 10 years. There were stretches when I wrote pretty regularly, then I'd stop for a few months. Usually I'd write down what happened, what I did, mayb
I've seen a lot of Super Mario showcases, but never a mashup or derivative work. So I took a Super Mario generation prompt from ChopSticksPlease's Post here - and added a Matrix twist - and found myse
Another new model dropped in the course of this week that is well deployable on consumer hardware: Nex N2.5 Mini I went with the recommended settings for the best generation quality and ran a few benc
Hi! I run both models on MTPLX on my m5 max, and since I have 128GB of ram I run the q8 27b. I think MTPLX only lets me run 'optimized for speed' which it says is a dynamic q4 with 8 bit attention. Bo
Hola all. Do you guys mind sharing your LLama.cpp config and system setup details for Qwen3.8 Flash Next? Model's quite big and tryining many combinations of llama.cpp options takes lots of time, so l