Knowledge catalogue
I’m not super knowledgeable of the entire AI industry, but as we see this industry grow and the Cold War that is happening between the US and China on AI development, I can’t help but notice what US c
I'm trying to figure out GPU Kernel optimization on older hardware like SM80(ampere) . Is there tools you guys use? Or frameworks? Im waiting for this framework https://www.reddit.com/r/LocalLLaMA/com
There is a lot of buzz around running 100B parameter models on cheap local hardware using ternary (1.58-bit) quantization like Microsoft's BitNet architecture. The theoretical hardware shortcuts are i
Hey everyone, I have been looking into how people source compute for their Inference workloads (and in general). I wanted to understand some specific pain points here. If you've used online services l
This Reddit thread from r/ollama features a user seeking community recommendations for motherboard, CPU, and RAM components to build a system optimized for running Ollama locally without a dedicated G
IC-LoRA (In-Context LoRA) enables conditioning video generation on reference video frames at inference time, allowing fine-grained video-to-video control on top of a text-to-video base model. Unlike t
This Reddit thread from r/ollama discusses community recommendations for purchasing or renting GPU servers to run Ollama and local LLMs. It likely covers options ranging from dedicated GPU server prov
A Reddit post in the r/ollama community seeking volunteers with diverse hardware setups to participate in a collaborative effort to benchmark the **behavioral reliability** of locally-run large langua
I have been working on this tool for months and there are a lot of new functionalities and tests that are going to be released in the next few weeks! The goal of the tool is to allow community members
I've been out of these forums for awhile but I figured I would provide a formal update on how this has been going now that it has some operation time under its belt, just to put the information out th
Ran DeepSeek V4-Flash-0731 — the full official checkpoint, not a re-quant — on commodity used hardware. Sharing because I couldn't find anyone else publishing Ampere results for this engine. Why bothe
A Reddit post on r/StableDiffusion in which a user shares a music video created entirely using AI-generated imagery produced on their own local hardware, likely using tools such as Stable Diffusion wi
This r/ollama community post is a practitioner's guide sharing personal, real-world experience running Ollama on everyday consumer hardware in 2026, covering which models, quantization settings, and c
A developer created a minimal IDE for Ollama that automatically detects hardware capabilities and provides a streamlined user interface without unnecessary complexity. The tool aims to simplify the us
OpenAI is fast-tracking development of its first AI-focused smartphone with mass production targeted for early 2027 , marking the company's entry into consumer hardware . The device, positioned as an
This Reddit post from r/ollama asks the community about AI agent orchestration software for managing multiple Hermes agents, posted by a retired individual with available hardware resources. The post
H3 weights went live on HuggingFace August 3rd and I started pulling them immediately. An omni-modal video model with native stereo audio in the same forward pass, where audio can actually drive the v
The past week I've been running DSv4 inference on my laptop by keeping everything RAM-resident except the MXFP4-experts (since expert pool is ~147GB and won't fit) TL;DR - read speed is the limiter mo
TL;DR: I am building a Windows-first local AI setup for people who want to try local LLMs without spending days choosing models, setting up Ollama, Docker, WSL, Open WebUI, agents, and tool permission
I am not a meteorologist, but I just read a very interesting article: https://arstechnica.com/science/2026/08/deepminds-hurricane-model-bought-forecasters-an-extra-day/ In a paper published on Thursda
ComfyUI is a node-based interface for Stable Diffusion that can be run on Windows using Docker and WSL2 (Windows Subsystem for Linux 2) for improved security and performance. This guide provides instr
This discussion explores using Ollama to run large language models locally for invoice processing while maintaining control over data. The thread likely addresses selecting appropriate smaller models
Models like Qwen3 and Llama 3.2 are commonly used for n8n RAG workflows , with selection depending on use case requirements. Running Ollama models locally requires at least 16 GB of RAM on your device
Hi everyone, I'm a systems analysis student researching a problem a lot of you probably know well: how much you actually trust the published VRAM/RAM requirements for open-source models before trying
I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama tangentially related Subreddit, and this was the conclusion. Feels pretty accu
I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o
torch-nvenc-compress is a Python library that leverages GPU NVENC (NVIDIA's hardware video encoding) to optimize PCIe bandwidth utilization by compressing data during transfer. The project implements
It's more accurate to say that many highly engaged online gamers are hostile to AI, not that 'gamers' as a whole are. Gaming is a huge community with hundreds of millions of people, and opinions vary
I'm hoping that one of you guys has been working on an inference engine or has somehow found improvements to running DSV4F on RDNA4 multi-GPU setups. I am currently building a custom inference engine
Been benchmarking DSv4 Flash 0731 on a Flow Z13 (Ryzen AI MAX+ 395, Radeon 8060S / gfx1151, 128GB LPDDR5X) for the past week. Figured I'd share what actually works and what doesn't — there are a lot o
Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Mu
https://github.com/NVIDIA-NeMo/Switchyard Finally an open source LLM router. An alternative to openrouter fusion and Sakana Fugu. Doesn't look like it does exactly what Sakana Fugu does according to i
A discussion on r/ollama exploring which high-performance LLM models can be effectively run locally on standard laptop hardware , likely covering model size comparisons, hardware requirements, and per
This Reddit discussion in the StableDiffusion community addresses recommendations for affordable, used or refurbished laptops equipped with GPUs suitable for running open-source image generation model
This Reddit post from r/ollama discusses a user's experience running Cline (an AI coding agent) with Ollama on a high-end local hardware setup consisting of an NVIDIA RTX 4090 with 24GB VRAM and an In
This Reddit thread from r/StableDiffusion discusses a performance issue where Roop Unleashed version 4.3.1 — a face-swapping tool commonly used alongside Stable Diffusion — fails to fully utilize the
A Reddit post from r/ollama in which a user shares their experience running a 31B parameter model locally using Ollama, reflecting on the surprisingly demanding hardware and infrastructure requirement
Code and instructions available here: https://github.com/albertoZurini/echo-dot-2-playground Hello there! After a few days of experimenting I was able to get a completely local voice pipeline running
LabyrinthBench measures the thing that actually kills long agent runs — whether a model can still use what it learned twenty turns ago — deterministically, with no LLM judge, on your own hardware, wit
Setup: GPU: RTX 3090 24GB RAM: 32GB ComfyUI 0.30.0 PyTorch 2.13.0+cu130 CUDA 13.0 SageAttention enabled (sageattention-2.2.0+cu130torch2.10.0andhigher.post6-cp310-abi3-win_amd64) Spectrum node + Euler
This post was not written by a clanker. Hey guys, I'm a comp sci major who wanted to introduce a cool project I built for quantizing models to ternary (1.58 bit) with as minimal of loss as possible, a
Hey everyone, I recently open-sourced a new Python package called ganfs (Generative Adversarial Network Feature Selection), and I wanted to share it with the community. The Problem: Selecting the best
Are there any benchmarks on these 4 bit quants, like how Artificial Analysis runs a slew of various benchmarks? If not, how can I run one (5x over for consistency) on them? I'm also very interested in
ML research is progressing at breakneck speed where frontier labs in both academia and industry have access to considerably large computes (GPUs). Where do small labs or independent researchers go in
Hi everyone! 👋 I built and trained the complete Transformer architecture from scratch using pure PyTorch (`torch.nn` primitives) based on the original 'Attention Is All You Need' paper. I trained the
I wanted to challenge myself and see how far I could push Krea 2 and LTX 2.3, so I decided to create a short cinematic action trailer. It ended up being one of the most enjoyable AI projects I've work
I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to
Quantized Krea 2 Turbo checkpoints for ComfyUI, up to 2x faster and about a third smaller than the usual FP8 version — no calibration dataset, no quality cliff. How to use it (short version): clone th
I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible. The current version
This handbook covers the technical aspects of running large language models efficiently at scale, focusing on GPU optimization techniques including GPU internals, key-value (KV) cache management, batc
Nvidia Cosmos 3 is an open physical AI foundation model built on a mixture-of-transformers architecture that combines vision reasoning, world generation, and action prediction for reasoning, simulatio
Xiaomi achieved over 1,000 tokens per second output from a 1 trillion-parameter model using a single standard 8-GPU commodity node through extreme model-system codesign . The approach combines FP4 qua
The Mac mini M4 uses unified memory architecture where CPU and GPU share a single 24GB memory pool, while the RTX 5060 has dedicated VRAM. Mac mini M4 is preferred for large model inference (70B param
Cosmos3-Super-Image2Video is a 64B model for temporally coherent image-to-video generation . NVIDIA's Cosmos platform is designed to accelerate Physical AI development by enabling machines to understa
Cosmos3-Super-Text2Image is a 64 billion parameter omnimodal world model capable of generating high-quality images from text inputs as part of NVIDIA's Cosmos 3 foundation model platform. The model is
NVIDIA's Pixel Diffusion Decoder (PiD) is an open-source decoder that replaces VAE decoders in image generation pipelines without retraining, producing sharper fine details and textures through a lear
This post describes setting up OpenClaw with a self-hosted Qwen 3.6:35b language model backend running on an H100 GPU for approximately $1.60 per hour, eliminating API rate limits. The user shares the
Flux-genotype is a self-evolving AI kernel designed to run on CPU hardware using Ollama, featuring the capability to mutate and adapt its own architecture dynamically. This project demonstrates an app
Setting a GPU power limit using nvidia-smi reduces heat output by approximately 20% with only a 5-8% inference speed loss. Undervolting the GPU can reduce power consumption by 5-15% with zero performa
Ollama not using GPU is commonly diagnosed by running `ollama ps`—if it shows 100% CPU, Ollama isn't detecting the GPU. The issue typically has multiple causes, with common ones including driver probl