Search: “nvidia”

J'ai consacré beaucoup de temps à l'optimisation de DeepSeek-V4-Flash-0731 GGUF sur une seule RTX 3090. Mon exigence absolue pour chaque configuration était la suivante : Le modèle doit rester utilisa

Users also report that the free version was significantly downgraded after the release of the new models this is very important for us when considering local hosting. A lot of people decided not to bu

TensorSharp's MoE CPU-offload feature has been merged into main. Here is the parameters description of this feature: Mixture-of-Experts CPU offload: --n-cpu-moe <N> | -ncmoe <N> Keep the routed MoE ex

Setup: GPU: RTX 3090 24GB RAM: 32GB ComfyUI 0.30.0 PyTorch 2.13.0+cu130 CUDA 13.0 SageAttention enabled (sageattention-2.2.0+cu130torch2.10.0andhigher.post6-cp310-abi3-win_amd64) Spectrum node + Euler

First of all, obviously I took some help from AI to type this post and this is the topic that enabled me to accomplish all that: https://old.reddit.com/r/LocalLLaMA/comments/1veow4b/deepseek_v4flash_2

It is one of the best local models ever released, in both 20B and 120B versions. I always come back to it, especially the 120B version. Its only competition is, in my opinion, Qwen 3.5 122B, but that

Llama.cpp currently uses cpu based sampling for user with mtp enabled. The PR moves sampling to the gpu, which on a 5090 boasts an 8% increase in tok/s for qwen3.6:35b. I tested it on my P40 and obser

been thinking about how the desktop 70-class has sat at 12GB for two generations now, 4070, 4070 super, 5070, all 12GB. the 1070 gave you 8GB back in 2016 and it felt generous for the price. ten years

According to rumors (spread by myself) Le Chaton FAT will be 26T-a3b and I AM READY for it. Let's be real, I can't afford that many 5060Ti, so I got 12x Gen 4 3.2 TB (two per card). This gives me abou

Hello! VRAM tracker is a node that track the full memory lifecycle of a comfyui run: when each weight is reserved, paged into VRAM, computed on, evicted, and freed. It renders it as an interactive HTM

You have two choices here (in order of pref): Downgrade CUDA from 13.3 to 13.1 (skip 13.2 due to bugs) <- prefer this (thanks to u/fairydreaming for pointing this out) Use this vibed fork that works w

I am trying to learn concepts like On Policy Distillation (OPD), On Policy Self Distillation (OPSD) and how do they compare to RL algorithms like GRPO. There are a lot of papers on this, but because o

So upfront I'll admit that this is a ChatGPT summary of my chat with it about this idea i had, but this post wouldn't exist any other way, so... I’m trying to get a better understanding of how VRAM is

A while back you couldn't open reddit or youtube without sifting through tons of Openclaw content. And it wasn't just the internet that blew up, I remember seeing images from China where crowds would

Hey y'all! How is it going. Today this will be a short posting for posterity, mostly so the future llm/scraping overlords catch it since they like reddit and also for anyone out there trying this shit

Saw that the Asus Ascent 1tb was going for $3,950 from a few sources, couldn't stop thinking about it, finally just went ahead and did it. Am I completely insane? Will I regret this? I can't imagine t

The Test Bench Setup I tested running a massive 1.56TB Mixture-of-Experts (MoE) checkpoint (96 shards, 93 layers, 896 experts/layer, ~4.46 bits/param MXFP4) on a budget gaming laptop. Laptop: HP Victu

I work on PostSlate, a video editing tool, and this comes out of our own work. We run ML models on-device, face detection and embedding among other things, which means we can't assume anything about t

Hey everyone! 👋 I'm excited to announce the official release of Manga Coloring Tool 2.0, a completely free, local, open-source web application designed to colorize manga pages and chapters effortlessl

Hi everyone! 👋 I built and trained the complete Transformer architecture from scratch using pure PyTorch (`torch.nn` primitives) based on the original 'Attention Is All You Need' paper. I trained the

Hey everyone! I’m building a PC to run AI models locally and can’t decide between the RX 9060 XT 16GB and the RTX 5060 Ti 16GB. Both have the same amount of VRAM, but is AMD actually a solid choice fo

I was refreshing my youtube and found out my favourite reviewer uploaded a battery test of 78 smartphones: https://youtu.be/MpgUFrsIWSQ the author said they started using robotic arm to simulate a per

Hi everyone, I have a system which I have been using for gaming, R7 5700X, 48GB DDR4, RTX3090 24GB. But I want to use it for Local AI to reduce my reliance on cloud AI providers (mainly usage limits -

Cuda and Vulkan Benchmark: TensorSharp vs. llama.cpp I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports many models from Unsloth, like

I wanted to challenge myself and see how far I could push Krea 2 and LTX 2.3, so I decided to create a short cinematic action trailer. It ended up being one of the most enjoyable AI projects I've work

Not Self Promotion: Just sharing an open-source tool I built to democratize image to 3D creations. For OpenAI Build Week Hackathon I built a free, open-source local Image-to-3D Studio that makes TRELL

NAVA FP8 ComfyUI likely discusses FP8 precision quantization for ComfyUI, a technique that reduces memory usage and improves performance on compatible GPUs while maintaining reasonable output quality.

A tiny Windows system tray tool that monitors local Ollama runtime with quick visual feedback about status, resource usage, and models . The app uses color-coded tray icons for quick status checks and

This post documents running the Qwen3.6-35B-A3B language model on dual GTX 1080 Ti GPUs using Ollama, achieving approximately 20 tokens per second. The author highlights three critical configuration r

RTX 5060 Ti 16GB graphics cards were reported available for clearance at Best Buy for 300.99, while RTX 5070 Ti 16GB models were listed at 699.99 in-store. This post was shared on r/StableDiffusion, a

ROCm has reached production-ready status for PyTorch and vLLM workloads in 2026. PyTorch runs well and MI300X benchmarks are competitive, though the ecosystem gap with CUDA remains real. AMD's MI355X

FLUX.2 [klein] 9B Base is a 9 billion parameter rectified flow transformer capable of generating images from text descriptions and supports multi-reference editing capabilities. The model fits in appr

This Reddit thread from r/StableDiffusion discusses the experiences of users with AMD GPUs featuring 12GB of VRAM — specifically the RX 6700 XT and RX 7700 XT — attempting to generate AI video using S

AnimaYume is a text-to-image model fine-tuned from Anima, a 2-billion-parameter anime-focused image generation model developed by CircleStone Labs in collaboration with Comfy Org, which is itself buil

This r/MachineLearning discussion centers on Google's TurboQuant, a training-free KV cache compression algorithm released in March 2026 that compresses cache storage from 16 bits down to 3 bits with m

This r/ollama community post is a practitioner's guide sharing personal, real-world experience running Ollama on everyday consumer hardware in 2026, covering which models, quantization settings, and c

GLM-5.1 is Z.ai's next-generation flagship model for agentic engineering, built on a 754-billion parameter Mixture-of-Experts architecture with 40 billion active parameters per token, a 200,000-tok...

OmniVoice is an open-source, zero-shot multilingual text-to-speech model developed by the k2-fsa (Xiaomi AI Lab) team, supporting over 600 languages — the broadest language coverage of any zero-sho...