Search: “hardware”

A little over a year ago I got frustrated trying to fine-tune SDXL on my RTX 3060. Every option either forced lower resolution, locked away important settings behind massive config files, or needed a

Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for the specific hardwa

tests done power limiting the GPU on LACT - real power usage varies wildy at 20W it ranges from 25W to 56W same behavior happens on every setting prompt for the test runs: https://github.com/lukesdevl

I’m not affiliated with this project, but I’ve been running it recently and I’m surprised it hasn’t received more attention here: https://github.com/fewtarius/CachyLLama CachyLLama is a fork of llama.

If you want to get the most out of MTP. You have to run some tests / benchmarks to do so. Turning it on with defaults will get improvements, but for many models and card combinations, you are leaving

Model quantization has become essential for efficient large language model deployment, yet existing approaches involve clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but

I've been building a C99 inference engine from scratch (no Python, no BLAS, just gcc and make) that runs BitNet's ternary models on CPU. A few weeks ago I got obsessed with the matmul kernel - wrote a

TLDR: I (with the help of AI) re-implemented every Blackwell-only kernel (DeepGEMM, FlashInfer sparse-MLA, block-scaled FP8) in Triton, because they simply don't exist for sm89. The performance is 2-3

In the past few months I’ve experimenting heavily and tortured my old 2016 Desktop PC to run the biggest Local LLM I can fit. I documented the whole process and research and I’ve published a repositor

TL;DR: Everyone says QLoRA past ~13B is dead on a 24GB card. I got the full 32B FLUX.2 dev transformer QLoRA-training resident on the GPU on a 7900 XTX under native ROCm on Windows (no ZLUDA, no CUDA

Paper:https://arxiv.org/abs/2607.19058 Code (GitHub):https://github.com/nuemaan/skewadam Hi everyone, I just published a preprint on a new optimizer designed to tackle the massive VRAM bottleneck in M

Hey everyone, I’m a Full-Stack Developer with 6+ years of experience. I’m relatively new to AI-assisted development workflows and want to build a production-ready, enterprise-level Next.js web applica

I have a 48GB M5 Pro and have far too many development projects going that just don't need the power of Anthropic to churn through so have started looking into running local models and while it certai

I managed to get this model working on 2x 3090s with full 262k ctx and N=4, if anyone is interested to try it, thanks to this quant: https://huggingface.co/danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B

Hey r/LocalLLaMA — solo researcher here. I built a new attention architecture and want independent testers. Wave Field LLM replaces O(N²) dot-product attention with FFT wave convolution on a field. Tr

A developer created a fully local, CPU-based voice interface for Ollama that enables hands-free conversation with AI models by combining three open-source components: Silero VAD (voice activity detect

This post discusses running Google's Gemma 4 QAT (Quantized Aware Training) 12B model on a GPU with 8GB of memory while maintaining a 16k token context window. The author likely shares performance ben

The best open-source image generation models in 2026 include FLUX.1 [schnell], Stable Diffusion 3.5 Large, HiDream-I1-Full, SANA-Sprint 1.6B, and HunyuanImage-3.0 . FLUX.1 [dev] holds the crown for ph

This post documents testing Latent Consistency Models (LCM) on a GTX 750 Ti graphics card with 4GB of VRAM, demonstrating that this older, lower-end GPU can still run AI image generation models with a

This post seeks guidance on configuring local AI infrastructure on an AMD-based Linux system (7900 XTX GPU with CachyOS), specifically covering the setup of large language models via Ollama, AI agents

A developer documented their experience building a fully-local paper Retrieval-Augmented Generation (RAG) system using two NVIDIA GTX 1080 Ti GPUs and one RTX 3090, sharing three significant challenge

Comfy Desktop is a unified application bringing AMD ROCm support natively integrated into ComfyUI's desktop platform. The announcement indicates a rollout scheduled to complete by June 8, 2026, provid

This post documents running the Qwen3.6-35B-A3B language model on dual GTX 1080 Ti GPUs using Ollama, achieving approximately 20 tokens per second. The author highlights three critical configuration r

This post documents a creative project combining multiple AI image and video generation tools—SDXL, DMD-2, SEEDVR2, and LTX-2.3—with video editing software Shotcut to create a multi-day production. Th

A user reports successfully running Qwen 3.6 35b MoE (mixture of experts) with Zoo Code on an M1 Max Mac, achieving local inference without external servers. The setup enables fully local, battery-pow

Ollama exposes an Anthropic-compatible Messages endpoint , allowing developers to run powerful open-source AI models locally with no API costs and pair them with Claude Code for a capable local AI cod

This Reddit post discusses a technical setup for running Ollama with three NVIDIA RTX 3060 GPUs (each with 12GB VRAM) on a Machinist motherboard paired with a Xeon processor and 32GB of system RAM. Th

This post describes a custom Stable Diffusion graphical interface optimized for older graphics cards, specifically the GTX 1060, incorporating features like zero-copy ADetailer (a detail enhancement t

A discussion from the r/ollama community about using local large language models for privacy-sensitive applications, particularly for handling sensitive documents like medical records. Local LLMs proc

This guide provides instructions for setting up local AI capabilities on a 2019 Mac Pro running Ubuntu 24.04, covering the installation and configuration of ROCm 7.2.3 GPU drivers, PyTorch 2.10 for ma

Comfyui-Mesh now supports LTX 2.3 with the ability to distribute models across multiple GPUs or networked machines using Nvenc codec for video generation. The update includes significant VRAM optimiza

This guide demonstrates that modern AI image generation models beyond Stable Diffusion 1.5 can run on a GTX 1060 6GB GPU using optimization techniques and ComfyUI, countering the common misconception

This Reddit post likely compares inference performance metrics across popular language models running on Ollama, measuring tokens per second as a key performance indicator. The post would help users a

LTX-2.3 INT8 benchmarks discuss INT8 quantization optimized for Ampere GPUs (RTX 30XX series), which offers a balance between speed, VRAM usage, and quality. These INT8 models are designed to speed up

Bare Metal: Z-Image Turbo - Flux.2 Klein 9b - Wan 2.2 discusses comparisons between these lightweight AI image generation models, with Flux.2 Klein 9B showing good prompt adherence and natural composi

This post provides a detailed review and practical guide for setting up and testing Ollama with DeepSeek models specifically on Ryzen APU systems. It likely covers performance benchmarks, configuratio

A Reddit post from r/StableDiffusion where a user expresses amazement at the realism of AI-generated images, specifically noting they achieved impressive results using Z-image-Turbo on a consumer-grad

Ollama on mobile phones enables developers and enthusiasts to build privacy-first apps that process data locally and create offline AI tools for tasks like summarization, translation, and chatbots, re

This discussion examines the practical performance gains of quantization techniques when deployed in production environments, particularly addressing whether theoretical speedups translate to real-wor

This post discusses upgrade recommendations for improving local LLM inference performance on a gaming laptop with an RTX 4080 (12GB VRAM) and 16GB system RAM. Community members likely shared options f

This Reddit post discusses the viability of using second-hand custom-built RTX 2080 Ti GPUs with 22GB VRAM for running Stable Diffusion models in ComfyUI, including whether two cards could be linked v

This discussion thread explores lightweight vision-language models that can run locally, including options like Llama 3.2 Vision, Qwen2.5-VL, and SmolVLM2, optimized for tasks like OCR and visual ques

A seasoned developer seeking advice on selecting an Apple product to locally host large language models in the 27-36 billion parameter range, discussing the trade-offs and specifications of different

MiMo-V2.5 is Xiaomi's multimodal AI model with native visual and audio understanding that supports up to 1 million tokens of context. The GGUF format refers to quantized versions of the model optimize

Qwen3.6 27B is a 27-billion parameter language model that can achieve approximately 60 tokens per second throughput when running on dual RTX 5060 Ti GPUs with 16GB memory each, using the vLLM inferenc

I need to check the actual content of this post to provide an accurate summary. FLUX.2 [klein] 9B is a 9 billion parameter image generation model capable of generating images from text descriptions an