hardware

https://github.com/NVIDIA-NeMo/Switchyard Finally an open source LLM router. An alternative to openrouter fusion and Sakana Fugu. Doesn't look like it does exactly what Sakana Fugu does according to i

The past week I've been running DSv4 inference on my laptop by keeping everything RAM-resident except the MXFP4-experts (since expert pool is ~147GB and won't fit) TL;DR - read speed is the limiter mo

I am not a meteorologist, but I just read a very interesting article: https://arstechnica.com/science/2026/08/deepminds-hurricane-model-bought-forecasters-an-extra-day/ In a paper published on Thursda

Setup: GPU: RTX 3090 24GB RAM: 32GB ComfyUI 0.30.0 PyTorch 2.13.0+cu130 CUDA 13.0 SageAttention enabled (sageattention-2.2.0+cu130torch2.10.0andhigher.post6-cp310-abi3-win_amd64) Spectrum node + Euler

I'm hoping that one of you guys has been working on an inference engine or has somehow found improvements to running DSV4F on RDNA4 multi-GPU setups. I am currently building a custom inference engine

I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o

Hey everyone, I recently open-sourced a new Python package called ganfs (Generative Adversarial Network Feature Selection), and I wanted to share it with the community. The Problem: Selecting the best

Are there any benchmarks on these 4 bit quants, like how Artificial Analysis runs a slew of various benchmarks? If not, how can I run one (5x over for consistency) on them? I'm also very interested in

ML research is progressing at breakneck speed where frontier labs in both academia and industry have access to considerably large computes (GPUs). Where do small labs or independent researchers go in

Hi everyone! 👋 I built and trained the complete Transformer architecture from scratch using pure PyTorch (`torch.nn` primitives) based on the original 'Attention Is All You Need' paper. I trained the

Hey everyone, I have been looking into how people source compute for their Inference workloads (and in general). I wanted to understand some specific pain points here. If you've used online services l

I wanted to challenge myself and see how far I could push Krea 2 and LTX 2.3, so I decided to create a short cinematic action trailer. It ended up being one of the most enjoyable AI projects I've work

Quantized Krea 2 Turbo checkpoints for ComfyUI, up to 2x faster and about a third smaller than the usual FP8 version — no calibration dataset, no quality cliff. How to use it (short version): clone th

I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible. The current version

This handbook covers the technical aspects of running large language models efficiently at scale, focusing on GPU optimization techniques including GPU internals, key-value (KV) cache management, batc

Nvidia Cosmos 3 is an open physical AI foundation model built on a mixture-of-transformers architecture that combines vision reasoning, world generation, and action prediction for reasoning, simulatio

Xiaomi achieved over 1,000 tokens per second output from a 1 trillion-parameter model using a single standard 8-GPU commodity node through extreme model-system codesign . The approach combines FP4 qua

The Mac mini M4 uses unified memory architecture where CPU and GPU share a single 24GB memory pool, while the RTX 5060 has dedicated VRAM. Mac mini M4 is preferred for large model inference (70B param

Cosmos3-Super-Image2Video is a 64B model for temporally coherent image-to-video generation . NVIDIA's Cosmos platform is designed to accelerate Physical AI development by enabling machines to understa

Cosmos3-Super-Text2Image is a 64 billion parameter omnimodal world model capable of generating high-quality images from text inputs as part of NVIDIA's Cosmos 3 foundation model platform. The model is

This Reddit discussion in the StableDiffusion community addresses recommendations for affordable, used or refurbished laptops equipped with GPUs suitable for running open-source image generation model

NVIDIA's Pixel Diffusion Decoder (PiD) is an open-source decoder that replaces VAE decoders in image generation pipelines without retraining, producing sharper fine details and textures through a lear

This post describes setting up OpenClaw with a self-hosted Qwen 3.6:35b language model backend running on an H100 GPU for approximately $1.60 per hour, eliminating API rate limits. The user shares the

Setting a GPU power limit using nvidia-smi reduces heat output by approximately 20% with only a 5-8% inference speed loss. Undervolting the GPU can reduce power consumption by 5-15% with zero performa

Ollama not using GPU is commonly diagnosed by running `ollama ps`—if it shows 100% CPU, Ollama isn't detecting the GPU. The issue typically has multiple causes, with common ones including driver probl

torch-nvenc-compress is a Python library that leverages GPU NVENC (NVIDIA's hardware video encoding) to optimize PCIe bandwidth utilization by compressing data during transfer. The project implements

Ollama's MLX engine runs on NVIDIA GPUs via CUDA v13 on Windows and Linux. Users have reported that Ollama crashes on RTX 3060 with cuda_v13 in versions 0.13.0 through 0.15.6, while deleting the cuda_

GPU Compass is an open-source tool that tracks and displays real-time GPU pricing information across over 20 cloud providers. The platform likely helps machine learning practitioners and researchers c

This r/ollama post addresses a commonly reported behavior where Ollama's GPU usage remains at or near 100% even after a model has finished generating a response. Certain models appear to 'hang' after

This Reddit thread from r/StableDiffusion discusses a performance issue where Roop Unleashed version 4.3.1 — a face-swapping tool commonly used alongside Stable Diffusion — fails to fully utilize the

This Reddit post from the r/ollama community discusses running local AI/LLM workloads using the NVIDIA RTX PRO 4000 Blackwell GPU via Ollama, a framework for running large language models locally. The

This Reddit thread from r/ollama features a user seeking community recommendations for motherboard, CPU, and RAM components to build a system optimized for running Ollama locally without a dedicated G

A Reddit post on r/StableDiffusion announces an in-person open source AI art hackathon held in Paris, co-organized with LTX (Lightricks' open source AI video model) and NVIDIA. The event likely invite

This Reddit thread from r/ollama discusses community recommendations for purchasing or renting GPU servers to run Ollama and local LLMs. It likely covers options ranging from dedicated GPU server prov

This Reddit post shares a Google Colab notebook that enables free AI voice cloning using Alibaba's Qwen3-TTS, which offers voice cloning, voice design, and ultra-high-quality human-like speech generat

This Reddit thread on r/ollama discusses whether the NVIDIA DGX Spark — powered by the GB10 Grace Blackwell Superchip and delivering 1 petaFLOP of performance — is a worthwhile investment for running

TurboOCR is a high-performance OCR project that combines PaddleOCR with NVIDIA TensorRT, implemented in C++ and CUDA, achieving throughput of 270–1,200 images per second using FP16 half-precision infe