Search: “hardware”

Nvidia Cosmos 3 is an open physical AI foundation model built on a mixture-of-transformers architecture that combines vision reasoning, world generation, and action prediction for reasoning, simulatio

Xiaomi achieved over 1,000 tokens per second output from a 1 trillion-parameter model using a single standard 8-GPU commodity node through extreme model-system codesign . The approach combines FP4 qua

The Mac mini M4 uses unified memory architecture where CPU and GPU share a single 24GB memory pool, while the RTX 5060 has dedicated VRAM. Mac mini M4 is preferred for large model inference (70B param

Cosmos3-Super-Image2Video is a 64B model for temporally coherent image-to-video generation . NVIDIA's Cosmos platform is designed to accelerate Physical AI development by enabling machines to understa

Cosmos3-Super-Text2Image is a 64 billion parameter omnimodal world model capable of generating high-quality images from text inputs as part of NVIDIA's Cosmos 3 foundation model platform. The model is

NVIDIA's Pixel Diffusion Decoder (PiD) is an open-source decoder that replaces VAE decoders in image generation pipelines without retraining, producing sharper fine details and textures through a lear

This post describes setting up OpenClaw with a self-hosted Qwen 3.6:35b language model backend running on an H100 GPU for approximately $1.60 per hour, eliminating API rate limits. The user shares the

Flux-genotype is a self-evolving AI kernel designed to run on CPU hardware using Ollama, featuring the capability to mutate and adapt its own architecture dynamically. This project demonstrates an app

Setting a GPU power limit using nvidia-smi reduces heat output by approximately 20% with only a 5-8% inference speed loss. Undervolting the GPU can reduce power consumption by 5-15% with zero performa

Ollama not using GPU is commonly diagnosed by running `ollama ps`—if it shows 100% CPU, Ollama isn't detecting the GPU. The issue typically has multiple causes, with common ones including driver probl

This post benchmarks three Qwen3.6 model variants against gpt-oss:120b when running on Apple Silicon hardware, evaluating their performance characteristics and practical usability. It documents both t

Ollama's MLX engine runs on NVIDIA GPUs via CUDA v13 on Windows and Linux. Users have reported that Ollama crashes on RTX 3060 with cuda_v13 in versions 0.13.0 through 0.15.6, while deleting the cuda_

The NVIDIA Tesla K80 is a legacy GPU that can run modern LLMs through Ollama with community-maintained patches, since official Ollama dropped support for CUDA Compute Capability 3.7 hardware. Each K80

GPU Compass is an open-source tool that tracks and displays real-time GPU pricing information across over 20 cloud providers. The platform likely helps machine learning practitioners and researchers c

Ernie Image Turbo is a text-to-image generation model that can run efficiently on consumer-grade hardware like RTX 30 series GPUs with limited VRAM by using INT8 quantization. The post discusses techn

This r/ollama post addresses a commonly reported behavior where Ollama's GPU usage remains at or near 100% even after a model has finished generating a response. Certain models appear to 'hang' after

Ollama consistently fails to run any model on Apple M5 hardware running macOS 26, with the Metal backend failing to initialize and the runner process terminating with a 500 Internal Server Error. The

This Reddit post from the r/ollama community discusses running local AI/LLM workloads using the NVIDIA RTX PRO 4000 Blackwell GPU via Ollama, a framework for running large language models locally. The

A Reddit post on r/StableDiffusion announces an in-person open source AI art hackathon held in Paris, co-organized with LTX (Lightricks' open source AI video model) and NVIDIA. The event likely invite

This Reddit post shares a Google Colab notebook that enables free AI voice cloning using Alibaba's Qwen3-TTS, which offers voice cloning, voice design, and ultra-high-quality human-like speech generat

This Reddit thread on r/ollama discusses whether the NVIDIA DGX Spark — powered by the GB10 Grace Blackwell Superchip and delivering 1 petaFLOP of performance — is a worthwhile investment for running

TurboOCR is a high-performance OCR project that combines PaddleOCR with NVIDIA TensorRT, implemented in C++ and CUDA, achieving throughput of 270–1,200 images per second using FP16 half-precision infe

A Reddit discussion from the r/ollama community where a user seeks advice on which AI language model to run locally using Ollama. Responses likely include hardware-based recommendations (such as RAM a

Having a ‘Killer Application’ that everyone wants to use helps sell hardware, plain and simple. DeepSeek V4 Flash 0731 isn’t an app of course, but I think it’s going to be the major catalyst for getti

Hey everyone, I could use some advice on setting up speculative decoding correctly with llama-server. My Hardware: GPUs: RTX 4090 + RTX 6000 Pro (120GB total VRAM) RAM: 32GB I am currently testing the

I'm mosty interested in 128-192GB VRAM with 128-256GB RAM to spare, so SSD streaming is basically not even necessary. Seems only FP4 is supported, so older hardware will likely be slow - no Unsloth GG

We run a local model instance in our company that the dev we hired built for us. We're a trade business and we want to further use our on hand hardware for it. The specs given we have is a 5090 gpu wi

I love the progress happening on open models but I feel like it is kind of getting clear that hardware to run good sized models is completely unaffordable for me right now. I know that you all love Qw

Built my 1st inference machine and have been tweaking models trying to get the most out of my modest hardware. I think I’m at a good place but I’m testing with my own prompts. I’ve looked into some of

Here is my experience, which I would like to share with you and I also would like to hear your thoughts and valuable tips&tricks. Hardware: Mac Mini M4 (32GB Unified Memory) Model Server: Ollama Orche

I keep seeing the 'use a big model via API as the architect, run local small/mid models as workers' pattern recommended for people with modest local hardware. I've been running it myself (orchestrator

I can run the mediums, but sometimes I want a faster option that's smarter than Qwen 27B/35B. On my hardware I get like 500 to 800 tok/s prefill and 16 to 22 tok/s gen on ~120B class models, which is

I've always found it a little strange that AI is everywhere, but the codecs we use in practice are the traditional hand-engineered systems like h.264, h.265, av1. Alexnet started the wave of neural ne

Greetings Community! 👋 I hope everyone is doing well! I'm Tauhid — Senior EEE student from a Bangladeshi University Today I'd like to share an open-source project I’ve been developing called P.A.I. (P

I was never a huge AI guy but I was starting to wonder if my ISP was really providing the speeds I was paying for, and it turns out they were, but I had a bottleneck somewhere in my hardware chain. I

I have a 48GB M5 Pro and have far too many development projects going that just don't need the power of Anthropic to churn through so have started looking into running local models and while it certai

Running a 70B parameter model in full 16-bit precision requires roughly 140GB of memory , which is beyond most consumer hardware. Professional-tier GPUs like the RTX PRO 6000 with 96GB GDDR7 enable fu

A Reddit discussion seeking Stable Diffusion model recommendations for faster and cleaner image outputs, addressing the reality that no single best model exists as the right choice depends on hardware

This post describes implementing DCGAN (Deep Convolutional Generative Adversarial Network) inference on resource-constrained microcontroller hardware, achieving image generation with a 12.6 million pa

This Reddit thread likely discusses hardware recommendations and considerations for purchasing a new laptop capable of running Ollama, a tool that enables running large language models locally. The di

The Qwen 3.6-27B and 35B models are designed as open-weight models that developers can run locally on their own hardware without requiring cloud services. Alibaba released a separate cloud-only produc

Flux 2-Klein-9B NVFP4 is a quantized image generation model that runs on mid-range GPUs like the RTX 3050. On this hardware, the model produces 1024-resolution images but with relatively slow inferenc

This r/StableDiffusion Reddit thread discusses community recommendations for photorealistic image generation models that can run within a 16GB VRAM constraint, a common hardware limit for consumer GPU

LogLens with local AI 'Ollama' is a Reddit post on r/ollama discussing the integration of LogLens — a log analysis tool — with Ollama to perform AI-powered log inspection entirely on local hardware, w

This r/ollama Reddit thread addresses a common question among the local AI community about whether reliable, self-hosted AI systems are genuinely achievable. Community members in this space typically

KIV is a project shared on r/MachineLearning presenting a drop-in replacement for HuggingFace's `DynamicCache` that enables up to 1 million token context windows on consumer hardware with only 12GB of