Search: “hardware”

If you’re trying to run local LLMs on a normal laptop, an older desktop, integrated graphics, limited VRAM, or simply the hardware you already own, r/LowEndLocalAI is meant for you. The idea is simple

I’m not super knowledgeable of the entire AI industry, but as we see this industry grow and the Cold War that is happening between the US and China on AI development, I can’t help but notice what US c

There is a lot of buzz around running 100B parameter models on cheap local hardware using ternary (1.58-bit) quantization like Microsoft's BitNet architecture. The theoretical hardware shortcuts are i

This Reddit thread from r/ollama features a user seeking community recommendations for motherboard, CPU, and RAM components to build a system optimized for running Ollama locally without a dedicated G

One of the things I like most about Ollama is how low it makes the barrier to experimenting with local models. The downside is that once you move away from high-end hardware, finding the right informa

We all want more tokens per second but I keep seeing confusion on what to expect for given hardware. For the decoding phase (TG/s) to produce one token all the model weights and KV cache needs to be r

A Reddit post in the r/ollama community seeking volunteers with diverse hardware setups to participate in a collaborative effort to benchmark the **behavioral reliability** of locally-run large langua

Yoyo I renamed https://github.com/syv-ai/qwen38-27b-rtx3090 to https://github.com/syv-ai/HyperQwen/ to focus more on Qwen models and 'local' hardware in general, than just a single card and a single m

Hi, I am thinking about putting some money / time into my local ai learning path and 'cleared' my attic where I found the following hardware. 3 x NUC11 (Core i5 1145G7 / 2,6 GHz, 64 GB RAM DDR4 2666MH

I have been working on this tool for months and there are a lot of new functionalities and tests that are going to be released in the next few weeks! The goal of the tool is to allow community members

A Reddit post on r/StableDiffusion in which a user shares a music video created entirely using AI-generated imagery produced on their own local hardware, likely using tools such as Stable Diffusion wi

After some recommendations for hardware (I'm starting from nothing), in the region of £6-7k. I will be mainly looking to use it for coding and have found Deepseek V4.1 Flash or Qwen3.8 27b good and so

Where to even begin on this one...I guess I should start by acknowledging the risk vs reward for vendors other than nvidia, so: Yes I understand that nvidia are dominant currently on speed (llm and im

So I would like to run qwen 3.8 27b locally for my ai agents, maybe even in parallel with other small LLMs (such as qwen3.5 9b, oss 20b etc..). What is the best hardware to do this? Not a video card b

A developer created a minimal IDE for Ollama that automatically detects hardware capabilities and provides a streamlined user interface without unnecessary complexity. The tool aims to simplify the us

OpenAI is fast-tracking development of its first AI-focused smartphone with mass production targeted for early 2027 , marking the company's entry into consumer hardware . The device, positioned as an

This Reddit post from r/ollama asks the community about AI agent orchestration software for managing multiple Hermes agents, posted by a retired individual with available hardware resources. The post

This r/ollama community post is a practitioner's guide sharing personal, real-world experience running Ollama on everyday consumer hardware in 2026, covering which models, quantization settings, and c

I kept picking local models badly. Parameter count, quantization, VRAM. I could read all of it and still not know whether a model would be any good on my machine. So I stopped guessing and wrote somet

We are considering to buy computer, server to run decent model on premises. I would like to run big open source model with more that 70b parameters roughly. I read the people used to run on Apple Stud

Results up front: I had to prioritize prefill or token generation - there was no happy medium. Using UD-Q4_K_XL, q8 kv cache, and 96k max context: focus on generation (MTP = 2): 350 pp and 36 tg @ 12k

Hey everyone! 👋 Feeding multi-gigabyte files (SQL dumps, videos, audio, PDFs, and spreadsheets) directly into LLMs often causes context window overflow, GPU VRAM bottlenecks, massive token bills, and

Hello everyone, I made a calculator for the theoretical HW roofline for decoding / prefill based on several parameters (LLM model architecture, quants, GPU, Memory, ..). It still a theoretical bound,

H3 weights went live on HuggingFace August 3rd and I started pulling them immediately. An omni-modal video model with native stereo audio in the same forward pass, where audio can actually drive the v

ComfyUI is a node-based interface for Stable Diffusion that can be run on Windows using Docker and WSL2 (Windows Subsystem for Linux 2) for improved security and performance. This guide provides instr

This discussion explores using Ollama to run large language models locally for invoice processing while maintaining control over data. The thread likely addresses selecting appropriate smaller models

Models like Qwen3 and Llama 3.2 are commonly used for n8n RAG workflows , with selection depending on use case requirements. Running Ollama models locally requires at least 16 GB of RAM on your device

IC-LoRA (In-Context LoRA) enables conditioning video generation on reference video frames at inference time, allowing fine-grained video-to-video control on top of a text-to-video base model. Unlike t

I've been wanting to test out LLM inference on FPGAs for a while now, but didn't have a big reason too since there were no frontier class models at 9B-27B scale (3.6 27B was great still but it didn't

Reupload: I'm adding this again, since previous post had a totally stupid title which made people assume this app is some kind of cloud/proprietary models UI - this is totally not the case here. In ve

Early this year when I was first looking at building up my inference capability you could get the 16GB Tesla P100s for between 60 and 80. Asked claude about it, told me absolutely not worth it. No ten

Its chat template has this line: {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %} xhigh is the most expensive of its three settings (low / medium / xhigh). If you never set one

Github link: https://github.com/thatblend/LLMPSP I wanted to see what the PSP can theoretically handle and I got my answer - a 90M model is about the max it can do without atrocious inference speeds.

Years ago I built a server, shelved it after some time, but recently realized I have it sitting there. I'm looking to pop in a Tesla V100 32GB card (with proper cooling). But are the component too old

I’m working on performance regression detection using machine learning/anomaly detection. My setup is basically: Healthy runs are used to learn normal behaviour Regression runs are used to see whether

Hi everyone, I'm a systems analysis student researching a problem a lot of you probably know well: how much you actually trust the published VRAM/RAM requirements for open-source models before trying

I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama tangentially related Subreddit, and this was the conclusion. Feels pretty accu

Note: This writeup was put together with the help of AI. So I've got dual AMD MI50 32GB cards. If you know these cards, you know the story - AMD dropped official ROCm support for gfx906 after ROCm 5.7

I am planning to buy hardware to run local models. I'm not sure what kind of hardware is best and what will I be able to run. I'm torn between getting a used M1 Max 64gb Macbook for €1500, M3 Max 128g

Yet another vLLM fork thread here, but this time its for older INT8-centric hardware. This is a complete INT8 serving stack for Qwen3.8 27B based on vLLM, AITER, and a 27B GPTQ INT8 quant w/ DFlash2.

torch-nvenc-compress is a Python library that leverages GPU NVENC (NVIDIA's hardware video encoding) to optimize PCIe bandwidth utilization by compressing data during transfer. The project implements

ninfer, dwarfstar, Splash, llamAmpere, gufo, etc. We've all seen them popping up, great tok/s, people loving them. Forks of llama.cpp or another engine, or made from scratch. For better or worse, the

https://reddit.com/link/1wi2zuc/video/rs1a4sx92xph1/player Hey everyone, I’ve been experimenting with local LLMs on Intel hardware, and I wanted a hyper-optimized, standalone CLI tool that didn't rely

This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all

hey again! I run a EU PC hardware price tracker PriceSquirrel, 25+ stores across 9 countries, and wanted to check: are GPU prices actually rising, or does it just feel that way? To make this defensibl

I'm trying to figure out GPU Kernel optimization on older hardware like SM80(ampere) . Is there tools you guys use? Or frameworks? Im waiting for this framework https://www.reddit.com/r/LocalLLaMA/com