Search: “models”

Hi everyone, This is probably a simple problem, but I can't get my head around it. I am running ollama 0.33.3 on Linux Mint Cinnamon 21.3 with separate partitions for root and home. The standard insta

So, ideally for this thread we exclude the ones that everyone on here is already well aware of and discussing on here a lot, like N-gram, quantization improvements, MTP, D-flash, and D-spark, since th

Muse Glimmer Cookbook Clone it, run it on the GPU you already own, and ship a working agent — one that plans, calls tools, and self-corrects — in a single sitting, fully offline. Muse Glimmer is an op

Hi! I'm really obsessed with LLM hallucinations for the last 6 days 😭 I started by designing system prompts to attack hallucinations but failed, obviously. Now I tried reading logprobs and... I think

Greetings Community! 👋 I hope everyone is doing well! I'm Tauhid — Senior EEE student from a Bangladeshi University Today I'd like to share an open-source project I’ve been developing called P.A.I. (P

TL;DR: I’m reproducing the trait-persistence result from arXiv:2606.24014 on one RTX 3090. Before I can test persistence I need to install a trait via RL — and my GRPO run moves the trait only +2.4 po

Looped language models repeatedly apply a weight-shared block to increase effective depth without increasing parameter count, but the resulting T sequential recurrent-block calls per generated token s

Early this year when I was first looking at building up my inference capability you could get the 16GB Tesla P100s for between 60 and 80. Asked claude about it, told me absolutely not worth it. No ten

Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastr

Tried the same detailed face prompt on a pile of models and in my Opinion KREA beats paid models Wanted something more real than the usual AI “pretty girl.” Same fictional character, same prompt, grid

Hey so a few months ago I got into local LLMs and didn’t really like the tools out there at the time, so I decided to build my own version of a Claude/Codex GUI for local models. I’ve been working on

Just ordered a TB5 enclosure plus 4TB SSD to prepare for M5 Ultra 512GB. I am planning to download the following models in advance. Would like to ask the M3 Ultra 512GB owners whether the quants are t

When using two GPU with llama.cpp with --split-mode layer you usually provide a --tensor-split value to fit the proportion of the allocation on each GPU, say you have 16GB + 12GB something like --tens

I read this somewhere and wasn't sure if it's still an issue or if it only affects external models versus all models used via the Claude Code CLI. 'This latest Ollama update finally fixes that issue w

The news about Qwen 3.8 Flash Next is the first I'm reading about n-gram tables. I may be completely misunderstanding how they work but it seems they could open the door for 1T+ parameter models to be

Hey Ollama community, If you are planning to fine-tune and build your own local Turkish coding models (using Unsloth / LoRA to export to GGUF and run in Ollama via custom Modelfiles), I've open-source

Hey guys! After a LOT of good feedback on our previous models like Supra-50M-Instruct and -Reasoning, many community likes, follows and upvotes we saw many community requests asking for new models. We

This is rather counterintuitive, since banning Chinese models would benefit them the most. Jensen Huang is also against the ban, although the interests here are more obvious. Do you think that if Arce

I wanted to know how cheap you can go and still run local models, so I ran Ollama CPU-only on a Youyeetoo X1S. It's a single-board x86 machine with a Celeron N5095 (Jasper Lake, 4C/4T, 15W), 16GB of R

There's a bunch of small MoE with around 1B active params, like LFM2.5 8B A1B and Granite 4.0h 7B A1B; and then there are models with 3B+ like Qwen 3.x ~30B A3B and Gemma 4 26B A4B, but those are alre

Two DGX Spark and a Connect-X7 cable give you about 250GB of usable memory for 7000 8000 USD. This allows using some interesting models at 4-bit. For what seemed like an eternity, the only serious mod

InvokeAI 6.13 is the largest community-driven release of the software, adding full support for Anima & Qwen Image models, API model integration (such as GPT Image), and new features including Prompt E

Google DeepMind released MedGemma, a collection of medical vision-language foundation models based on Gemma 3 in 4B and 27B parameter sizes, demonstrating advanced medical understanding and reasoning

I was unable to retrieve the content of the specified Reddit URL directly, as my web search tool does not fetch raw URLs or Reddit threads directly, and I've exhausted my search attempts for this t...

Users are running Google's Gemma 4 models locally via Ollama and accessing them on their phones through Open WebUI, reporting a positive experience. Since Ollama doesn't run natively on iOS or And...

With the vLLM Semantic Router team we released Decision 3.0 (d3), our new SOTA decision model family ranging from 0.6B to 27B, Apache-2.0, all reading text, images and video. How it works - Built on Q

I'm developing a benchmark for a low resource language and I don't want it to be leaked and used for training when it is being used to get predictions. For locally run models it shouldn't be a problem

I've been wanting to test out LLM inference on FPGAs for a while now, but didn't have a big reason too since there were no frontier class models at 9B-27B scale (3.6 27B was great still but it didn't

Meta came out with a banger paper https://arxiv.org/pdf/2606.00206, but it did not look at various quantizations supported in llama.cpp. So I did a run on 50 random MATH-500 questions (https://hugging

Sometimes it’s easy to forget that this sub and others like it are probably the extreme minority when it comes to this hobby. Most people, I would think, don’t use or can’t afford one good GPU, let al

Hey everyone, I’m new to running AI models locally and I’d like some advice. My laptop specs are: HP OMEN 15 CPU: Intel i5-8300H RAM: 16 GB GPU: NVIDIA GTX 1050 Ti 4 GB VRAM OS: Windows I’m planning t

Since we know Qwen 3.8 27B thinks quite long, but gives at least a good one-shot result where you can leave it to do everything on it own, how does it compare to the older series of models for very si

We have SWE-bench, Terminal-Bench, OSWorld, BrowseComp, etc. But I haven’t seen a good apples-to-apples benchmark for platforms like OpenAI Agents, Anthropic’s agent stack, AWS AgentCore, Google’s age

I'm using two different models. One Gemma 4 and the other qwen 3.5. Both of them are responding to my input either by silence ('thought for a moment.... then nothing') or nonsense: <|turn>user <|turn>

I'm looking for more information to expand my creativity around this. I don't really have a realistic idea of what people actually do with local AI yet, I mostly just want to explore the possibilities

I have been trying to squeeze TTS stack down far enough to run in a $3 chip which has 512kb of SRAM without NPU. While trying to get to that milestone i built sanoTTS which has - 11 voices, 6 language

qwen3.8 27B has seriously impressive benchmarks on its model card, but that's for the unquantised version. Almost everyone here will run one quant or another. Are there good benchmarks for how those q

A few people asked for this after the bandwidth thread, so here it is on its own instead of buried in a comment. The dense rule was simple: every token reads every weight, so tokens/sec ≈ bandwidth ÷

Ollama automatically unloads inactive models from memory based on inactivity parameters, with models being removed after a specified duration of non-use . By default, Ollama unloads a model after 5 mi

This Reddit post from r/ollama asks about story-writing language models that can run on a 5080 GPU with 16GB of VRAM . The discussion likely covers recommended open-source or quantized models suitable

A guide on disabling thinking mode in Ollama models using the CLI by running models with the `--think=false` flag or using `/set nothink` followed by a prompt. The post likely discusses how to configu

The Ollama Cloud Subscription includes a feature described as 'Run cloud models at a time,' which refers to how many AI models a user can have simultaneously loaded and running in the cloud at any giv

This Reddit post from r/MachineLearning discusses the concept of decomposing machine learning models into a graph database representation, treating a model's components — such as layers, weights, and

I was unable to retrieve the specific Reddit thread at the provided URL, and the search results did not return content from that exact post. The search results returned related but distinct issues ...

Quick note, not my character, original came from schmede_AI on youtube. But yes my question is do any local models currently have the ability to match and generate additional images of a character fro

Opus5.5, Sol6.1, Fable, and Astra have all proven they can and the scene has exploded this past week. Part of that is from the tools and feedback loops maturing though. Are any open weight models (at