Search: “models”

If you haven't checked the paper: https://arxiv.org/abs/2608.09867 TLDR: the authors show that you can swap out the 'encrypted' reasoning of the biggest model, like Opus, Sol, and put them into weaker

Warning: Kinda long post. If you don't like reading, please skip for your own sanity. Also, I've got nothing to sell, just a tinkerer, so I just want to share ideas and learn from you guys too. When I

Hello everyone! ​I like dumb things. I like working with weak computers and microcontrollers. I like the simplicity and low electricity usage. Simply put, the efficiency of a 'dumb' PC. ​The first tim

Link: https://arxiv.org/pdf/2604.27883 Hi, Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may

Spent two nights getting deepseek-ai/DeepSeek-V4-Flash-0731 (284B MoE, 13B active, native FP4/FP8, 1M context) running production-grade on two DGX Sparks connected by one QSFP DAC cable. Everything —

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Mu

Avid claude code user here looking to do equivalent things with local models. Just want to plug in something like Qwen and have the interface be 1:1 with claude code. Any suggestion? submitted by /u/N

Today the most important factor in AI is cost. My idea is to split ML models inference (closed ones, proprietary) across server and edge computing on clients, and I would like to hear what do you thin

Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization

Tweet by u/hackerllama Could be copium, but I would love to see Gemma 4.1 there with unified audio input for all model sizes perhaps even up to 120B, much improved tool calling (even with the latest t

I have various quants of this model and am curious how they perform. can anyone recommend which benchmark would be a good test case for quantization effects? Maybe that can be completed with about 1 m

I’ve been using 3.6 27B Q4, and that quant is fast on an M5. The code has been average, but consistently “good enough.” And, after a year, I can see home LLMs being served at home much like streaming

I compared Qwen 35B-A3B MoE against Qwen 27B dense on a series of local coding-maintenance tasks. On my R9700/llama.cpp setup, the MoE model generated about 3.9× faster (~116 vs ~30 tok/s), but the co

Looking for V100 users to share your config and it's performance. GPU: Tesla V100 PCIE 32Gb Qwen3.6 27B Q4_K_M + Q8_0 MTP 128K context length Pi coding agent llama.cpp model preset: [*] spec-default =

Is anyone here successfully running DeepSeek-V4-Flash-0731 locally with vLLM, especially on AMD MI325X? My setup: GPU: 1x AMD Instinct MI325X Model: deepseek-ai/DeepSeek-V4-Flash-0731 vLLM: 0.26.0 ROC

Just came across this coding benchmark: SciCode Artificialanalysis.ai reports a ranking which contradicts the feeling we've towards those models in real life coding. Is Gemma 4 really that good, or a

Just wanted to share my agentic coding benchmark run of DSv4F 0731 at both High and Low reasoning efforts (not Max)... I ran a 109-question subset of Aider Polyglot (the JS/C++/Python languages), base

It is one of the best local models ever released, in both 20B and 120B versions. I always come back to it, especially the 120B version. Its only competition is, in my opinion, Qwen 3.5 122B, but that

AI9Stars has released G9v3-39A5B an open weights language model designed to deliver even stronger reasoning capabilities than ai9stars/G9v3-3B with its 39B and 5 active experts. It is released under t

https://preview.redd.it/3zcvpbds14hh1.png?width=1911&format=png&auto=webp&s=a79aafb71eeca97638da93d2591902631e897fd5 I tried a test similar to the recent model-quant comparisons, but this time I focus

Sorry if this is asked a lot, but I was wondering if there is any clear winner on the Quantization versus Model Size debate? I can run Qwen3.6 27b at Q8, Laguna at Q6, and the new Deepseek Flash at Q3

Every time a model drops from a Chinese lab the thread fills with people who already know who made it, and the guess is usually Alibaba. There was a thread here recently asking what separates the open

I've been thinking about building a pipeline to generate reasoning training data for LLMs, but I want to avoid the common failure mode of synthetic data where you just generate the same template with

Context: I want to give the community an Open Research (well open under Apache 2.0 clause) - tool that allows everyday users like us to look deeper into the local models we use consistently. Mechanist

Most quantization works like this: pick a bit depth, apply it everywhere, maybe let imatrix take a rough guess at what matters, ship it. Most don't check which specific weight groups can take a hit an

Recently I bought an Ollama Cloud sub and accidently spent my whole 5h quota upon using DeepSeek V4 Pro... but why? isnt it supposed to be a cheap model? Youd think there would be a correlation betwee

from kwaipilot: Following the release of KAT-Coder-V2.5 in July, we are pleased to release the open-weight version KAT-Coder-V2.5-Dev, an MOE model with a total parameter count of 35B and 3B activated

Shoutout to this awesome guy - https://www.reddit.com/r/LLM/s/IDUyU3v9ap Thanks to his project, BigMoeOnEdge https://github.com/Helldez/BigMoeOnEdge, I managed to successfully run a 35B MoE model on j

This post discusses running Google's Gemma 4 QAT (Quantized Aware Training) 12B model on a GPU with 8GB of memory while maintaining a 16k token context window. The author likely shares performance ben

Mistral-7B v0.3 model achieves significant memory optimization when running at 128K context length in llama.cpp, reducing live VRAM usage from 22,657 MiB to 13,235 MiB while maintaining minimal perfor

This Reddit post discusses a common ComfyUI issue where users cannot locate spatial upscaler models for img2vid workflows. The problem typically stems from placing spatial upscaler models in the wrong

Stability AI announced the launch of Stable Audio 3, a family of three AI music models and one audio-based special effects model. Most of these releases are 'open weight' models trained on licensed tr

Ring-2.6-1T is a trillion-parameter flagship reasoning model designed for real-world complex task scenarios, now available as an open-source model. The model features about 63B activated parameters pe

LTX-2.3 is an open-source video generation model released in January 2026 that supports LoRA fine-tuning for customizing styles, characters, and use cases. The Reddit post compiles available LTX-2.3 m

Kimi K2.6 is an open-source model featuring advanced coding, long-horizon execution, and agent swarm capabilities that is now available via Ollama Cloud . The model excels in coding and agentic tools

A user reported satisfaction with Gemma 4 Cloud but switched to GLM 5.1 due to API errors, noting that the alternative model consumes significantly more resources. The post likely discusses the perfor

Ernie Image Turbo is a text-to-image generation model that can run efficiently on consumer-grade hardware like RTX 30 series GPUs with limited VRAM by using INT8 quantization. The post discusses techn

This r/ollama thread addresses a known compatibility issue where importing multimodal GGUF models into Ollama using a separate mmproj (multimodal projector) file fails to enable vision capabilities. O

This r/ollama post addresses a commonly reported behavior where Ollama's GPU usage remains at or near 100% even after a model has finished generating a response. Certain models appear to 'hang' after

This r/StableDiffusion Reddit thread likely discusses the topic of hand generation quality when using ZIB and ZIT — two AI image generation models from the Z-Image ecosystem used in Stable Diffusion a

ERNIE Image is an open-source text-to-image generation model developed by Baidu, built on a single-stream Diffusion Transformer (DiT) paired with a lightweight Prompt Enhancer that expands brief user

A Reddit user on r/ollama conducted a hands-on benchmark comparing Gemma4:e4b (Google's compact ~4.5B effective-parameter edge model) against Gemma3:27B, GPT-4o-mini, and Gemini 2.5 Flash, all run or

This r/ollama post describes a streamlined method for performance-testing locally-running large language models using the Ollama framework, achievable with just three terminal commands. It likely intr

A developer shared on r/ollama how they built a custom VS Code extension that routes Claude API requests through locally-run models via Ollama, dramatically reducing cloud API costs to approximately $

A Reddit thread in r/ollama where a user reports experiencing AI hallucination issues when running local language models through Ollama. The discussion likely covers symptoms such as models generating

I was unable to retrieve the specific Reddit thread content from that URL through my search. Reddit threads often require direct access to load user-generated content, and the search did not return...