Search: “models”

Innanzitutto, due cose per evitare fraintendimenti. Non sto cercando di sostenere che un modello o un'azienda siano migliori. Non ho alcun interesse personale in nessuno di essi. Volevo solo verificar

First 3 disposition results. Hi there! My name is Jerry and I recently built Enclosure, e deterministic environment for testing LLMs. It's a simulated office enfironment https://github.com/openconstru

Hi all, I'm looking for some advice picking models. I'm looking to try a project to ingest quite a large amount of docments into a knowledgebase, allowing me and others to ask questions about the data

I have 3 machines, My main one can run Qwen3.8 Flash Next at 13-15 tps, i also have an MacMini 16GB whcih can run Orninth 9B or Gemma4 12B easily and i have a Pi5 8B that can run a 3B model well. I wa

Hey everyone, I'm getting into running LLMs locally with Ollama and would like some recommendations on which models I should realistically run with my hardware. My PC specs GPU: RTX 5060 Ti 16GB VRAM

Been building this for a few months, mostly for myself, and it just got a proper release so figured I'd post it. It's a native GGUF inference runtime with OpenAI/Anthropic-compatible APIs and a chat U

Hey everyone! I'm curious to hear from people that use a combination of cloud-based frontier models and local ones for development. I'm planning to set something similar up and wanted to hear about ac

In july I posted here about aletha codex and I got a lot of support, and a decent amount of people started using it. I also got a lot of criticism too, which really helped me funnel this version 1.5 t

I've hit again a point where me as a developer have to take a break from all this slop shit. Im a Developer for 13+ years and i loved it. But i fell for the slop trap. First it started with copilot an

I've made a modification of llama.cpp MTP for people that want to run models like QWEN 27B on 16GB and similar setup, the focus is reducing the memory cost of MTP allowing more speed for less ctx cost

Repost because reddit keeps thinking this is piracy or illegal. It is neither. A lot of people are skeptical Nvidia will keep huggingface intact now that they will buy huggingface. There's a lot of do

Just noticed this today. I've been trying to be careful to not max out my pro budget, but alas I am just a fool. Anyway, just noticed these free models. Some of these are pretty high in terms of compu

A lot of people are skeptical Nvidia will keep huggingface intact now that they will buy huggingface. There's a lot of doom and gloom about not having any alternatives, removing nsfw models, saying th

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detai

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models. I saw anthropic post about how they'll start adding watermarks to their model responses an

Yesterday I downloaded Qwen3.8 as MLX from the Ollama library from the commandline on a MAC. Simple 'Ollama Pull xxx' This is the answer I get when I ask it what model it is... I swear to god this is

Hello all, I'm fairly new to the whole Llm space and could really use some guidance from people who've done similar work. Im building a pipeline to extract structured data from large-batch of invoices

So many comments begging for these models, and I get it. People from the labs frequent r/LocalLLaMA, so maybe the begging comments aren't useless. They make demand known at least. Same with polls, etc

Good day to all. I'm fairly new running local models and would like to pick your brains on an issue I'm running into. I'll first tell you the issue, then give you my set up. I run into issues where th

Needs a lot of requests compared to Qwen (almost twice) and Gemma (almost x3). Final score is fine, even though it is 'not a coding model' https://wonderrico.github.io/local_llm_benchmark/benchmark-ma

Hi all, I'm seeing so many conversations from people discussing how they're 'using the models wrong' and 'don't use Sol max as high is enough for you', yet all the conversations lack the nuance of wha

[Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing] When i first started this, it was meant to be a fully lightweight, extremely m

152 generations, deterministic grading (exact number/string/JSON/regex), no LLM judge on my 16GB laptop. task type | deepseek-r1:1.5b (1.1GB) | llama3.2:3b (2.0GB) | gemma:2b | codellama 7b arithmetic

I can run the mediums, but sometimes I want a faster option that's smarter than Qwen 27B/35B. On my hardware I get like 500 to 800 tok/s prefill and 16 to 22 tok/s gen on ~120B class models, which is

https://preview.redd.it/qoogv5m1skfh1.png?width=688&format=png&auto=webp&s=3f06fb28ceec3cd3ab95bcebd0c71374332aac9b Built a simple desktop GUI (Tkinter) that supports multiple local TTS engines (curre

This is a surprisingly large real-world deployment: 'GovGPT' is part of Austria’s Public AI initiative, running on sovereign infrastructure (in their BRZ - federal datacenter) with Mistral open-weight

A discussion on r/ollama exploring which high-performance LLM models can be effectively run locally on standard laptop hardware , likely covering model size comparisons, hardware requirements, and per

Ollama exposes an Anthropic-compatible Messages endpoint , allowing developers to run powerful open-source AI models locally with no API costs and pair them with Claude Code for a capable local AI cod

This post provides a detailed review and practical guide for setting up and testing Ollama with DeepSeek models specifically on Ryzen APU systems. It likely covers performance benchmarks, configuratio

Anthropic's Claude Code desktop app is strictly optimized for Anthropic's models , creating a 'walled garden' effect that restricts users to Claude exclusively. The redesigned Claude Code desktop app

This r/StableDiffusion post likely discusses the community challenge of using OneTrainer — a locally-installed, GUI-based LoRA training tool — to train LoRAs specifically compatible with Z Image (Tong

I've been investigating some serious ChatGPT reliability issues over the past several days, and my latest tests revealed something particularly interesting: Both GPT-5.6 and GPT-6 showed problems unde

UT-based small models, despite being trained from scratch on these tasks without internet-scale pre-training, consistently outperform most standard Transformer-based Large Language models (LLMs) by a

We built an engine, Kyojin, on top of ExLlamaV3 for Strix Halo (gfx1151, ROCm), and packed two 300B-class MoE models so each fits one 128 GB machine. First release, all measured on Ryzen AI Max+ 395.

https://preview.redd.it/xs01ds930nsh1.png?width=4800&format=png&auto=webp&s=939762bcf3fbd93e137d4f97e927dda38aaf30fd Back in July I shared LokalBot here. It's a free, open-source Mac app that records

There's so many new techniques like RL, RL LoRA, QLoRA, CPT LoRA. I believe i would have a usecase for them, but i don't know where to learn, youtube is filled with bad quality tutorials if i just sea

Been dealing with the same thing a lot of people here probably are - RAM prices keep climbing and buying a single machine with enough memory for the models I actually want to run just isn't realistic

I had Astra run a multi-hour investigation into speeding up local MoE inference while keeping the model weights and quantization unchanged. The experiments covered Qwen3.6-35B-A3B and Qwen3.8 Flash-Ne

I developed https://lora-hunt.info/ for finding LoRA/PEFT adapters that might work with local LLM setups. The site collects public model metadata from Hugging Face and tries to make base-model compati

Tested on Laptop with RTX 5060 8gb vram and 16gb ram Claude code - autocompact = false Ollama - Flash attention = 1, kv cache = q8_0 Gemma4 12B 64K - https://ollama.com/SetneufPT/Gemma4-12B-IT-QAT_Q4_

I got a lot of ads for meshy recently and went to try it with hilariously bad results. It apparently can't do anything but decorative figurines. I wanted a body shell for an rc car and it just couldn'

For a few days I've been working on creating a custom local-only harness for some work related research using Codex / GPT 5.6 Sol and the model feels not only dumber than usual, but straight up counte

I have deleted tons and tons of older models to make space since I can't afford storage anymore. Easily 10TB... Anyways, I have been considering deleting DeepSeekV3.2 but decide to run it one more tim

Worked some time ago on one of the ad teams at Yahoo, and this grew out of a question I kept returning to while there are there 'simple' models that are both simple, scalable, interpretable, and contr

This is invisible with llama.cpp or derivatives, but ninfer helpfully logs the sampler settings on each request and auto-configures the correct ones for the model. Basically, OpenCode will always send

I've been running Gemma 4 E4B with oMLX and I can't find any chat interfaces that directly send the audio file to the model instead of running the audio through a separate STT layer. I can confirm the