Search: “models”

One of the frustrating things about local AI generation on Mac is getting different models and tools to work together smoothly and fast. Valtz is a new native macOS app powered by Vpipe, a high-perfor

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones. I had some weird looping issue in their chat, and some

Hey everyone, Sharing an educational tool called AskCyph Health Explorer to explore how single-letter DNA mutations lead to disease, how they alter 3D anatomy, inheritance probabilities, and available

Quick context: LocalLightChat (LLC) is a chat frontend for local LLMs. Download it, run it, that's it. No installation (unless you want to), no dependencies, and it runs on pretty much anything, inclu

I run local LLMs pretty much around the clock, but I also enjoy gaming on the same PC. Having models sitting in VRAM while gaming isn't exactly ideal, and constantly unloading and reloading them manua

https://preview.redd.it/r2ledkyuntuh1.png?width=1684&format=png&auto=webp&s=9ed4518b2d0f20f44835ebbfac605581da186a57 AesCode generates information-rich visual artifacts such as slides, posters, and da

No more excuses for GPU-poor folks not to start LLMing! Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) ~9.3x smaller than FP16 (ideal) | 98.2% of FP16 intell

I have an app that processes ~500 different documents every day. An agent analyzes them, sorts them, tags them, extracts various legal nuances, creates descriptions, etc. So far I've been using Qwen 3

[ I'm posting this on behalf of u/LodestoneRock since for some reason his post on this sub keep getting auto removed by Reddit. ] Hey everyone, First of all, thank you to everyone for supporting this

I wanted the uncensored Qwen3.8-27B (llmfan46's Heretic build, MTP head preserved) on my RTX 4080 with MTP on (because I've been testing TONS of configs, and realized that would be the one) And none o

A bit of backstory : i got lucky in August and was able to grab a CMP 170 HX 8GB for around 500 on alibaba. I tried later to get a second one for around 1k but my order got cancelled and prices went t

Post title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterpris

Disclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is

Liquid just dropped d1-omni-600M and it's a decision model! I ported it to runntime, the WebGPU inference library I'm currently working on. It's plain TypeScript on top of TypeGPU with no WASM and vir

Seems like a bit of a long shot, but since it’s so mystifying I figured I might as well ask: I have been trying to get Qwen 3.6 35B to function as a local implementation agent in my subagent driven de

I don't know if anyone else has ran into these issues but when using Qwen Flash Next, my confidence in it at iq4_xs is high but not 100%. I notice it not following instructions, hallucinating more oft

I made AnimeGen, app that allows you to run Anima on your mobile phone. Since last post I added hd image generation, and different aspect ratios. The app is now available on the App Store: https://app

Picture from a post in r/amodei . People were praising qwen and I'm just wondering, what kind of new technologies are at play here? Does qwen just have 'better' pre training data? That's more high qua

Hey r/LocalLLaMA, Like many anime fans, I've always been frustrated by traditional MT engines (like Google Translate or base DeepL) when dealing with raw Japanese anime: - They completely butcher Japa

Finally finished my fork: https://github.com/IceFog72/ik_llama.cpp Nothing else I wanted to add/try currently works In short, it now has: Bandwidth-adaptive CPU/GPU execution Shared expert-residency i

Purpose-built to stress-test computer vision models, depth cameras, and spatial AI against severe specular glare and geometric reflections. This 425-asset production archive features a custom faceted

IQ3_XXS weights are just under 80GB and my slowww DDR4+7900XTX is stabilizing around 45-70/s (sometimes higher while coding depending on mtp). Looking online I'm seeing similar results for users with

So this is maybe a niche problem, but at my job I work on a huge Python codebase and every time I change some shared function I'm basically playing roulette. grep tells who mentions it in the code bas

'TypeSafe AI sells Jev as a frontier-class reasoner that cannot hallucinate, built by the co-inventor of ChatGPT - fast, and almost free. We ran it live on 16,379 benchmark requests, measured its late

There seems to be a whole category of extremely narrow inference runtimes appearing: Strata, ninfer, DwarfStar, Splash, llamAmpere, gufo, etc. They deliberately give up the thing llama.cpp/vLLM are gr

I am one of the authors of FLEET - an algorithm that enhances Best-of-N generation by attributing external rewards to particular tokens and then uses MCTS to adjust logits during the next run. I find

I'm tired of watching my compute budget and topping off my Ollama account. It feels like I can use a mix with more flash models on Openrouter (things not all available at Ollama) and control my spend

Hey guys - half a tutorial/guide, and half an I built this, so I went for resources. This is something that I created for myself recently when I couldn't find any good existing solution. I wrote this

You may think that jev, OpenClaw or TurboQuant are super cool, but actually the coolest LLM invention happened two years ago As we all know, the best source of reliable information about LLMs is YouTu

Holo4-27B-GGUF is a vision-language model (VLM) for Computer Use, built on the Qwen3.8 dense architecture and developed by H Company. Used with the hai-agents harness, it can send screenshots and tool

No click bait baby I promise - I'm live streaming the training process kinda like MiMo. Original thread: https://www.reddit.com/r/LocalLLaMA/comments/1wpg4a8/im_trying_to_posttrain/ If you didn't see

DistribAI is a platform for distributed training! You can train massive or tiny models across tiny or massive amounts of consumer devices with ease! DistribAI is a platform I've been working on for a

About a month ago I was having FOMO and was going to spend coin I don't really have on new graphics cards. Instead of doing that though I decided to spend the money on a new motherboard+cpu and try to

I maintain TensorSharp, an open-source inference engine. It can now run Qwen-Image 2.1 locally for text-to-image generation and image editing, with support for its LoRA adapters. I’ve added configs fo

Original post: https://www.reddit.com/r/LocalLLaMA/comments/1woscea/contrastive_language_models/ (sorry I felt it wasn't giving CLM the highlight it deserves) What it is: a new projection head for Qwe

Many a praise have been sung on Qwen-3.8, but here is mine. Qwen-3.8 and I had a rocky start, because it thinks so much. Watching it working is painful, so you have to stop doing that. You have to let

Ming-Image-0.1-Design creates complete UIs, dashboards, infographics, and posters from structured prompts of up to 8K tokens, with consistent layouts, typography, colors, and imagery. It can also nati

Almost every single gen has this wispy, burned, piss-yellow ChatGPT-distilled thing going on. Background faces are absolutely abysmal. Even skin texture has this wispiness, like semen texture, and it'

I want to try using AI to find possibly overlooked interventions for irritable bowel syndrome (IBS) and create a cocktail of interventions that AI thinks would give the highest expected benefit, attac

You can simply run any GGUF with llama.cpp with n_predict=1 and n_probs=10, disable reasoning, and prompt it such as 'If the following email is spam, respond with 1, if not spam, respond with 0. Do no

Jev from TypeSafe AI got me thinking about non-autoregressive decision engines, so I built a small proof-of-concept called AES to test the concept locally on open weights. It uses Qwen/Qwen2.5-1.5B-In

we're a small lab (switzerland + south africa) and just open-sourced our first model: a 27b fine-tune of qwen3.8-27b specialized for creative writing and short-form personal text (stories, dialogue, r

I did a small test of the new hyped quantisation of Qwen3.8 vs the biggest quant which fits into my limited 16GB VRAM with decent context. The results are interesting. Of course, the smaller file give

This post is a follow up to this other post where I let Qwen 3.8 run for 63 hours autonomously to try to solve the Riemann hypothesis: https://www.reddit.com/r/LocalLLaMA/comments/1wi9fau/qwen_38_27b_

Been working on this backend migration for like two weeks and my chat with ChatGPT is one of those endless scrolls. Last Tuesday it confidently told me we'd 'decided' on the wrong database. We had lit

It's an old Mitch Herdberg routine about tricky survey questions. But every time Qwen 3.8 spits out one of those sentences that is structurally correct but makes no actual sense, it pops into my head.