All entries

I wanted to test what Qwen 3.8 27B running locally on a single 5090 could do in Row-Bot. I gave it just one prompt: 'I want you to create an animation showing what the current local model can do. Test

Follow up post of https://www.reddit.com/r/LocalLLaMA/s/nGMyKswrch. Thanks everyone who replied. I didn't change the specs. Might be loosing some of the memory bandwidth but will scale in future if I

What taste in humor do LLMs have? Do LLMs like certain types of jokes? Do they have strong opinions? Our report: LLMs have some overlap in humor taste, about 3/4 agreement but it ranges from 54.9% to

With the recent Nvidia Huggingface acquisition and frontier AI labs screaming about safety and putting guardrails everywhere, I think it's important that we have local models that aren't affected by a

I've been working on something for this community (and other self-hosted communities) for almost 5 months now. Personally, I would grade it as probably the closest open source equivalent of Manus and

https://reddit.com/link/1wi2zuc/video/rs1a4sx92xph1/player Hey everyone, I’ve been experimenting with local LLMs on Intel hardware, and I wanted a hyper-optimized, standalone CLI tool that didn't rely

I'm pretty sure it can be done with any model based on qwen4exp, which Qwen's next local models will be based on. You can use a quant that barely fits in VRAM and still run at the model's maximum cont

I implemented expert lookahead, a trick that gets a 10%+ performance improvement on MoE models running on low-memory macs (Qwen 3.8 flash in this case) while using expert-offloading/ssd-streaming (usi

So I would like to run qwen 3.8 27b locally for my ai agents, maybe even in parallel with other small LLMs (such as qwen3.5 9b, oss 20b etc..). What is the best hardware to do this? Not a video card b

Hey r/LocalLLaMA, We’ve released our full ShapeLearn GGUFs for Qwen 3.8 27B. Blog / Download models TL;DR 3.84 bpw (GPU-5) reaches 99.63% of BF16’s aggregate score of 8 benchmarks, being the most accu

example: https://huggingface.co/audnai/penclaw-GLM-5.3-abliterated-for-offensive-cyber. edit: good context for why this was done here https://www.reddit.com/r/LocalLLaMA/comments/1whbvpx/comment/pa16g

Disclaimer: no AI was used whatsoever to write this post Cautionary tale about chasing cheap tokens. exposé: https://kendell.dev/blog/crofaifalse/ reaction by nahcrof, announcing the shutdown of the s

EDIT: Sorry for the unclear title. This model is UkisAI's Swift-Qwen3.8-27B, not a new version of BottleCap AI's 3.6-ThinkingCap. All credit goes to UkisAI for making great fine-tune, and I made this

I liked DeepSeek V4.1 Flash as an agent model but at 16 t/s on ds4 it was painful to sit through a real turn. It seemed some redditors and m3 ultra owners appreciated my glm 53 flash optimizations, so

One thing I love about both Qwen 3.8 27B and Flash Next is the effort it puts in to make a good product. It is the opposite of a lazy model and 'good enough' is never what aims to accomplish. It goes

I have tried using qwen 3.6 and 3.8 q5 64k context and I'm just not getting the experience everyone here seems to get. I'm using open code and it will loop, forget what it's doing, and mess up the ui.

After two days of fighting with benchmarking infrastructure, I finally benchmarked IFM/K2-Horizon-7B on 16GB VRAM. TL/DR: it works, but far behind Qwen-3.8-27B. Setup I compared 3 models - Qwen3.8-27B

Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you. That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from

I have a custom fork of llama.cpp designed around the ampere architecture specifically (though many of the upgrades also translate to faster performance of blackwell + lovelace). The recommended confi

For years, the JVM has watched the AI revolution from the bench. Every model, AI framework, every breakthrough, built with/for Python. jinfer is an inference engine built for the JVM from first princi

I am a beginner, Took a while to get started, get everything right. This setup is native not container. Still not sure if I did this right, or if I can tune this more. Environment=HF_HUB_OFFLINE=1 Env

As you know, Qwen3.8 27B has built-in support for embeddings. It gives pretty decent results in llama.cpp with the following parameters: --embedding --pooling mean The problem is that enabling embeddi

Hey everyone! We’re launching Voice Acting Arena with LAION x TTS Arena, and we’d love your help trying it out 🙂 You’ll hear two anonymous AI performances of the same scene and script, then compare th

Two months ago I announced I had found a new dynamic quant method called Voodoo Quant which was SOTA for the most aggressive quant levels on some smaller Qwen3.5 GGUF models. I kept the methodology pr

Personally, I think the whole 'AI (LLMs) is going to take over' is just BS marketing. They've been pushing this doom-and-gloom since 2019, and we all know the models back then were nowhere near as cap

I reallise this is not exactly a locally hostable model, at least definitly not on my hardware. But I figured some of you might have some insight on this. I run an agentic coding benchmark. The model

I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison. All intelligence and price per task are sam

I know I know, it's great, we know. I've been working on tweaking inference engines for a week now and it's been one shotting most of my vague prompts without any issues. It will even write tests and

So we’re seeing how Anthropic and OpenAI are gunning for regulations. I think most of us realize that this is a ploy for them to achieve regulatory capture, thus securing their moat and kicking out op

Edit: Spelled Baseten not Base-10 Full episode of this available at https://www.youtube.com/watch?v=PrSf7IOYu-I It's interesting to see how Dwarkesh has had to come around to the evidence that we are

Do you have something you'd like to share with the r/LocalLLaMA community. This is the place for it! Recommendation on presentation: Please share plain english description of what your project does an

Note - This is translated from the actual blog link right at the bottom. A few days ago, DeepSeek v4.1 was released. It raised the ability of small models to a new level. AI is improving much faster t

https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-3 AA shipped a new benchmark last week as part of the Intelligence Index v4.3 update — a brand-new private eval that re

Suppose I create two machine learning models suppose tree and neural network for a task let's suppose regression problem, now suppose I am sending both of this paper to two different journals, now the

From initial testing it seems pretty solid so far. Asked it to compile the latest llama.cpp for CUDA and its doing well so far. If this thing holds up to its score then its SHOCKINGLY good for its siz

Hi, For single-GPU training, I’m using Hugging Face SFTTrainer with auto_find_batch_size=True, which automatically reduces the batch size after a CUDA OOM until it finds a batch size that works. I wou

I'm like you guys and am constantly experimenting with new models, seeing what they're all good at, how I can make use of them for certain projects and goals. I've been using Qwen 3.8 27b for minor co

My prompt is: Epic high-fantasy concept art, ultra-detailed. A road crossing golden plains leading toward a colossal dwarven mountain hall entrance recessed into themouth of a giant cave. The whole en

Model: https://huggingface.co/internlm/Intern-S2-397B Collection: https://huggingface.co/collections/internlm/intern-s2 From Intern Large Models on 𝕏: https://x.com/intern_lm/status/209942518458737097

I'm currently using Qwen3.5 4B as the brain of my local AI assistant because my hardware is relatively limited. One thing I really like about it is the speed. On my system, I'm getting around 40–50 to

The full K2 Horizon lineup is out on Artificial Analysis. The AA intelligence vs. parameters plots show that - 0.9B and 375B are bad - 3.7B and 7B are SOTA - 36B A4B is SOTA for hardware with poor mem

Hi everyone, Inspired by tools like Chessvision.ai, I wanted to take a different architectural approach and build a browser extension (ChessInsights AI) that performs chessboard detection and piece re

I have created a VAE model using PyTorch on White Wine dataset. Basically, the main goal is to discover a brand-new white wine recipe. It puts all the wines into a latent space, finds the best part wh

I'm starting a PhD where I will be doing Graph ML (somewhere along the lines of graph signal processing/ graph deep learning.) My eventual goal is research scientist at big tech, or whichever company

Pushed out this new update, hopefully decreases the instances of crashes. I torture tested this one for ~12 hours after my fixes and found no instability. Q4 K XL needs more fine tuning, which I will

Repo here- https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder Alrighty, back again with an update to my (manual, no LLM connected) Fantastic MiniMax H3 Prompt Builder. Yeah it's