Search: “reasoning”

Forgive the typos and rambling - human actually wrote this post 😂 I've had a Strix Halo board for about a year and been playing around with it for various projects when it's not just being a beefy lin

Upgraded to a 5070 Ti so I could run Qwen 3.8 27B, which works perfectly, but didn't want to let the old 4070 Ti go to waste. The cards would touch if I put them both in the PC and I knew the heat wou

*part 2 of an earlier post: previous quant comparison with voxel island creation this time I rented three rtx pro 6000 96gb, on each one I launched a qwen 3.8 27b quant and gave them 4 identical promp

I ended up fine tuning a mistral 7b to outperform our costly foundational model and saved $300k. I previously thought that fine tuning was pointless (it's definitely not) and that all these problems c

With hopes of a Qwen3.8-35B-A3B release now mostly dashed, many people including myself are looking at fine-tunes and other variants of Qwen3.6-35B-A3B to run on VRAM-limited hardware. I decided to tr

v4.5.0: Added native desktop control, agent execution budgets, concurrency limits, loop protection, and safer local embedding fallback. v4.6.0: Rebuilt agents around durable parent-led orchestration.

I’ve been experimenting with a custom inference/runtime research project called CRANE V2, mostly because I wanted to answer a stupid question: How far can you push absurdly large MoE models on an ordi

Reposting here after sharing this on [r/MachineLearning](r/MachineLearning) a few days ago, where it got a much better response than I expected (300+ upvotes, great questions, zero roasting) GitHub is

Hi, Back in late march I begun playing around with Qwen 3.6 27b and found like everyone else that it's notoriously good at tool calls, where every model I tried before just derailed after a few turns

I'm curious what other people are using for local LLM coding / agentic coding with only 8 GB of VRAM. My current setup is: Intel Core i7-11800H RTX 3070 Laptop, 8 GB VRAM 32 GB DDR4 RAM openSUSE Tumbl

I haven't had a lot of luck in the reliability of the output of any models so far for local coding use, and find myself relying on Claude's limited free sessions to assist with coding. My coding appli

I'm running Qwen 3.8 27B locally on a single GPU. I can push the context to 131k, but I'd rather run it faster at 64k if the agent can manage context properly. What I have in mind is pretty simple: on

Gemma 12B is obviously a very well trained model, I always thought the fine tuning they did on it wasn't really cut out for agentic coding. From my own experiences it struggles to use the tools it's g

Hello, So I've been trying lots of combinations in that never-ending landscape of options and settings. I wanted a proper quant of 3.8 27B running as fast as possible on my 5090 at 400W, with vision a

Earlier this year I posted about building what at the time I believe was the first 16x DGX Spark Cluster. I’m now adding 20 more Sparks to the cluster in my homelab server rack, giving me 4.6TB of uni

Hi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo

I've been working on a project for a few days and now all of a sudden chagpt is unusable.. It will list the next steps of implementation but never actually start working on them. anyone else? UPDATE I

Gemax 1.9.15 — What’s new LAN server — share Gemax on your local network; open the real IP from another PC or phone Mobile-friendly UI — responsive layout for phone browsers on the same Wi‑Fi Workspac

Sharp v22.1 is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. NInfer is a hyper-tailored inference engine that only runs certain Qwen models on

I finally made the move from LM Studio to vLLM thanks to this post https://www.reddit.com/r/LocalLLaMA/s/NmS9CgHvqz. I may not know what it all means yet but I’m going to start diving into the docs to

I kept running into the same wall. Every agentic coding plugin ships an autocomplete, and almost all of them lock it down — no model choice, or a subscription, or both. I don't want a chat panel, I do

ChatGPT is usually treated as a prompt box. Ask for a cinematic shot, add a few camera words, copy the result into a video model, and hope the ending survives. My problem was that the ending usually d

Hi fellows fully-local halos, after manually following existing guides, I decided to build an LLM API endpoint installation and optimization guide that works even when autonomously followed by my pi a

From the screenshots: Hy4 is now live, labeled 'Expert-Level Model' + 'Use Tools to Solve Problems' Hy3 is tagged with 'New Upgrade,' positioned as a brand-new general-purpose model DeepSeek, focused

None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research. Two key highlights: - They use WSM to replace LR decay with

llama.cpp pr #27342 adds dflash2, so i rented an rtx 6000 and ran the same four prompts through four decoding setups on qwen3.8 27B median results over the four tasks: baseline 47.4 tok/s mtp 114.7 to

Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It

Aloha! 🌺Introducing Ornith-1.5, a family of open-source LLMs spanning 9B Dense, 35B MoE, and 397B MoE, trained with self-improving strategies. It achieves state-of-the-art performance among open-sourc

Any LLM will hallucinate the world knowledge, even a 3T model. Use a 4B with a kiwix skill and local Wikipedia, 50gb and no more hallucinated world knowledge. Ask your coding agent to build your own,

DeepSeek released on a paper on pretrained engrams in January. I’m surprised ds didn’t release engrams with v4 pro. When will ds or another lab release a fixed engram model? Fixed engrams will be the

Two days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak) - yesterday's update took that to 99 tps single-user / ~1,000 tps at 64 concurr

Here's the DFlash2 announcement, and I was pretty excited for this after trying out DSpark on llama.cpp a few days ago and being somewhat disappointed that it wasn't really working. Anyways, I spent a

When Qwen3.8-27B dropped on Thursday the obvious question came up for us : does a 27B dense model actually fit on the 24GB Mac Mini machines? Ran it properly over the weekend on an M4 Pro (24GB unifie

EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment (TabbyAPI), has a OpenAI compatible API so it shouldn't matter. Why won't this tool m

Support for the new ling 3.0 models has been merged into llama.cpp: https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828 Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

pagedMark removes AI provenance from content you generated yourself. Two different things, and it is worth separating them. The first is metadata: C2PA Content Credentials, EXIF, XMP, IPTC, the genera

Single prompt 'make me a 3d scene of an anime girl' (stolen from someone on discord, I like it a lot since it seems like every model struggles a lot with it) Everything is one shot. No vision, no iter

So a bit of context, I am a cybersecurity senior analyst I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts I started this field by doing a

https://huggingface.co/Motif-Technologies/Motif-3-NVFP4 On benchmarks, Motif 3 scores very, very close to DeepSeek V4 Flash 0731 in English vs. DeepSeek V4 Flash 0731 in English in various benchmarks,

I’ve experimented with Cursor, Antigravity, Claude Code/CLI, Ollama, Gemma 4B, MiniMax, Hermes Agent, and different local/cloud models. My problem isn’t knowing what these tools are—I don't know how t

So, I have been dreaming of getting 17 tokens per second using my RTX 3050 6GB version on a decent context window for Hermes needed above 60k. The hope is that has was a 22GB of DDR 4, hoping they can

Hey all! This started as a way to mark my 15th anniversary working in video game cinematics. I thought it would be fun to make a completely ridiculous, fictionalized version of how I got into the indu

dots3-note preview is the first open-weight model in the dots3 family. It is a Mixture-of-Experts model with 280B total parameters, 16B activated parameters, and support for a context length of up to

The maximum frame count allowed by the native 'MiniMax H3 Reference to Video' node technically is 1008, even if that's way over the training range of the model, which is 324 frames. But why not try? S

I’m genuinely pretty disappointed with how my Ollama Pro annual subscription turned out. For context, I’m a technology editor and developer, and I had actually used Ollama Pro before. My previous expe

I may have stumbled onto something interesting while trying to figure out a recurring artifact in ChatGPT image generation and editing (maybe applicable to other models as well?). It started with a ve