Search: “models”

https://huggingface.co/Nanbeige/Nanbeige4.2-3B-DSpark 4b model for the gpu poor that I think is stronger than qwen 3.5 9b now faster! I was getting roughly 35 t/s AR which was this models downside, it

Hey all, You’ve probably noticed that using LLMs and AI agents for pentesting has become pretty common lately. The problem is there isn’t really a good way to figure out which model is actually best s

We're releasing Qwen3.8-27B quantized with our newest methods, GSQ + RCO. Higher-quality models, same file size, now with the search and the quantizer both learned. What's inside: GSQ (Gumbel-Softmax

Hey all, and hello fellow DGX Spark-ers! Today I managed some pretty crazy numbers: 181 tok/s aggregate on 2× DGX Spark on Qwen3.8-Flash-Next at 512Kcontext (2.8M kvc) I hit 181 tok/s aggregate today

I’ve been working on a small project called gemma4.c. The idea is pretty simple: you can download a modern language model, compile one 700-line C file, and have it generate text on an ordinary CPU. Th

TL:DR: power management was doing auto-suspend and dumping model weights into RAM. Models weights 18gb+ RAM 16gb = no Bueno immediate crash. amdgpu.runpm=0 in your grub disables runtime power manageme

Further to my last post, https://www.reddit.com/r/LocalLLaMA/comments/1vldngi/tested_in_coding_bf16_muse_glimmer_vs_bf16_qwen36/, I bring this requested comparison of Qwen3.8 27B. Context: Both models

What models and configs are we using? Please share here On windows, I am using this copium pared down model https://huggingface.co/Bucoid/Qwen3.8-27B-Uncensored-IQ4-XS-MTP-16GB-VRAM-GGUF with MTP disa

TL;DR: The result looks TOO GOOD TO BE TRUE — that's exactly why I'm posting it. I ran Ox alpha on the entire SWE-bench Verified-Mini set with the official mini-swe-agent scaffold — the same agent use

I'm testing it since release, now with UD 3.0 in my AMD R9700 with ROCm, I always read everywhere that F16 and q8_0 for KV cache are essentially the same... well, I tested it and I can see differences

Hey everyone, Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then. Repo: [https://github.co

None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research. Two key highlights: - They use WSM to replace LR decay with

I trained three LLMs from scratch in raw PyTorch then post-trained each one with SFT and then GRPO. Same process every time: same synthetic arithmetic curriculum, same reward function, same hyperparam

Hey everyone, I am trying to make a LLM interface optimized for AI-cowork experience. Heres what I've thought of and tried to fix: 1)You shouldn't have to copy and paste context for the AI model. 2)Yo

I have a lot of content in my personal Obsidian vault and would feel much more comfortable using LLMs with an offline model. I've experimented with Ollama and various small 3b/8b local models (given c

EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment (TabbyAPI), has a OpenAI compatible API so it shouldn't matter. Why won't this tool m

Including the rationalisation for the data below - this is a more robust version of an earlier post I did similar to this - explaining below: How I chose the comparisons The basic question I’m trying

I’m running Ollama on an M4 Pro Mac with 48 GB unified memory and testing local coding models for repository work. Direct Ollama inference is fine. Qwen3.8 27B-MLX runs well enough for me, and the loc

Poll at beginning of the year: https://www.reddit.com/r/LocalLLaMA/comments/1qj935h/poll_when_will_we_have_a_30b_open_weight_model_as/ The least voted option, 6 months, wins in my opinion, with 18 vot

Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level. The basic idea is to generate a custom ima

Hi LocalLlama, we're super excited to release Unsloth Desktop today! 🦥 It's the first desktop app that enables you to run and train models locally. Open-source. Available on Mac, Windows, and Linux Su

GitHub: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader Release v1.0.39: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader/releases/tag/v1.0.39 Also available through ComfyUI Manag

I did some testing this morning, and I was surprised to find that Muse Glimmer actually comfortably fits on a single RTX 3090 with full context + DFlash + mmproj at Q4_K_XL, unlike Qwen3.6-27B and Gem

table bench https://huggingface.co/bartowski/endless-frontier_BigBang-v1-GGUF I'm downloading this model only because Bartowski converted it to .gguf, so it might be interesting. Doubts : The headline

Hi folks, I hate slop as much as you do, so instead of starting with 'The Problem', I'll just cut to the chase: I just published Lophius, which is the culmination of more than two years of fighting wi

Code and instructions available here: https://github.com/albertoZurini/echo-dot-2-playground Hello there! After a few days of experimenting I was able to get a completely local voice pipeline running

I mean, Deepseek V4 Flash is an absolutely fantastic model, even though I can't run it on my machine it's so fascinating to see how it performs. Knowing that potentially it could be run at home is rea

Everyone's been doing 1.58-bit for LLMs, so we tried it on a vision transformer: Swin2SR (lightweight ×2 variant, 1.01M params), quantized so every weight is −1, 0, or +1 with a small per-group scale

It was developed under Phase 2 of Korea's Sovereign AI Foundation Model Project. ​Size: 750B parameters (3x larger than their 236B v1 model). ​- License: Apache 2.0 ​Languages: Expanded to 10 language

I really love this model, I have been using the q4_k_l by Bartowski (I have heard QAT is quite the downgrade in some aspects) and it handles every task I throw at it easily. Agentic and coding perform

From the description: 'Reasoning-Medical-27B is designed for universal advanced medical reasoning in professional medicine, medical genetics, college biology/medicine, and clinical knowledge. The mode

Running local models means every token counts — an 8K or 16K window fills up fast when you're stuffing graph context into prompts for RAG. I benchmarked 10 graph serialization formats (JSON, GraphML,

Cuda and Vulkan Benchmark: TensorSharp vs. llama.cpp I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports many models from Unsloth, like

If you want to get the most out of MTP. You have to run some tests / benchmarks to do so. Turning it on with defaults will get improvements, but for many models and card combinations, you are leaving

Ideogram released version 4.0 of its text-to-image model as an open-weight model with native 2K resolution, bounding box control, and improved text rendering. The model weights are available on GitHub

OpenStudio is a hybrid AI router that combines local model inference with cloud-based model access through OpenRouter, a unified API providing access to hundreds of AI models through a single endpoint

The Qwen 3.6-27B and 35B models are designed as open-weight models that developers can run locally on their own hardware without requiring cloud services. Alibaba released a separate cloud-only produc

This Reddit post from the r/ollama community discusses running local AI/LLM workloads using the NVIDIA RTX PRO 4000 Blackwell GPU via Ollama, a framework for running large language models locally. The

This r/ollama thread discusses user-reported issues with the Gemma 4 26B (a Mixture of Experts model) and its 'thinking' mode when used through Open WebUI. Key problems include the model getting stuck

This Reddit thread from r/StableDiffusion addresses the challenge faced by users in China trying to access and download models from Civitai, which may be restricted or slow due to network limitations

This is a ComfyUI automation workflow for the ACE-Step 1.5 XL music generation model that uses Qwen (a language model text encoder) to randomly generate music tags/captions without requiring LAMA, ...

I understand this kind of product may not be exactly right for this subreddit, but I still think it deserves some attention, as it's an LLM frontend for local engines in the first place, and can be us

Going to try these out to see if they have the same looping behaviors that other Bonsai models have had. Still requires their fork of llama.cpp (branch prism) so dont expect it to work with vanilla ll

Just downloaded the new LFm2.5-24B-A2B LiquidAI model for testing. They claimed some pretty good performance, and I've had good experiences with their prior releases. Some context - I use Ornith 1.5-3

This continues the small-model browser experiments from the previous posts. Two standard, off-the-shelf models ran locally on the same Samsung Galaxy S21: Gemma 4 E2B, Q3_K_M as the reasoner Qwen3.5-0

I built BitRun because I wanted to run 1-bit ternary LLMs on ordinary laptops without melting my battery, spinning up a dedicated GPU, or wrestling with complex build environments. Right now, BitRun g

Follow-up to last week's post about streaming MoE experts from disk in a browser tab. That ran on hand-written engines for two models. This week the same disk tier sits under llama.cpp's own WebGPU ba