Search: “models”

As you know, Qwen3.8 27B has built-in support for embeddings. It gives pretty decent results in llama.cpp with the following parameters: --embedding --pooling mean The problem is that enabling embeddi

I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison. All intelligence and price per task are sam

I have created a VAE model using PyTorch on White Wine dataset. Basically, the main goal is to discover a brand-new white wine recipe. It puts all the wines into a latent space, finds the best part wh

I know Sage and Triton are nothing new but if you're a beginner or doing a fresh re-install like me, these resources helped me get faster gens on the 4-step turbo workflow I use (will post a link to t

One is from Sam Altman, the other Claude suggested and it works extremely well. The first question I ask: What are you least confident about right now. The AI will list like 6 to 7 things that it didn

Using the RTX3090 on a linux machine I built for it and running Qwen 3.8 27B getting average 100t/s compared to my MacBook 20t/s I think I can finally get rid of my Claude subscription, this is good e

This is an evolution on top of Raymond's KV cache streaming fork - all credits to what enabled this goes to him. The basic idea behind what he enabled was a pool of memory in VRAM that is used differe

I'm documenting a reproducible setup for running current llama.cpp on a mixed AMD Vega system with both: Radeon RX Vega 64 / Vega10 / gfx900 (8GB HBM2) Radeon Instinct MI50 (Vega20) / gfx906 (16GB HBM

For many, Vram is precious, I see many people recommend using --no-mmproj-offload to save gpu vram but it is painfully slow. Especially if you are using it with agentic coding. If possible, add that s

I recently moved to Japan and ran into an issue where a lot of anime on Netflix only has Japanese subtitles here. I wanted something similar to the subtitle translation extensions that already exist,

Planning to build a PC mainly for local LLMs/coding agents. I keep seeing 3090 + Qwen 3.8 Flash Next benchmarks, but could not find enough info for the 7900 XTX 24GB. 3090s are hard to find where I li

Hi! I've been following this community for quite a while and have difficulty figuring out what to put on my 3080 12gb - I know Qwen 3.6 35B 3A was the go-to choice when it first came out, but I'm curi

I am just sharing my config for Qwen 3.8 27b that fits on a 5060TI, what is cool about this is that you can even get vision! and a 85K context (I have 1.5gb of headroom for more context or a better qu

Long time lurker, but I'm finally upgrading to 128GB VRAM, and I'm trying to figure out what to run. I had been leaning towards Qwen3.8 Flash-Next at ~Q4, and I generally prefer to not run anything be

We’ve been building PipesHub for a while now, and I’d love to get more teams to try it and tell us where it breaks. The problem we kept running into was pretty simple: Building an AI app over company

Ok I know I'm likely to get steamrolled here but I also have a sense of optimism and hope I can get some sort of answer that will be beneficial. We all know since the new price structure, usage has sh

I've built and open-sourced ParaRNN, a PyTorch/Triton library that parallelizes the training of non-linear recurrent architectures across time using Newton-Raphson iterations mapped to associative pre

Long time user of 3.6 27b, switched over to Next Flash since it's a logical step up even from 3.8 27b. It's soooo verbose, i'm talking 13 minutes of thinking time on single turn coding requests at app

https://preview.redd.it/3p6234jzk2oh1.png?width=900&format=png&auto=webp&s=b9eab3351d6c6dbd4d5b3fd677a6c40b57f18167 https://huggingface.co/tencent/EVIE-8B 🌟 Highlights SOTA Retrieval Performance: 66.7

TL;DR: switching KV cache to f16 may give a boost in speed if using MTP and ngrams. I have a self-built 'AI mega-cluster' with 2x P40s on a cheap Chinese motherboard and a Xeon CPU (around $1,100 to b

I'm one of the developers. We said in August it would go open source in September and it did last night. MIT or Apache-2.0, pick one. The repo you see is our internal repo, kernels included, so from n

I've tried both and been having this debate with myself for the last few days, on two Asus Ascent GX10s (effectively the same as 2x DGX Spark): DeepSeek-V4-Flash-0731 (official weights) GLM-5.3-Flash

If you are like me and use Google flow for NB2/NBP there is hope to finally ditch it. Let me start off by saying I’m not very impressed with any of the current local image edit or Image Reference mode

I’ve been running qwen on my personal Mac but I’m getting to the point where’d I’d like to have something always on, running various jobs, and some more ability to experiment and earn about fine tunin

ik_llama.cpp merged qwen4exp MTP support yesterday (PR #2369, mine, reviewed and tested by four other people on their own hardware). It's on main now, no fork or patch needed. Posting since the last c

https://preview.redd.it/74bmvel9b5nh1.png?width=1602&format=png&auto=webp&s=0d0c1adaa016a486ffd97c4c466e980dc611b139 I've only recently started looking deeper into vLLM after running llama.cpp for a g

Extended reasoning and post-training appear to be the keys used by DeepSeek, Qwen, and GLM to boost performance (leveraging higher token counts). And Qwen 4 hasn't even been released yet. Of course, w

Years ago I built a server, shelved it after some time, but recently realized I have it sitting there. I'm looking to pop in a Tesla V100 32GB card (with proper cooling). But are the component too old

Stumbled around a post about optimizing new Qwen up to 178t/s with a patched version of sglang : https://github.com/jpezzulli/sglang-rtxpro6000 I managed to reproduce results (kudos to jpezzulli, whom

Interesting new paper from Alexia Jolicoeur-Martineau (of Tiny Recursive Model fame) and collaborators. They seem to be able to replace quadratic attention with sliding window attention + attention si

I see this kind of approach quite often with local AI models. For example, when optimizing models for an RTX 3090, people sometimes convert them from FP8 to INT8 so they can run efficiently on Ampere

Setup: MacBook Pro M5 Max, 128 GB unified, macOS 26.5.2 · llama.cpp b10686 (Metal, 12 threads, batch 2048, flash-attn, kv-unified, ngram-mod spec decode) · Qwen3.8-Flash-Next UD-Q2_K_XL (Unsloth), 78.

Hey guys, FastVideo team here. We saw how important speed and quality is for everyone. And we've been working to create our own step distill checkpoints and LORAs for MINIMAX h3. Here's is our v1 rele

The initial reason for this was the Comfy H3 Sync & Sound Community Challenge: Comfy H3 Sync Sound Community Challenge! - by Allyson Toy I made a short rap track in Suno, then used Hermes Agent to bui

Using https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 and https://old.reddit.com/r/BlackwellPerformance/comments/1w04xb7/qwen38_flashnext_on_1x_rtx_pro_6000_171_ts_c1_428/ As usual, all the d

I wanted to see whether NInfer could run Qwen3.8-27B on an RTX PRO 4000 Blackwell 24GB. Hardware/software: NVIDIA RTX PRO 4000 Blackwell, 24GB 145W power limit CUDA 13.3 NVIDIA driver 610.x Ubuntu LXC

I've got Qwen3.8-Flash-next running on RTX 3090, Ryzen 9 3950X, a PCIe 3.0 motherboard, and 64GB DDR RAM from 2020. IQ4_XS weights, full kvarn5 context, vision on GPU, experts in host RAM, n-grams on

After getting tired refreshing and searching sub reddits for qwen 3.8 35 a3b i decided to give qwen 3.8 flash next a try . So I dual booted and built llama cpp I am able to hit 6-7 tps constantly insi

I am sure for those of you who have been following my 4 step Krea 2 Turbo LoRA, you would know from my previous posts the work of progress I have been sharing with you ( if not see here - https://www.

specs hardware: M4 Max 128GB Studio inference engine: oMLX & lllama.cpp insights it still very early, so had to disable oMLX K/V caching, qwen4_exp architectureis not yet supported + the obvious n-gra

Hi, I'm pretty sure I'm not the only one in this scenario, I have the Max Plan but I'm downgrading it to Pro and will buy GPT Plus because it's too much usage for my use cases. So either I waste money

Recently, I've been building a project where sandboxing and isolation of certain features was required. However, for the life of me, I couldn't ask Fable 5 a simple question without it accusing me of

Upgraded to a 5070 Ti so I could run Qwen 3.8 27B, which works perfectly, but didn't want to let the old 4070 Ti go to waste. The cards would touch if I put them both in the PC and I knew the heat wou

*part 2 of an earlier post: previous quant comparison with voxel island creation this time I rented three rtx pro 6000 96gb, on each one I launched a qwen 3.8 27b quant and gave them 4 identical promp

The Artist: Qwen3.8-27B-UD-Q3K_XL, q8_0 caches, xhigh, temp 1.0, image-min-tokens 1024, froggeric template) I was screwing around with different Qwen3.8-27B quants and thought of this very simplistic

So I've been going down a rabbit hole trying to run local LLMs without dropping 800+ on a Mac or GPU. RAM prices are insane right now and I realized that used phones have decent unified memory for dir