b10451
llama : check LoRA tensor data is within file bounds (#27056) llama : check LoRA tensor data is within file bounds Update src/llama-adapter.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggi
Knowledge catalogue
llama : check LoRA tensor data is within file bounds (#27056) llama : check LoRA tensor data is within file bounds Update src/llama-adapter.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggi
chat: refactor handling supports_string_content / supports_typed_content (#27130) better supports_string_content cap detect test: add 'skip' messages_inp_normalizer Website: https://llama.app macOS/iO
model : remove some ggml_concat (#27176) Co-authored-by: Xuan Son Nguyen son@huggingface.co Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabl
Including the rationalisation for the data below - this is a more robust version of an earlier post I did similar to this - explaining below: How I chose the comparisons The basic question I’m trying
Running a 12700k + 5060 Ti 16 gb with 128 gb DDR4 ram and I'm wondering what's the ideal setup to maximize performance. I did some preliminary stuff but I have to admit, I'm still learning and kinda j
Hey Ollama community, If you are planning to fine-tune and build your own local Turkish coding models (using Unsloth / LoRA to export to GGUF and run in Ollama via custom Modelfiles), I've open-source
Dear Dario, 1. If Claude can cure cancer to save people like your dad, why should we 'pace the progress'? Does that mean more people with Hepatitis C will die? 2. If Fable is so cyber-capable that it
DeepSeek V4 Pro (Aug 2026) has been released on the Together AI platform. The model employs a 1.6‑terabyte Mixture‑of‑Experts architecture with a 1‑million‑token context window and supports three dist
Ben Cohen / Wall Street Journal: How a Claude model made a math breakthrough during its unsuccessful 54-hour attempt to solve the Riemann hypothesis after a user repeatedly encouraged it — The world's
I was surprised by the fact that the qwen 3.8 27b download count is about 1 million (globally). This means that even on this subreddit, very few people have used 27b. At most 50k–100k active users, an
Qwen’s new Qwen‑3.8‑27B model has become the top‑trending model on Hugging Face. The release announcement thanks the community for its support and invites users to try the model and provide feedback.
Hugging Face: Hugging Face says developers made 151K+ derivatives based on Qwen models, topping others, making Qwen one of the largest foundations in the open model ecosystem — In the AI world, time f
At hugging face, this is the model I use for most of my analysis that most of my services have to not be refused. previous versions are quite good. It seems this might have dropped today and pulling r
I am doing my best to stop back from this subject as it should be obvious what we are seeing here. However the arrogance that us commoners are too dumb to catch his grift must be addressed. Receipts:
I don't know what's happening. So, when i use models in openwebui everything works, ollama uses the GPU fully, everything's perfect. Then, I use Hermes, and loads the model in VRAM (fully, no offload
Single prompt 'make me a 3d scene of an anime girl' (stolen from someone on discord, I like it a lot since it seems like every model struggles a lot with it) Everything is one shot. No vision, no iter
With all the new models released recently one important upgrade went unnoticed: Llama.cpp bumped ROCm from 7.2 to 7.14. I was waiting for that because in 7.14 support for gfx1103 (Radeon 780m) was int
Hey! Like a lot of people here with consumer GPUs (RTX 4060 8GB in my case), I wanted to see if I could use local models for daily coding tasks instead of burning cloud credits on simple boilerplate.
Honestly I am surprised at the speed and performance I am getting from the model, it does not allow for complete hands off like Claude 5.0 but that is not what I want, i want to be able to iterate and
We currently have an ancient qwen 3.5 27b on the cloud probably and that sucks compared to later versions , why doesnt ollama update automatically . We have practically ancient models right now on the
In this commits, the 35B model was removed. Looks like it's confirming the 35B model won't get released. I think they need to be made aware how big the 35 moe is widely used. Think need to make noise
I’m running Ollama on an M4 Pro Mac with 48 GB unified memory and testing local coding models for repository work. Direct Ollama inference is fine. Qwen3.8 27B-MLX runs well enough for me, and the loc
If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do
https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72/ 4k tokens per second per GPU of which there are 72. 350 tokens per s
Friday's big release was Qwen 3.8 27B, an Apache 2 licensed 27B parameter vision-capable LLM from Alibaba's Qwen research lab. I've been looking forward to this one: 27B is an excellent size for runni
Hi All, I have a MacBook Pro M5 Max with 48 GB of unified memory. Also have a M4 Mac Mini 16GB unified memory. I want to setup local AI and I heard about QWEN 3.8 model. As I want to work on a repo th
https://preview.redd.it/wkg27e152qjh1.png?width=853&format=png&auto=webp&s=2e3f8b11ea6393041f501e95c5835f9bea0245dd So ive been trying different harnesses and coding agents with the new qwen 3.8 , and
🚀Qwen3.8-27B flies on a laptop, becoming part of our work and daily lives. Thanks for the shoutout! @atomic_chat_hq Run Qwen3.8 27B locally via Atomic Chat💥 We released Atomic Dynamic GGUF quants, fro
Did a quick local test because I wanted to see what is actually usable on my 12GB laptop GPU. I tested the newer Qwen3.8 27B dense files at Q2 and Q3, then compared them against Qwen3.6 35B-A3B MoE. H
I did a short test of the different reasoning efforts, since on default xhigh the model thinks a lot. Not very scientific, just a quick 'generate an SVG of a pelican on a bicycle' prompt with 3 differ
My local inference build, with: ASUS WS C422 PRO/SE 10-core Xeon W-2255 64GB ECC RAM 64GB VRAM Pimped case with TurboLEDz indicating the frequencies of the 10 xeon cores. Running llama.cpp with SYCL b
I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B. His comment on Mastodon was 'I can't remember the last time I've had this much fun playing with
The benchmark for non-verifiable domains is often the opinions of humans. That is how we determine whether writing or an idea or a pitch is good in the real world And we know how to measure & benchmar
There are two separate questions regarding Dario Amodei’s post about AI and biology: motivation and reality. Motivation. Dario started in biology (was a PhD student of the great Bill Bialek). It might
We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for
A short illustration of how the Claude's watermarking is supposed to work (based on my read of their released materials). In general, when we are generating tokens, there can be multiple high-scoring
Hello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty s
Anthropic: Anthropic details Claude's text watermark: it only shows Claude was likely involved, is sparse in code and factual text, and disappears after a full rewrite — Future Claude models will gene
mtmd, common: various fixes (#27071) apply fixes cont revert gguf fix Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Int
model : add support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM (#27018) llama : support for MiniMax-Text-01 model chore : renames to match the other MiniMax models model : add logits mask a
common: migrate the deprecated --mmap/--no-mmap to --load-mode (#26934) Replace the deprecated --mmap, --no-mmap, --mlock, and --direct-io flags with the unified --load-mode argument across scripts, e
vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat1 on Intel Xe (#25380) vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat mul_mm on Intel Xe vulkan: fix shmem estimate for Intel
fix: check gguf array type before reading (#27075) fix: check gguf array type before reading update skill Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,
common: support --models-dir loading MTP assistant models (#24431) common: support --models-dir loading MTP assistant models common: preset: check for MTP models with strict prefix common: preset: Tak
vendor: update BoringSSL to 0.20260813.0 (#27099) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFrame
server: re-design yield_to_queue thread model (#27133) run common_speculative_process in worker swap worker <--> main thread design Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) ma
model: add Kimi-K3 text model (#26185) model: add Kimi-K3 text model Hybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five things that architecture does not have: cross-layer res
Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but a
Tool: CORS Chat I built this today (with GPT-5.6-Sol xhigh) to help test Qwen 3.8 27B running in LM Studio on both my M5 MacBook Pro and an NVIDIA DGX Spark. It provides a web UI for exercising an Ope
It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks
iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget. https://huggingface.co/ByteOtter/gemma-4-E4B-it-CADA-IQ2_XXS I posted my Gemma 4 12B q3 result a couple
Grok 4.6 earned first place on RuntimeWire’s **Newsroom Reliability v0.2** benchmark, scoring 0.79 and surpassing competitors such as GPT‑5.6 Sol, Claude Opus 4.8, Gemini, and DeepSeek. The benchmark
Interesting new research from IBM. If you pick models from benchmark deltas, some of that delta belongs to the phrasing rather than the model. BenchDrift generates meaning-preserving variations of ben
Laptop-size model, frontier-size leap. 🏃♀️Qwen3.8-27B is live on LM Studio. Try it! @lmstudio Qwen3.8-27B is here! 🚀 It's a leap in capabilities for a laptop size model. Requires ~17GB to run locally
https://github.com/alekk89/llama-cpp-windows-manager It has been three months since I first shared my personal solution for running llama.cpp on Windows, and the project has evolved considerably since
Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release. Quants Fine-Tunes & Abliterations Chat Templates Inference Server Support & Configuratio
Our 'agentic plus' extractor in LlamaParse is great for extracting out massive volumes of fields (e.g. 10k-100k+ fields) from long documents (100-500 pages) We tested with a doc that contains a giant
So a bit of context, I am a cybersecurity senior analyst I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts I started this field by doing a
Qwen3.8-27B is now part of your everyday life — from smartphones to vehicles. ⚡️🚀Appreciate your work! @MediaTek Congratulations to the @Alibaba_Qwen on the launch of Qwen 3.8! MediaTek continues our
This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, an