Search: “model-releases”

sycl: split long rows in TOP_K instead of one work-group per row (#27847) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43958406 macOS/iOS: macOS Apple Si

hexagon: support for device discovery and create sessions on demand (#27785) hex-devices: add support for lazy session allocation and cleanup dev interfaces Co-authored-by: Marco Colombo mcolombo@qti.

rpc : fix pre-rdma macOS versions (#27815) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43962220 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

vendor : update cpp-httplib to 0.54.0 (#27919) vendor : update cpp-httplib to 0.54.0 vendor : update cpp-httplib to 0.54.0 and 0.54.1 Website: https://llama.app Attestations: https://github.com/ggml-o

metal : Add fa-vec tuning for M3 Pro (#27963) Related issue: #27668 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43966636 macOS/iOS: macOS Apple Silicon

metal: add fa-vec tunings for M3 Ultra (#27999) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43968762 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple

rpc: fix apple rdma error spew on teardown (#27908) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43970664 macOS/iOS: macOS Apple Silicon (arm64) macOS Ap

ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960) some backends (Metal, SYCL, WebGPU) require additional memory for fleeting data for certain ops, which is reflected in their ge

common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969) Rename the --tensor-read-lazy CLI argument to --lazy-mode, to match the internal lazy_mode parameter, and add a -lzm short

dflash: pass missing NVFP4 scales to attention operations (#28000) DFlash2 NVFP4 draft models produced almost no accepted speculative tokens because the Q, K, V, and output projection scales were not

hip : optimize Q2_0 dot-product path for gfx1201 (#26753) hip/gfx1201: optimize q2_0 vec_dot_q2_0_q8_1 with native amdgcn perm Broadened HIP's Q2_0 perm optimization Remove redundant HIP perm availabi

hip: tune rdna 3 mmq config (#26284) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44012223 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (ar

CUDA: use the fast mm_ids_helper path for any n_expert_used (#27978) The optimized path grouped warp lanes by token and required warp_size % n_expert_used == 0, with a single hardcoded exception paddi

llama: improve TENSOR_READ_LAZY handling (#27837) force lazy tensor on cpu if lazy is on llama: improve TENSOR_READ_LAZY handling Website: https://llama.app Attestations: https://github.com/ggml-org/l

When using two GPU with llama.cpp with --split-mode layer you usually provide a --tensor-split value to fit the proportion of the allocation on each GPU, say you have 16GB + 12GB something like --tens

Leo @Synthwavedd 🚨 SCOOP: Astra recently graduated from the dogfood stage, and is now being made available to select OpenAI partners under the name 'ultima-alpha' If feedback over this weekend is posi

I'm running Muse Glimmer 30B EXL3-SC 3.00bpw H4, fully resident on my 12GB VRAM GPU at 100K context with Q8_O KV cache. It's a joy to use a dense 30B model at this size and still get 30 tok/s on a VRA

(not written by Claude, all errors and crappy text are result of too little coffee on a Sunday morning ;) Our home server is a 2018 Thinkstation P520, bought for about 600eur in 2023. It's been upgrad

Hey guys I have been working on a open source LLM development repo. Anyone can train any amount of parameter against any arch (GQA, MLA, Dense...) When I tried to implement MLA I read the technical re

Since I spent the time to figure it out and it is not like it will make me any money ever. I think I'd share with you all what I managed to cobble together. First and foremost here the relevant bits a

The intelligence in my context is an aggregate index, I called the Agentic Coding Index, across most relevant agentic coding benchmarks: SWE-bench Pro, DeepSWE v1.1, Terminal-Bench (v4, v3, v2.1), Cod

I DON’T THINK PEOPLE REALIZE HOW MASSIVE THIS IS YET @HuggingFace and @PollenRobotics just released Microduck. It is a new legged robot for 399. It passed 1 million in sales in under SEVEN hours 🤯 The

I built SpeakoFlow Mini, an Apache-2.0 fine-tune of Qwen3.5-0.8B for dictation cleanup. It is not a chat model or a general rewriter. It takes speech-to-text output, applies corrections the speaker ac

LLMs will always be bulls in a China shop. Nailed it, December 2022. We are still suffering the consequences. And they are about to get so much worse. No wonder Silicon Valley tried for so long to dep

I am doing a lot of translation work with different languages, and German is just exceptionally well done by Qwen 3.8 27B. It is lengths ahead of GPT-5.6 and even Fable 5. With the frontier models I o

Running on a M1 Max 64 GB: - SSD streaming for tensors - SSD streaming for engrams - SSD streaming for MTP How is it possible? * Custom Q4 quant: benchmarked all metal tensors then picked and spliced

Setup: MacBook Pro M5 Max, 128 GB unified, macOS 26.5.2 · llama.cpp b10686 (Metal, 12 threads, batch 2048, flash-attn, kv-unified, ngram-mod spec decode) · Qwen3.8-Flash-Next UD-Q2_K_XL (Unsloth), 78.

Been working really hard for the past month to bring to the community all these models, the hardest was for sure LongCat-Flash-Lite-Sparse who required TONS of work, first I needed to have Heretic sup

OpenAI announced ChatGPT Work on July 9th, and have been furiously iterating on it ever since. It is an extraordinarily confusing and very powerful product. Here's what I've figured out about it so fa

I find both Qwen 3.8 27b and Qwen 3.8 Flash Next difficult to read. Here's some examples of what I mean: **Model-visible tool set per turn** (assembled by the host at provider-request time): persona t

We used HFlow to evaluate the latest open weights VLMs for processing egocentric data. This was based on Build AI's Egocentric-10k evaluation, which used Gemini 2.5 Flash to measure hand visibility an

I have been using the unsloth/Qwen3.8-27B-NVFP4 with 157k ctx on vllm currently. But recently I have been seeing post about ninfer lately and how it is like the best way to run qwen 3.8 on RTX 5090 32

Hey all, You’ve probably noticed that using LLMs and AI agents for pentesting has become pretty common lately. The problem is there isn’t really a good way to figure out which model is actually best s

// Your LLM judge disagrees with the experts // LLM Judges can be tricky to build. Here is an interesting showcasing why: There propose a reference-full benchmark of hundreds of complete human-to-huma

I got a 50% performance boost (20 t/s -> 30 t/s) in llama.cpp for MoE models that don’t fit entirely in VRAM—in my case, Qwen 3.8 Flash Next. The idea is simple: instead of offloading entire layers to

Anthropic recommends that users clear or update their **CLAUDE.md** file after each new model release, ensuring the documentation reflects the latest changes. The message includes an informal comment

@claudedevs: Anthropic will “permanently” raise weekly Claude Code limits by 25% on Sept. 14 for most plans, which will work out a 17% reduction, given the current 50% boost — Starting September 14, w

You guys remember sesame AI? With their really good AI voice model? Obviously ChatGPT has their voice model that’s also really good. Is there any smaller local variant that runs on like consumer grade

specs hardware: M4 Max 128GB Studio inference engine: llama.cpp (qwen4exp branch) judge: claude-opus-4-6 AtomicChat/Qwen3.8-Flash-Next-GGUF Qwen3.8-Flash-Next is a great model I benched in my previous

vulkan: Change mul_mat_id to pad K rather than N (#27925) The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id, we indirect the row index through a shared memory lookup table which

metal : add fa-vec tunings for M1 Max (#27932) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43859747 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S

vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526) vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs remove one more fastdiv W

sycl: make --fit respect --fit-target better (#27629) improve the --fit algorithm to take into account the actual peak required VRAM for a given context size on a SYCL backend. This includes both prop

metal : add remaining fa-vec tunings for M4 Pro (#27915) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43865923 macOS/iOS: macOS Apple Silicon (arm64) mac

metal : assert shared memory padding (#27951) metal : assert shared memory padding cont : add ref Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43867822 m

opencl: use a better matmul path on two Adreno GPU generations (#27640) opencl: default the Adreno xmem F16xF32 GEMM on for X2E kernel_mul_mm_f16_f32_l4_lm is the slowest matmul this backend has on Ad