b10219
cli : persist reasoning_content in chat history (#26362) cli : persist reasoning_content in chat history llama-cli collected reasoning from the stream for display but only stored assistant content in
Knowledge catalogue
cli : persist reasoning_content in chat history (#26362) cli : persist reasoning_content in chat history llama-cli collected reasoning from the stream for display but only stored assistant content in
ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts (#25956) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check and some comments Fix build error caus
ggml-zendnn : group matmul direct API for mul_mat_id (#25918) ggml-zendnn : group matmul API for mul_mat_id ggml-zendnn : scale MUL_MAT_ID fallback threshold by expert count Website: https://llama.app
llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized (#25871) llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized
SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc… (#25025) SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt processing fattn-mkl: fix interleaved dst layout in nor
server: correct accepted tokens when need draft token replay (#26320) spec: correct accepted tokens when need draft token replay cont : naming Co-authored-by: Georgi Gerganov ggerganov@gmail.com Websi
llama : load MTP tensors only if they are really used (#26296) llama : load MTP tensors only if they are really used llama : skip loading MTP (if not used) in remaining models that support MTP Co-auth
vulkan: Introduce driver version check for Windows Intel GPU to mitigate crashing (#25192) Removed crash guard for Intel Crash fixed from driver 32.0.101.8860 Added driver version check for windows Ch
vulkan: add POOL_1D op (#25431) vulkan : add pool1d push constants and pipeline field Declared data structures needed for POOL1D OP, which are the vk_op_pool1d_push_constants struct and pipeline_pool1
DeepSeek V4 Flash: Preview → 2026-07-31 Benchmark Preview 0731 Δ Terminal Bench* 56.9 82.7 +25.8 Toolathlon 51.8 70.3 +18.5 NL2Repo — 54.2 new Cybergym — 76.7 new DeepSWE — 54.4 new Agent Last Exam —
metal: fix memory unwire if model is freed without any GPU operations (#26082) metal: fix memory leak if model is freed without any GPU operations metal: run dummy work only if residency sets are used
ggml-cuda: Allow transpose-free gemmv computation (#26171) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. Website: https://llama.app macOS/iOS: mac
Test support for alternative conv layout (#25617) add bool cwhn = true to conv_2d test cases add layout check at graph building time extend layout checks for conv2d.cu kernel in CPU back-end kernel ne
I've always found it a little strange that AI is everywhere, but the codecs we use in practice are the traditional hand-engineered systems like h.264, h.265, av1. Alexnet started the wave of neural ne
model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) (#25980) model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) Adds GLM-5.2 NextN/MTP as a --spec-type draft-m
server : add trace logging for slot similarity checking (#26271) Adds trace logging in server-context.cpp for slot similarity checking during prompt cache slot selection, including skip reasons and si
sycl: contiguous fast path + 32-bit index math for unary elementwise ops (#25946) sycl: contiguous fast path + 32-bit index math for unary elementwise ops sycl: use fastdiv for elementwise index math
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (#26141) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed an
llama: move suppress_tokens handling to common/sampling (#26276) llama: move suppress_tokens handling to common/sampling address security issues rm has_logit_bias Website: https://llama.app macOS/iOS:
arXiv:2505.02271v2 Announce Type: replace Abstract: The proliferation of Generative Artificial Ingelligence (AI), especially Large Language Models, presents transformative opportunities for urban appl
ggml-metal: FWHT kernel for metal backend (#25924) metal fwht wip shape guard and formatting formatting Formatting and typos Co-authored-by: YiChen Lv 63285796+forforever73@users.noreply.github.com fi
ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration (#22675) ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC.
vulkan: add iq4_nl support back to FA (#24585) vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practic
ggml : set output of view src (#25729) llama-graph: set_outputs to t->view_src change set_output to GGML_ASSERT about views not being outputs sampler : avoid views in outputs cont : fix dist sampler c
opencl: skip the Adreno KQ/KQV image kernels for multi-stream batches (#26189) The Adreno KQ/KQV image1d kernels (ggml_cl_mul_mat_kq_kqv_adreno) ignore dim 3 entirely: the sub-buffer covers only nb02*
Hugging Face is being used to make nonconsensual deepfakes, and the popular open-source AI model repository is doing very little to prevent it. That's according to a new report published by the Europe
mtmd: Add Vision Support for Minimax-M3 (#25113) Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, Deep
server + ui: fix stream routes for model names containing a slash (#26137) server + ui: refactor resumable stream routes to query string conv_id The conversation id can embed a model name containing s
args: add -lm mlock where it mlocks but doesnt mmap (#26135) arg: add -lm mlock where it mlocks but doesnt mmap Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: rm unwanted docs changes Signed-off-by
common: fix explicit -md precedence over draft sidecar resolution (#26165) common: fix explicit -md precedence over draft sidecar resolution Follow-up of #25955, an explicit --model-draft file given w
model: Add support for Nanbeige4.2 (#25994) support nanbeige4.2 model fix fix flake8 Lint check fix loop bound check and drop redundant head_dim Co-authored-by: root lizongqiang@kanzhun.com Website: h
I got tired of my Ollama server sitting idle between chat experiments, so I pointed it at my Navidrome library and made it run a radio station. The DJ is an agent, not a shuffler. Each turn it gets to
args: refactor mlock/mmap/directio into load-mode (#20834) args: overhaul mmap/mlock/dio into single arg Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: update docs with llama-gen-docs Signed-off-by
https://reddit.com/link/1v5rvuq/video/bgmwc754i9fh1/player My goal was to create a benchmark to measure the spatial awareness and memory of models. Eventually, I came up with the simple idea of a maze
ggml: enable PowerPC backend variants on AIX (#25983) ggml: enable PowerPC backend variants on AIX Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMak
Fix DeepSeek4 crafted template (#25414) chat: fix DS4 template to explicitly follow reference behavior Support DeepSeekv4 flag (drop_reasoning). fix: hook DS3.2 parser for DS4 as well fix: add tool re
common: infer the speculative type from the draft repo sidecars (#25989) With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars and no --spec-type given, the draft resolved to a full mo
hexagon: activation ops update (#25974) hex-geglu: optimized all-in-one geglu microkernel hex-geglu: enable non-contiguous src and strided DMA hex-act: enable non-contiguous srs and strided DMA for re
CUDA: Improve NVFP4 W4A4 activation quantization (#25730) Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic w
We built this experience based on feedback from early testers and physicians. With your permission, ChatGPT can use relevant context you’ve connected in Health across your conversations — to help you
mtmd : use align_corners for qwen3vl vision position embedding interpolation (#25781) The Qwen3-VL learned position embedding is interpolated to the runtime patch grid with the default bilinear+antial
cuda: GET_ROWS quants (#25962) cuda: add k-quant support to GET_ROWS Device-side embedding lookups require GET_ROWS to handle the k-quants used by common GGUF recipes (Q4_K_M stores token_embd as q6_K
webgpu : add CONV_2D_DW (depthwise conv2d) kernel (#25847) webgpu : add CONV_2D_DW (depthwise conv2d) kernel Implement GGML_OP_CONV_2D_DW for the WebGPU backend, ported from the Vulkan backend's conv2
Most AI SDKs are a wrapper around one engine on one platform. We built the opposite. 7 SDKs. Swift, Kotlin, Flutter, React Native, Web, Electron, rcli. Python. All of them are thin skins over one C++
I've been considering Ollama's Cloud Max plan to replace my GLM Max plan but couldn't find good documentation on how their limits actually translate into GLM 5.2 usage. I know it's by GPU time but tha
Automated lint: 20 errors, 8743 warnings, 3 info
opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU (#25745) opencl: workaround for A850 compiler compat opencl: fix DX compiler version parsing and cleanup Co-
[SYCL] Flash Attention with XMX engine via oneDNN (#25222) [SYCL] F16 (default) Flash Attention with XMX engine via oneDNN graph API; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at p=80
sycl: Increase minimum buffer size for USM system allocations (#25525) Raise the threshold for minimum buffer size from 1 GiB to 4 GiB, based on real-world experiments of overcommitting device memory
server : refactor prompt cache state ownership (#25649) server : clear checkpoints upon prompt clear server : move the prompt state data to the server_prompt_cache Assisted-by: pi:llama.cpp/Qwen3.6-27
arXiv:2607.06598v1 Announce Type: cross Abstract: Heart rate measurement is one of the key requirements for real-time health monitoring, in particular for health caring of elderly people. Traditional
arXiv:2607.02680v1 Announce Type: cross Abstract: MLLMs have shown strong zero-shot capabilities across diverse inputs such as across images, video, audio, and text. A crucial, yet underexplored, appl
The July 4th weekend All-In @theallinpod turned into a long argument about who owns the intelligence layer. The besties think enterprises just woke up to a trap they had been walking into, here's how
arXiv:2606.28374v1 Announce Type: new Abstract: LLM agents are increasingly improved without weight updates by evolving a natural-language artifact, such as reflections, workflows, playbooks, cheatshe
arXiv:2606.25476v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes appl
arXiv:2602.17431v2 Announce Type: replace-cross Abstract: Uncertainty quantification has emerged as an effective approach to closed-book hallucination detection for LLMs, but existing methods are larg
arXiv:2606.10170v1 Announce Type: new Abstract: This paper extends the concept of Learning Entropy (LE) from temporal adaptive systems to spatial learning in multilayer perceptron networks (MLPs) appl
arXiv:2606.07834v1 Announce Type: cross Abstract: LLM judges increasingly turn verdicts into system commitments. Under mixed evidence (claims with both supporting and refuting sources) this is unsafe:
longmemeval experiment arch: 1) deterministic ingestion/extraction (85.6% accuracy, 86.2% retrieval) 2) semantic ingestion/extraction (84.8% accuracy, 94.9% retention) 3) semantic ingestion/determinis
Aleph 2.0 Introducing Runway Aleph 2.0. Edit videos using AI, keeping camera and motion all consistent. Select specific frames in your video to reprompt with GPT Images 2.0 or Nano Banana 2, then appl