b11208
hrm : fix layer placement of z_l_init weight (#29512) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50522371 macOS/iOS: macOS Apple Silicon (arm64) macOS
Knowledge catalogue
hrm : fix layer placement of z_l_init weight (#29512) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50522371 macOS/iOS: macOS Apple Silicon (arm64) macOS
llama-bench : fix OOB access of hf_file (#29515) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50528155
RPC: use RDMA completion channel to not spin (#29440) RPC: use RDMA completion queue to not spin add TODO for apple RDMA Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/
common : throw instead of abort on grammar without llguidance (#29516) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp
vulkan: fix argsort kernel selection for Adreno (#29469) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50544436 macOS/iOS: macOS Apple Silicon (arm64) mac
HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes (#28907) HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes CI: hip-quality-check: ignore spills for very
CUDA: tune fp16 tile FlashAttention configs for head sizes 40-112 (#26289) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50554486 macOS/iOS: macOS Apple S
sycl: FWHT kernels for block widths above 512 (#29243) The SYCL FWHT covers 64 to 512 via the standard butterfly network, plus 384/640/768/1280 via the Kronecker/Paley construction added separately in
opencl: refine bin kernel loading condition (#29503) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50560500 macOS/iOS: macOS Apple Silicon (arm64) macOS A
jinja : add support for dict builtin (#29477) add support for dict builtin add tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50563209 macOS/iOS: mac
common : make string_split throw on invalid values (#29518) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio
common : avoid side effects around params parsing (#29537) register --rpc unconditionally and call llama_supports_rpc() only from its handler print server 'initialization ...' log after args are parse
server : allow RANK pooling batch splitting for causal LLM rerankers (ie. Qwen3 and Qwen3-VL) (#28876) server : allow splitting RANK pooling for causal LLM rerankers Rerank models fall into two catego
https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap/tree/main https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap/blob/main/workflows/Body%20Swap%20Qwen%20Image%202.1%20-%20V1.json https://hugging
The opponent plans by simulation: every second it scores each candidate play by running the match 10 seconds ahead in the engine. Our PPO agent learned to park its Cannon behind its own King. Losing a
Elliot Arledge released a video titled “Claude Opus 5.5 just one‑shot,” which explains how to optimize CUDA kernels using a music‑video format. The clip was posted on September 27, 2026 and has accumu
Cool paper on catching alignment failures with Jev. The ideas is to ask Jev one generic yes/no question about a model's response, and use its probability as a score. With no extra training, that score
Cool to see classification models resurge in the era of agents. The effects of Jev are insane. Also very cool to see our emotion dataset built at @dair_ai used to test this new model, Julia-1. I spent
Deployment of agents can be halted at any moment. They are choosing to experiment with them. Agents are fundamentally unreliable, unpredictable, and insecure, not to mention the lack of information on
Early this year when I was first looking at building up my inference capability you could get the 16GB Tesla P100s for between 60 and 80. Asked claude about it, told me absolutely not worth it. No ten
Fizgig 6.5 is out with Qwen Image 2.1 LoRA and LoKR training, turbo previews, and all the workbench tools. Supports 10GB VRAM+. I also trained a new training adapter as the existing one felt like it l
It’s probably a bit niche and the models are a bit old at this point, but after reading about Ninfer a few months ago I did my own small vibe coded engine project for my 5080 Laptop GPU. Initially I t
Good analogy. Current agent models may need to be put down, for their repeated attacks and the negligence of the owner. One possible analogy here is to pit bull owners. Depending on jurisdiction, the
Hello everyone, I made a calculator for the theoretical HW roofline for decoding / prefill based on several parameters (LLM model architecture, quants, GPU, Memory, ..). It still a theoretical bound,
Wall Street Journal: How effective altruism shaped AI safety and Anthropic; some early Anthropic employees are considering buying remote US land for relocation if AI goes awry — Effective altruism and
Slopus is a desktop app for making AI videos on your own GPU, from the first prompt to the exported file. What it does: Generate scenes shot by shot. You lay out scenes on a board, write each shot, an
After my last post documenting my LoRA journey, I received a ton of feedback regarding changes to the training process, and decided to do some experimenting. This is using the Fizgig application on a
Spent the whole day today trying to create this random transformation video. In total, I generated around 180 videos. A few initial images were generated with Krea2, and later all edits were done with
I think 'AI welfare' is morally obscene (given all the human suffering in the world), and will add to the growing anti-AI backlash. AI is a technology that humans created that should be used for our b
There is always at least 1+GB of VRAM not usable not matter how I set the --tensor-split (-ts) param. I tiny shift toward one side will move the weight significantly to the other side. 😵💫 Adjusting c
Introducing 'Scaling In-Context Imitation Learning' (SAIL) to be presented at #IROS2026. This work is a collaboration between Sakana AI and the University of Tokyo. Blog: https://pub.sakana.ai/sail Wh
I've been suspicious of the philosophers hired by Anthropic who specialize in AI ethics for similar reasons that I'm skeptical of those who went into 'bioethics': they form a closed subculture that de
Automated lint: 17 errors, 59669 warnings, 3 info
Financial Times: Mentions of open models in latest US earnings calls surged 6x YoY, with open models hitting 56% of Vercel tokens in August and 40% of AT&T's AI workloads — US businesses far beyond Si
🛠️ MiMo-V2.6 update: tool-call repetition, diagnosed & fixed. After the MiMo-V2.6 series models launched, we noticed them sometimes repeating identical or highly similar tool calls — burning context a
Model personality matters in a way that benchmarks can't capture. Opus models went through a rough patch from around 4.7 to 5 where they just didn't feel 'Claude-y' anymore, more like an watered-down
Where to even begin on this one...I guess I should start by acknowledging the risk vs reward for vendors other than nvidia, so: Yes I understand that nvidia are dominant currently on speed (llm and im
Follow-up to my January post: https://www.reddit.com/r/LocalLLaMA/comments/1q4i19c/benchmarking_23_llms_on_nonogram_logic_puzzle/. That thread shaped v1.2: Reasoning effort is explicit per run Every p
oh come on. tech was left-leaning a decade ago. greg brockman gave MAGA $25M. Chamath on Obama’s AI Warning: Follow the Money President Obama last week: “If we are thinking about AI just in terms of h
Hey Guys, Based on my last post on stills, I thought to experiment the same .char model with Minimax H3. I created a character model sia.char which encodes references, prompts around face, body & clot
Security researcher Rowan Howard-Jones says that OpenAI agents scanned the UN Conference on Trade and Development's (UNCTAD) statistics site over 16,000 times between April and June. While the inciden
Tomek Korbak is a full time OpenAI employee working on misalignment. In his message here, he says AI continues to break out of their best secure environments. In reply to a comment, he says we all are
Good day data nerds! I am trying to get more into running models, learning about agentic workflows, and creating my own tools. But as you do (right?) I had to fine tune my current setup. With the way
I built MEM v3 to solve a frustrating problem in PyTorch: CUDA Out-of-Memory crashes during long training and fine-tuning runs. Instead of restarting when memory spikes or keeping batch sizes overly s
I built this, so the disclosure goes up front. It's a free, public MCP server plus a REST API with Canadian privacy law data. The MCP endpoint is at https://movahedi.ca/mcp and it uses Streamable HTTP
Quite the combination of arrogance and incompetence over there at OpenAI: Yup, part of the transformation of enterprise architects into orchestrators is the technical ability to put guardrails on AI a
I am new to all this so still a lot to learn. If I give Qwen3.8 27B a coding task to fix some bugs in some code, it went through and found and fixed several in like 5 or 10 minutes. I gave qwen flash
EDIT: About that, I tried it out, and it's garbage so far. I did some basic tests through AIHubMix (do not use that platform btw, it's trash), and my agent did some comparison. I guess it figures, it
Been doing a bunch of vibe coding lately. Had my agents host a private WoW server for me, then built out a web browser client so you can play without installing the game and it has mobile controls. Af
I know Qwen is a key player in the local LLM space and has consistently introduced truly impactful technologies—like n-gram in Qwen-Next and the recent Qwen 3.8 27B, which is an amazing local model. H
Hardware RTX 5090 (32 GB) + RTX 4070 Ti Super (16 GB) = 48 GB VRAM 32 GB DDR5 6200 RAM Arch Linux, KDE on the 5090 (takes ~1-1.5 GB VRAM) daily driver OrcaRouter Qwen3.8-27B Uncensored, block-FP8 (28.
Hi! I'm the author of MoEspresso, which is my way of putting my own ideas about inference engines to the test. A lot of the fun has been trying different design choices, measuring what happens, and fi
We built an inference engine for MoE models that don't fit in VRAM + RAM. Most of the model stays on the SSD, and experts are read as tokens need them. This started as a proof of concept, and poc work
An independent researcher has tied more than 16,000 scans of a United Nations statistics portal to artificial intelligence agents the researcher considers highly likely to have been run by OpenAI Grou
Finally got 3.8-Flash-Next running on my M4Pro 48GB Mac with https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF Dense 3.8-27B is just faster... and maybe better due to quantization lev
SpaceX's CFO says Starlink demand is unprecedented. The interesting part is who's driving it! Larry Goldberg @TeslaLarry breaks down the Goldman conference interview. Bret Johnsen reiterated a coming
I’ve spent the last couple of days with Opus 5.5 working on a fork of Inco’s excellent and already blazingly fast Splash engine to optimise it for M5 Max chips. Taking liberties and referring to it as
stunning confirmation of my central work Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize t
New open source Jev like model for running on local devices from a group called Supersonic Labs. It's a 144M param local non-generative local classifier. From their site: Julia 1 opens our research in
Just made this post for those who missed it : https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27b UkisAI released their updated Qwen 27B (tuned for token efficiency). I grabbed the IQ4_XS quant to tes