Knowledge catalogue
Overview ggml_cpu_fp16_to_fp32 leverages hardware F16C intrinsics (AVX-512, AVX2, etc.), faster than the software-only ggml_fp16_to_fp32_row, bringing 17-31% gain in prompt processing rate for a small
🧩 DeepSeek Harness v0.1 is now available in Developer Preview! 🔹 We’re opening it up to developers building agent harnesses worldwide and open-sourcing the codebase in MIT license. 🔹 Powered by the Co
To my fellow crazies, the few. Those who dared wrestle with llama-70b, mistral-large, goliath, mistral8x22B, DeepSeekV2/3, wept when llama4 behemoth was announced, picked yourself up and are now wrest
Same old prompt, just appended a TIP in the end: 'Write a single HTML file with a full-page canvas and no libraries. Simulate a realistic side-view of a moving car as the main subject. Keep the car vi
Hey everyone, Non-native speaker, writing my post by hand, let me know if I make mistakes (can only learn from it!) Muse Glimmer 30B is so far quite nice, but I haven't found a clear-cut case yet what
The maximum frame count allowed by the native 'MiniMax H3 Reference to Video' node technically is 1008, even if that's way over the training range of the model, which is 324 frames. But why not try? S
Diffusers has a PR with deets: https://github.com/huggingface/diffusers/pull/14456 Minimax is working on this repository right now and put up a bunch of samples: https://github.com/MiniMax-AI/music3-d
Given Gavin Bakers recent interview where he says: > Multiple private companies are planning on spending at least 2x more per GPU for compute as contracts roll-off and some have spoken about this publ
I’m genuinely pretty disappointed with how my Ollama Pro annual subscription turned out. For context, I’m a technology editor and developer, and I had actually used Ollama Pro before. My previous expe
https://preview.redd.it/nqr9nj5028jh1.png?width=1516&format=png&auto=webp&s=47af40a0737429dc65f2e455b87b2db81725cb14 So I wanted to see... is it possible/viable to run this perhaps once in a while som
I may have stumbled onto something interesting while trying to figure out a recurring artifact in ChatGPT image generation and editing (maybe applicable to other models as well?). It started with a ve
This release brings smaller prompts, longer-running conversations, safer remote access, and more reliable model streaming. External tools and skills are now discovered only when relevant. Long chats g
Stumbled across this new vision model, SenseNova-Vision. It's a 7B MoT model, Apache 2.0 license, which is cool. The main idea is it treats pretty much all computer vision stuff as just one generation
If you haven't checked the paper: https://arxiv.org/abs/2608.09867 TLDR: the authors show that you can swap out the 'encrypted' reasoning of the biggest model, like Opus, Sol, and put them into weaker
With all the Suno drama over their download limits and their heavy watermarking that could be used for future copystrikes if you stop paying them, this Minimax Music 3 couldn't be more on time. I trie
I've been googling 'tar extract gz' for about ten years. and I finally did something about it. It started out as a research project and I ended up with a Fine-tuned Qwen2.5-Coder-1.5B on 125k natural-
I’m working on performance regression detection using machine learning/anomaly detection. My setup is basically: Healthy runs are used to learn normal behaviour Regression runs are used to see whether
They highlighted 3 things on countdown page: VLM, Agentic Improvements, and Think mode. What improvements do you expect? Reply here! Personally, I want to meet a sage who has attained enlightenment. T
I've been building an open-source tool for diagnosing world models, the kind that predict future frames from a starting context and a sequence of actions. It compares a rollout against ground truth an
In OCP APAC 2026, Tai AMD SVP of compute and enterprise AI said agents don't cut GPU demand but they just pile on a whole extra layer of orchestration, retrieval, and tool-calling work that runs on CP
North Micro Vision Instruct is a 2.4B-parameter open-weight vision-language model with native-resolution image support, released under the Apache 2.0 license. It is designed as a compact foundation fo
Since lot's of people were sceptical or whatever, heres how to uncensor / jailbreak V4 flash and proof. No it is not lead on whatever, first prompt, first try, every time. Put this in System message:
Link to the article: KV Cache Quantization on Gemma 4 31B: Non-QAT vs QAT KLD benchmarks with BeeLlama.cpp v0.4.3, fork of llama.cpp with more KV cache quantization options, comparing Gemma Q4_0 non-Q
Yesteday a paper showed a gap that allows to see 100% of the reasoning tokens form ALL Claude and GPT models Stealing Reasoning Traces from Proprietary LLM APIs. check it out, they have published lots
ChatGPT desktop app will not download images even after a complete reinstall I am on Windows 11 and the Download button in the ChatGPT desktop app does nothing when I try to download generated images.
Hey guys, Just finished benchmarking DeepSeek V4 Flash 284B + DSpark on a single RTX PRO 6000 96GB. Short version: DSpark: ~15–17% faster generation on my coding workload On this setup, the DSpark dra
Sometimes you have tasks that are outside of your expertise and the idea is this workflow automation could be leveraged to manage to have local AI figure it out using research from his workflow gather
I’m not super knowledgeable of the entire AI industry, but as we see this industry grow and the Cold War that is happening between the US and China on AI development, I can’t help but notice what US c
Liquid AI put out LFM2.5-VL-3B today, which is a 3.1B vision model that weighs roughly 2GB and fits well on a phone Benchmarks are benchmarks so I tried something sillier. Took a photo of a little Ste
LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment. It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both
Hey Folks, I've been making quants for a while - recently I took a short break to get into hardcore research (submitted my first EMNLP paper during it!). Along the way, I built up a little arsenal of
i was hoping they would release both today but the big one just dropped now , and it says its been listed since 5 hours ago on huggingface. I guess the real date for 27b is https://modelscope.cn/model
Muse Glimmer 30B feels significantly more precise and reliable, it almost never drops the ball or breaks rules. However, its designs lack creative depth and richness. Qwen3.6 35B, on the other hand, i
One of the reasons I got into local LLMs was the possibility of getting answers using my own documents and books (a few hundreds) instead of having to search through them manually. However since I'm n
I also have 64 gb ddr4 ryzen 5600 Using llama.cpp Ubuntu distro Settings are as follows --n-gpu-layers 999 --n-cpu-moe 36 --no-mmap -ctk q8_0 -ctv q8_0 -fa 1 -c 9000 So rebuilt my llama.cpp build to r
Ran the model with quants (Q5) and MTP by bartowski with llama.cpp server. It takes ~24GB ram running on M5 Pro with 48GB at about 65t/s. On some tasks it was quite the overthinker. Overall, the quali
I'm trying to figure out GPU Kernel optimization on older hardware like SM80(ampere) . Is there tools you guys use? Or frameworks? Im waiting for this framework https://www.reddit.com/r/LocalLLaMA/com
Warning: Kinda long post. If you don't like reading, please skip for your own sanity. Also, I've got nothing to sell, just a tinkerer, so I just want to share ideas and learn from you guys too. When I
A few things right off the bat: it reasons very efficiently. Like Grok 4.5 levels of efficient thinking it quantizes very well. My first few tests with iq3_xxs were better than Qwen/Gemma behaved at t
Hello everyone! I like dumb things. I like working with weak computers and microcontrollers. I like the simplicity and low electricity usage. Simply put, the efficiency of a 'dumb' PC. The first tim
Seems like we're limited to qwen finetuned MoEs for now. Looking at the current landscape - focus seems to be on dense models (muse glimmer 30b, qwen 3.8 27b) for smaller setups. Is upgrading to 24GB
I just heard of this model. Seems to be a competitor to DeepSeek V4 Flash. About the same size and active parameters. Anyone tested it compared to V4 Flash? Link: https://huggingface.co/upstage/Solar-
Hi! I'm really obsessed with LLM hallucinations for the last 6 days 😭 I started by designing system prompts to attack hallucinations but failed, obviously. Now I tried reading logprobs and... I think
Back after ~6 months — rebuilding my spiking language model around CPU-first inference Hey everyone. It’s been around six months since I last posted anything about this project here. Some of you might
Link: https://arxiv.org/pdf/2604.27883 Hi, Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may
Spent two nights getting deepseek-ai/DeepSeek-V4-Flash-0731 (284B MoE, 13B active, native FP4/FP8, 1M context) running production-grade on two DGX Sparks connected by one QSFP DAC cable. Everything —
Been benchmarking DSv4 Flash 0731 on a Flow Z13 (Ryzen AI MAX+ 395, Radeon 8060S / gfx1151, 128GB LPDDR5X) for the past week. Figured I'd share what actually works and what doesn't — there are a lot o
I'm very happy with some Linux admin tasks I'm throwing at a locally running DeepSeek. My request was simple, check why 'samples' folder is taking more and more space on one of the machines on my LAN,
Everything started with the sudden death of my old ASRock J1900. While looking for the perfect ITX replacement, I stumbled upon the Chinese CW-NAS-ADLN-K motherboard, which looked perfect on paper: In
I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon
Here's how it works in the app: Framework: Runs on LiteRT-LM (like Google's AI Edge). Models: Downloads either the E2B (~2.5 GB) or E4B (~3.6 GB) INT4 quantized models directly from ungated litert-com
Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Mu
Lots of rumor and misinfo bouncing around, so I put some of these old mining cards to the test. I used 4 of the 8GB cards, set to 64GB each. Lots of models fit entirely on a single card, and you can a
Laid off then my partner of 10 years said he's leaving, have to move, etc... I will post the r/hardwareswap link when I make it. I'm willing to add some incentive for r/LocalLLaMA folks. I will also a
I WILL PAY 1 MILLION FOR THIS 😭 I will literally pay 1,000,000 to the developer who fixes this. ChatGPT macOS: ⌘C text → ⌘Tab back to ChatGPT → ⌘V …and the search box ISN’T FOCUSED. 🤦♂️ Why do I have
Hi LocalLlama, we're super excited to release Unsloth Desktop today! 🦥 It's the first desktop app that enables you to run and train models locally. Open-source. Available on Mac, Windows, and Linux Su
Testing with 3-5 agents. Decode performance is superb, however if one performs a web search and needs to process a few thousand tokens, ALL other agents will grind to a halt: I've tried tuning a littl
aetherbird has done some great work getting Ling-3.0 to work in llama.cpp. The architecture is generally identical to deepseekv2. I recently added a microscopic 40 line PR to his that adds support for
Needs a lot of requests compared to Qwen (almost twice) and Gemma (almost x3). Final score is fine, even though it is 'not a coding model' https://wonderrico.github.io/local_llm_benchmark/benchmark-ma
Hey everyone, Today we release Luth-2-0.8B and Luth2-2-2B, two non-reasoning models that set a new state of the art for French across a wide variety of tasks for their size 🚀 A few notable scores on F