All entries

just saw some benchmarks on the omlx website for the m5 ultra (don’t know how official they are but they seem reasonable): Link For Qwen 3.8 27B q4 it gets 50 tok/s th and 1800 tok/s pp at8k context a

I'm at a crossroads with two grad school options that would take me in somewhat different research directions, and I wanted some general advice on these fields, their growth, and industry alignment. I

What I want to know is how your daily interaction routine looks like. Do you wake up, take your coffee/tea/drinkname.txt, have a nice morning, then click a button and somethings starts and creates som

We compared 20 model versions on 360 frozen joke prompts, with four jokes requested per prompt and model names hidden from our humor-trained judge. Fable 5 had the highest estimated score: 66.8 points

I developed https://lora-hunt.info/ for finding LoRA/PEFT adapters that might work with local LLM setups. The site collects public model metadata from Hugging Face and tries to make base-model compati

Everyone now talks about the architecture that's not auto regressive and does lightning fast probability prediction with a json schema. I worked on this literally one year back in March 2025, publishe

IFM released K2-Horizon-7B , diffusion augmented LLM at upto 5200 tps, with claimed lossless speedup. causal LLM architecture and adds a plug-and-play diffusion adapter alongside the autoregressive we

CPU : epyc 7262 MB : ROMED8-2T GPU : V100 16GB PCIE x6 I have built a server with six V100 GPUs. I am now testing it and plan to eventually run Qwen3.8-Next-Flash configured with TP2 and PP3. Because

UPDATE https://github.com/ggml-org/llama.cpp/pull/29009 MERGED More Gen AI coverage and frameworks integrations to minimize code changes New models supported: On CPU: Gemma-3n On CPU, GPU: Kokoro-82M,

I've been training character LoRAs specifically for Krea 2 and wanted a simple place where people could browse everything instead of having the models scattered across individual posts. I currently ha

I released KaozKit this week, a Swift library for running LLM agents written in JavaScript inside a Mac app. Built from the start to keep everything local. Providers: MLX with Hugging Face model manag

I've been building Selfcoder, a VS Code coding agent focused specifically on local models. I originally started working on it because I was frustrated with the context management when using local mode

Hello, I'm at the old pro plan. With the upcoming cloud model retirement of deepseek v4 0731 flash in mind because 4.1 is supposed to be more efficient while providing even better results I remembered

I was testing this quantization IQ3_XXS from GSQ-RCO with PI. It is a heavy quantization case, the model is in IQ3_XXS and KV cache in (Q4_0, Q4_0). I asked it to make the flight simulator, using that

That account is an official Qwen account. Looks like it might take a while though. That whole early access program is taking applications for the next 9 days and after that it's probably gonna be some

Hey Reddit. This is a post asking for advice / user experience. The goal is simple: deploy a small private server for a developer to run a harness that rivals/beats Claude Opus (in perf/intelligence,

I spent the first week of September building a personal assistant over Gemma 4 in Ollama on a 16 GB M1 Pro: Node backend, React web UI, Expo phone client, two-model routing, cross-session memory, RAG

Hi Ollama Team, I’m writing regarding the announced retirement of GLM-5.1 from Ollama Cloud on 25 September 2026. I wanted to provide some fairly detailed technical feedback because GLM-5.1 is not sim

The model is derived from Qwen3.8-27B, a 27B hybrid-attention causal language model (architecture unchanged), but uses ternary weights to shrink model size down to <6GB in size. According to the model

Hey everyone, Jovan from UkisAI here, a small lab building the tech to make tiny frontier LLMs possible (and doing it open-source!) The purpose of this post is simply to thank the community for all th

omg why are they doing this ? im getting so much request from deepseek-v4-flash:0731 comparing to dsv4.1-flash :( anyone know some alternatives which still uses 0731 flash ? submitted by /u/subimpact

Not sure if here’s the right place to ask this but whats ur usecase to use AI in video editing ? I dont believe its up to par. Not looking for FULL VIDEO EDITS, but i got a feeling many use it it smal

Another question asked 1 thousand time, but I can't find actual threads. Sorry in case of mistake. Due to interstellar price of hardware like high-end GPU (more than 5k euros for a 5090 and similar pr

Anyone else trying out the new Union stealth model on openrouter? It’s pretty good for a free model, and of course I had to run tokenizer tests and training cutoff tests which established it’s the 99%

I posted something about a 9b model a few days ago. The real problem was 2 fold. Someone suggested the size was too big to prove it all out. It's a fair argument. I needed the model depth though. The

We've been tuning vLLM/SGLang/llama.cpp setups for a long time and got tired of the profiling part: nsys trace, open the GUI, squint, change a flag, repeat. The profilers assume a human is looking at

Hey guys, really wondering now why we got (SOTA or at least good enough) open weights on nearly every use case, like LLMs, T2I (Krea2), music (Yue2), video generation (Minimax H3), but not image edit?

I find another new model at HF: https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B 'Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by Chin

hey folks, it's Merve from Hugging Face 🤗 with the latest cool things happening in between LLMs and 3D, we have written an illustrative explainer guide on 3D asset landscape, what 3D representation yo

Apple is considering offering an AI server built around 'M8' series chips and has discussed incorporating Nvidia networking hardware, The Information reports. The system would be sold to outside custo

Anyone interested in trying out a new inference provider? We're testing Gemma 4:26b and Qwen 3.8:27B behind an OpenAI-compatible API and looking for a few users to give them a spin and tell us how it

Hey experts! I genuinely have this question and would love a general consensus from people actually using these models. LLMs have advanced a lot in benchmarks and in practical usage. Even GPT-4o and G

can you draw a picture of a flower in gimp? So I ran through this setup to install the MCP server suite from GITHUB, it's pretty easy to follow the install - and installs the libraries necessary to co

GoBench evaluates LLMs on 9x9 Go games against a ladder of KataGo opponents, from random to superhuman. It measures general reasoning ability, strongly correlates with ARC-AGI 2 (r=0.83 correlation),

I've been working on something that started as the training backend for my Hierarchos architecture, but it has grown into a much broader project: https://github.com/necat101/Hierarchos-Native Hierarch

It turns out Krea 2 was trained on a number of different pixel art resolutions. These loras help you get some of these resolutions more consistently and avoid ugly 'mixel' resolutions with inconsisten

GitHub: https://github.com/pfekin/LARA I've been working on LARA (Lightweight Additive Residual Adaptation), a research project on making post-training modular for frozen language models. I've also de

We gave two small LLMs a real customer-service task. One got 100/100. The other stopped with the job half-done. I wanted to test something more meaningful than benchmark scores: can a small model actu

A follow-up to my earlier post about small models on phones. Last time it was fixed tasks in a single browser. This is something else. Ministral 3 3B Q4_K_M in llama.cpp on a Samsung Galaxy S21, four

It's late at night in the lab, I've been working on a basic script for a virology project, and holy hell the safeguards have been pissing me off. Mirroring detectEVE data over rsync to my laptop by ma

It's an old Mitch Herdberg routine about tricky survey questions. But every time Qwen 3.8 spits out one of those sentences that is structurally correct but makes no actual sense, it pops into my head.

I let Qwen 3.8 27B 4bit quantized with 100K context window run autonomously for 63 hours (50 million+ tokens) to try to solve the RH. Of course it did not solve it, but the experiment still shows it's

Saw the new Introducing System One Models & Jev - TypeSafe AI Blog Jev model which just outputs probabilities given choices and I thought it sounded a something the Qwen reranker models could do? Then

Setup for one long agentic coding session (tools + vision) on a single 5090: Qwen3.8-27B RVN Heretic (ARA abliterated) at Q6_K, 159744 context, 1 slot, q8_0 K/V, DFlash2 speculative decoding, vision p

Achieved steady 15 tokens/second output and 100-120 promp processing per second with 12GB GPU (RTX 5070 SFF) with Qwen3.8-Flash-Next-GSQ-RCO-GGUF at 3 bpw (IQ3_XXS which maches BF16 on AIME25). Around

At last my small pebble of contribution has been merged. Thanks to CISC. Support for the model DFM Mimir 1B gguf here: https://huggingface.co/noctrex/DFM-Mimir additional optimization will be added in