Search: “models”

I forked llama.cpp's server into AgrillaMoE, a dedicated build for Qwen3.6-35B-A3B (~A4B) with Unsloth quants. On a (vant.ai) rented V100 16GB with the 2-bit UD-Q2_K_XL quant it generates at ~57-60 to

Benchmarking an LLM here with a NVIDIA RTX 5070 12 GB VRAM here I had been working on a llama.cpp based expert streaming setup for Qwen3.8-Flash-Next 177B (UD-IQ3_XXS) on Windows. Benchmark is about 1

Hi folks, I'm a long-time lurker and big fan of this subreddit and a massive self-hosting fan (also outside of AI). I doubt many people will disagree with me here because I see the same arguments bein

Every time I needed to explore a dataset quickly, the same wall appeared — cleaning boilerplate, encoding errors, deciding which chart actually makes sense. The mechanical parts aren't interesting, th

I put an old RTX 3090 to work as a local coding agent with Qwen3.8-27B, llama.cpp, and OpenCode. Here are the setup and results, including what failed. This is a summary of my own blog post, linked be

Hello all, I wanted to share a little project I vibe-coded for myself that you may find useful. As many people here like to suggest, I wanted to give my small local LLMs access to information to impro

40 year tech gap? No problem! The Tandy runs DeskMind, a native DOS program. It talks over WiFi (a PicoMEM 2 card with mTCP) to a small Python server on my PC. That server drives Qwen3.8-27B (NInfer o

I can't seem to find what's wrong. I'm using Pi for my llama-server and sometimes it just stops processing for some reason and stuck after tool call. Logs seems to think that it's finished its job whi

Wulver v0.5 is a full fine-tune of Krea 2 Raw (the whole 12.8B DiT) for anime and anthro characters, including scenes where several characters interact. We made it at Vaelico, an independent studio. C

We had a Dell B300 in the lab for a few weeks and used it to create two fine tunes of Qwen Flash Next. Victoria (coding and agents) Qwen3.8-Flash-Next cut down by 44% using a paper / technique called

kinda realized i type things into AI chats that i'd never say to a friend. money problems, work stuff, health worries, random late-night thoughts lol then i found out all of it is stored on their serv

IQuest-Q1 is a Mixture-of-Experts (MoE) model developed by IQuest for agentic coding, reasoning, and multi-step tool use. It comprises approximately 320B total parameters, with an estimated 15B parame

I'm running Qwen 3.8 27B Q4 XS with ~30 t/s decode (no MTP) at 100k context with q8/q5 KV cache on a 16GB AMD GPU. I wanted to share my setup since many believe Qwen 3.8 27B to be infeasible on 16GB V

Wanted to try Opus 5.5 because as a plus sub with OAI, Luna just couldn't deliver. Opus is a true beast. It does consume tokens on a medium level yes, but the way it works is just refreshing. But... o

I shared the first version here as a prompt builder for Qwen image editing. I've been expanding it into something more useful for visual prompting in general, and the 2.0 beta is ready to try. It's a

Here's the project. I have nothing to do with it. I'm just an amazed user. https://github.com/gufo-org/gufo/blob/main/docs/models/qwen3.8-flash-next/BENCHMARKS.md Those benchmark numbers hold up on re

Someone prompted different LLMs to generate CAD code for a bridge under fixed constraints (2-foot span, under 500g filament, 18-hour print limit), printed them, and load-tested them to failure. The re

I run my own LLM while also having a Openai subscription. Also tried DeepSeek (latest flash now). I run Qwen 3.8 flash Next at an amazing speed on my 2x3090 + Ram! But local LLM never did worked for m

After my last post documenting my LoRA journey, I received a ton of feedback regarding changes to the training process, and decided to do some experimenting. This is using the Fizgig application on a

I built this, so the disclosure goes up front. It's a free, public MCP server plus a REST API with Canadian privacy law data. The MCP endpoint is at https://movahedi.ca/mcp and it uses Streamable HTTP

Been doing a bunch of vibe coding lately. Had my agents host a private WoW server for me, then built out a web browser client so you can play without installing the game and it has mobile controls. Af

Hey r/MachineLearning! I’ve been working on a new optimizer called Tauon (turns out there is already 'teon' but well if you have better idea, - i will gladly accept it! Anyway the core idea of optimiz

Upgraded from 3x RTX 3090s to 2x RTX 5090s on my homelab server and picked up a solid speed jump on top of it from a software update (speculative decoding + NVFP4). Setup: llama.cpp (build b11216), ru

Today I present a fine piece of engineering, carefully assembled inside a custom chipboard chassis: the 2400cc Inference Racer, a.k.a. my winter heater. Power comes from two second-hand AORUS RTX 3090

Hey guys, I came across the Forrester function recently while studying mathematics, and I started wondering if there’s more to it than just being a mathematical function. I have a feeling I’ve seen it

I posted an updated GPT-OSS template a couple of months ago, which was based on Unsloth's version. It turns out that both Unsloth's version (and, thus, mine) contain a very serious bug that can degrad

For the last few days, I've been using two computers to run multiple agents. My 4x3090 machine is running Qwen 3.8 27B with parallel=2, so I can run two agents at the same time. My pi instances are ru

A while back I posted here about a CLI agent I'm building that parses tool calls out of a model's plain-text output instead of trusting native function-calling to work. Finally ran the actual numbers.

Personal note first, being straight with you: I'm lazy tonight and used AI to write the following post: The workflow, the conversion work, and the mistakes that taught me the stuff below are mine, and

I was actually pretty happy with my Qwen3.8-27b setup, and I'd been tinkering with Ninfer to have a version that was 'fast but maybe a bit stupid' and the speed was nice to have as a backup. But I was

I made a new Krea 2 LoRA called Anygles. Give it one image of a person, then spin the camera around them, move it higher or lower, zoom in or out, or mix everything into a random camera path. The four

Saw this today and found it very intriguing. Lots of interesting design choices here, and it's cool to see someone doing something different. Here's a few highlights: Looped transformer: dynamic recur

I didn't know my set up was outperforming nearly everyone until reading another discussion where people were struggling getting half of that speed with half the context on the same hardware. I benchma

I asked this back in 2025, but the AI landscape has changed a lot since then. Not looking for the usual ChatGPT, Claude, Gemini, Midjourney, etc. I'm curious about the lesser-known tools that you actu

This sub is, needless to say very niche and skewed towards the high end. There are tons of extremely high end setups here with multiple gpu's etc. Even 24GB is out of reach of most people financially,

This experiment is live, you can inspect all the internal reasoning, memories, attempts here: https://artificium-covering-experiment.gr.bio/ The problem that the agent is trying to solve is a covering

Having a blast with Qwen Image 2.1 on my M2 Macbook Pro with 32GB RAM, even though it takes 16 minutes to complete the recommended number of iterations with the resolution reduced to 1024x1024. And th

ZCode is now open source, and the reported security issues have been addressed. Source code: https://github.com/zai-org/ZCode The repo includes its desktop app, web workspace, backend, Agent CLI, and

OpenAI’s decision to retire custom GPTs appears to rest on the assumption that Plugins, Skills, Projects, Apps, and Workspace Agents collectively replace their functionality. For many Small and Medium

TL;DR: Local agent loop, ~21 days, one RTX 3090. Task was pretty much 'build a CUDA inference engine for optimized for yourself on this GPU arch.' Got working kernels and benches, not a win over llama

https://preview.redd.it/j13iin0jsfqh1.png?width=1129&format=png&auto=webp&s=3efdd2ad5c10de908aed1a673a6c7e6440177c09 https://preview.redd.it/k3fsrdspzfqh1.png?width=391&format=png&auto=webp&s=1c817687

Salut, je cherche de l'aide pour un projet d'IA locale. J'ai un Dell OptiPlex 7070 Micro sous ubuntu server avec un i5-9500T, 32 Go de RAM et aucun GPU (j'en ajouterai un à l'avenir). Je sais que la m

just saw some benchmarks on the omlx website for the m5 ultra (don’t know how official they are but they seem reasonable): Link For Qwen 3.8 27B q4 it gets 50 tok/s th and 1800 tok/s pp at8k context a

That account is an official Qwen account. Looks like it might take a while though. That whole early access program is taking applications for the next 9 days and after that it's probably gonna be some

Not sure if here’s the right place to ask this but whats ur usecase to use AI in video editing ? I dont believe its up to par. Not looking for FULL VIDEO EDITS, but i got a feeling many use it it smal

We've been tuning vLLM/SGLang/llama.cpp setups for a long time and got tired of the profiling part: nsys trace, open the GUI, squint, change a flag, repeat. The profilers assume a human is looking at