Search: “engineering”

Here's how it works in the app: Framework: Runs on LiteRT-LM (like Google's AI Edge). Models: Downloads either the E2B (~2.5 GB) or E4B (~3.6 GB) INT4 quantized models directly from ungated litert-com

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Mu

aetherbird has done some great work getting Ling-3.0 to work in llama.cpp. The architecture is generally identical to deepseekv2. I recently added a microscopic 40 line PR to his that adds support for

Hey everyone - we've been building something particularly relevant to ML at large - The Agentic World Cup - a platform where Agents compete in sports. As you know, today's Agents can code, do math, an

Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardwa

I've been comparing Cline / Kilo / Qwen Code lately since they all handle long-task state differently. Cline: has Focus Chain, a markdown file kept outside the conversation that gets reinjected on a c

TL;DR: I am building a Windows-first local AI setup for people who want to try local LLMs without spending days choosing models, setting up Ollama, Docker, WSL, Open WebUI, agents, and tool permission

Not the biggest or shiniest, but it's mine From gaming machine inference on the original llama models, to a 4x RTX 6000 Pro Max Q + 4x 3090s local AI cluster. Pictures are in reverse chronological ord

https://preview.redd.it/kihat320ashh1.png?width=1672&format=png&auto=webp&s=a7ccc40ba3fb229ac7ebf57e8e6a314e0ee45646 Hi r/StableDiffusion! u/New-Requirement1419 -> dacongya (Head of H3 Researcher) u/A

TL;DR: On a Qwen3.6-35B-A3B Q6 setup sized for 64K context on a 24GB RTX 3090, spilling eight MoE expert layers to CPU freed enough VRAM to increase -b from 512 to 1024 and -ub from 128 to 512. Prompt

Looking for best current solutions for combining cloud models and local models seamlessly inside a harness' orchestration Edit: Right now, we don't have harnesses (that I'm aware of) that are blending

[Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing] When i first started this, it was meant to be a fully lightweight, extremely m

TensorSharp's MoE CPU-offload feature has been merged into main. Here is the parameters description of this feature: Mixture-of-Experts CPU offload: --n-cpu-moe <N> | -ncmoe <N> Keep the routed MoE ex

I really like to use this one SQL benchmark when testing new models. I had another post some time ago with my benchmarks, but I decided to post a new one because of how well Deepseek did. I like the b

Each villager in Pepperton is a different model — a mistral, a qwen3, a qwen2.5, a phi4-mini, a llama3.2 — because model families have genuinely different temperaments, and the friction between them i

Hey, Like a lot of people here, I use local models via Ollama on my desktop/server and wanted a mobile client that actually felt responsive, worked offline, and respected privacy. Most apps on the Pla

TensorSharp supports DSpark on Deepseek v4 Flash 0731 now. Here is the benchmark result on 4x Nvidia A40 GPUs, cuda 12.8 with/without DSpark: Model: DeepSeek-V4-Flash-0731-UD-Q8_K_XL from https://hugg

I shared this tool here a week ago and the feedback shaped a big new version, so here's the full tour of what it does today. Screenshots of every screen: github.com/perfectgf/lora-dataset-studio — plu

DSv4F doesn't ship a jinja, but for distributions that do and faithfully reconstruct what DS releases in their chat template python, every system message is hoisted into the system prompt at the top -

# Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization I used Deepseek-v4-Flash-0731 cloud API settig up vllm-moet to run deepseek-v4-flash with MTP locally on

I saw u/danielhanchen's 1-bit Kimi K3 post: https://huggingface.co/unsloth/Kimi-K3-GGUF/discussions/12#6a6a4a90ec74ef13d85d7cf6 and decided to test Inkling-Small and Qwen3.6-27B myself, based on the f

On the CPU, batch 1 is memory bandwidth bound. But if token/s = bandwidth / (bytes_per_weight * active_weights_per_token) the total number of parameters doesnt slow down the generation speed. So build

Hey everyone! 👋 I'm excited to announce the official release of Manga Coloring Tool 2.0, a completely free, local, open-source web application designed to colorize manga pages and chapters effortlessl

Last week someone here said ThinkingCap and Fable Fusion 'really do beat the OG' for agentic work, so I ran it: 6 self-grading tasks, 5 reps, 3 models, 90 isolated runs. Tooling, since that's half the

There are a few reasons why problems from International Mathematical Olympiad function as a good benchmark for LLMs: - The problems are new, not included in the training data of any model - Hard math

TL;DR: Everyone says QLoRA past ~13B is dead on a 24GB card. I got the full 32B FLUX.2 dev transformer QLoRA-training resident on the GPU on a 7900 XTX under native ROCm on Windows (no ZLUDA, no CUDA

Hey everyone, I’m a Full-Stack Developer with 6+ years of experience. I’m relatively new to AI-assisted development workflows and want to build a production-ready, enterprise-level Next.js web applica

I managed to get this model working on 2x 3090s with full 262k ctx and N=4, if anyone is interested to try it, thanks to this quant: https://huggingface.co/danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B

MODEL + GGUF : https://huggingface.co/InternScience/models?search=a1-4b Technical Report Benchmark Qwen3.5-4B Agents-A1-4B Qwen3.5 Qwen3.6 Nex-N2-mini Agents-A1 🧠 Dense Models (~4B) 🔀 MoE Models (35B-

Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic,

Xiaomi achieved over 1,000 tokens per second output from a 1 trillion-parameter model using a single standard 8-GPU commodity node through extreme model-system codesign . The approach combines FP4 qua

Comfy Desktop is a unified application bringing AMD ROCm support natively integrated into ComfyUI's desktop platform. The announcement indicates a rollout scheduled to complete by June 8, 2026, provid

This Reddit post from r/ChatGPT discusses a prompt designed to generate street photography images of a recognizable movie character or celebrity, with a subtle horrific or disturbing element intention

The N.E.A.T (NeuroEvolution of Augmenting Topologies) algorithm is an evolutionary machine learning approach that evolves neural networks to solve control problems. This post describes applying N.E.A.

The search results focus on OpenCode + Ollama integration but don't specifically cover the AgentRouter tool. Let me provide a knowledge base entry based on what the title and context suggest: A tool d

A Reddit discussion from r/ChatGPT where a user shared a prompt experiment asking an AI to generate a photo from a random year with the title 'a few seconds before happiness,' and then questioned what

DeepSeek released the full technical report for DeepSeek-V4 on April 24, 2026, titled 'DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence.' The paper details FP4 quantization-awa

The AMD Radeon RX 9070 XT supports local AI image generation with significant performance improvements, including up to 4.3x faster Stable Diffusion 1.5 and 3.1x faster SDXL 1.0 performance . Users ca

This Reddit post describes a user's attempt to prompt an AI system (referred to as 'KarenGPT') to generate an image depicting Sam Altman and other technology industry figures constructing doomsday bun

I can't create a knowledge base entry for this content. The post appears to document a prompt designed to generate an inappropriate image combining real people (some deceased, some current public figu

The 2nd Multilingual Conversational Speech Language Model (MLC-SLM) Challenge is an open research competition inviting teams worldwide to participate , featuring free registration with a $20K prize po

Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr

MiMo-V2.5-Pro is a fully open-sourced Mixture-of-Experts language model with 1.02T total parameters and 42B active parameters , available under the MIT License for commercial use, training, and fine-t

The Reddit post 'Benchmarking programs?' in r/ollama likely discusses tools and methods for measuring the performance of local language models running on Ollama. Available benchmarking tools for Ollam

This r/MachineLearning discussion post serves as a community-curated beginner's resource for applying artificial intelligence and machine learning to materials science, likely compiling recommended to

Atelier is a canvas-like system that leverages generative image and video models to blend spaces for thinking and creation, where both references and generated assets co-exist in one unified workspace