Search: “engineering”

A few people here mentioned interest in a way to test their custom Pi setups, so I figured I’d drop this here: RoastMyHarness The basic idea is a small engine that sets up an environment to run DeepSW

Since the Qwen3.5 0.8B model is an interesting one for small specialized fine tunes, I was curious how fast it can run on CPUs. Why CPUs? Mainly because I want to use it as a local dictation cleanup m

Yoyo I'm back with updates to the fastest inference engine with minimal quality loss for Qwen3.8-27B. The last few weeks I've been optimizing decode speed and I don't think it can be pushed further, u

specs hardware: M4 Max 128GB Studio inference engine: llama.cpp (qwen4exp branch) judge: claude-opus-4-6 AtomicChat/Qwen3.8-Flash-Next-GGUF Qwen3.8-Flash-Next is a great model I benched in my previous

Hey! I forked NInfer (a from-scratch C++20/CUDA inference engine for Qwen models) and added two things: tensor-parallel across two GPUs, and YaRN ×4 rope scaling. Together they let Qwen3.8-27B NVFP4 r

Hey everyone! I’ve been working on a 100% local cognitive memory engine designed specifically to give multi-agent systems and NPC societies human-like memory. It's called Eternal Thread. Instead of ju

Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl

I build LoRA Dataset Studio — free, open source, self-hosted, no account and no telemetry. It is not a competitor to ai-toolkit: it orchestrates it. ai-toolkit is the trainer; this is everything befor

Four days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak). Since then it went to ~114, then ~138 tps single-user with DFlash2 drafting a

Edit: Title says 134 tps, it's actually 138 -- keep in mind my 3090 is power limited to 250w. Three days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single re

Two days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak) - yesterday's update took that to 99 tps single-user / ~1,000 tps at 64 concurr

I'm back. Yesterday I released the first version of hyper-optimized Qwen3.8-27B inference engine for a RTX 3090, reaching 82 tps on single request and 672 peak. Over the last 24 hours I've been explor

Hi, After a long night of optimizations, I believe I have made the fastest inference engine for Qwen3.6-28B on a 3090. Quick metrics: - 250w power capped - Up to 195k context (ships with 150k for safe

GPT-5.6 Sol High and X-High (Web Chat) feels severely nerfed since 08/06/2026 update (https://openai.com/index/improving-gpt-5-6-sol-in-chatgpt) I use it for a pretty complex Unreal Engine 5 project (

There's an interactive chart and some extra data in the blog post if you're interested. There are plenty of KL-divergence benchmarks for GGUF models, but most of them compare one GGUF quant against an

I'm a retired platform engineer, coding mainly in Rust, and involved with a ML study group. I developed a ML programming language (alternative to Python, Colab) to help me learn (and teach) ML concept

Xiaomi-Robotics-1 is a robot foundation model trained on over 100K hours of real-world manipulation trajectories. It is a Vision-Language-Action (VLA) model engineered for out-of-the-box mobile manipu

Following up on my Qwen 3.6 port, I wanted to keep adding models and ended up fixing a bunch of things along the way, so it's its own engine now: Mference. Same core idea from TurboFieldfare, MoE mode

I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o

Krea 2 has been my favorite base model for character work, but the moment you put two character LoRAs in the same generation they smear into one blended face. Attention bias, prompt engineering, and C

Greetings Community! 👋 I hope everyone is doing well! I'm Tauhid — Senior EEE student from a Bangladeshi University Today I'd like to share an open-source project I’ve been developing called P.A.I. (P

While building this workflow a thing that surprised me was that, initially I thought the pipeline was complete: From Goal to → Decompose → Research → Specification → Implementation It successfully bro

Hi everyone! We’ve just released a major update to the leaderboard! We are expanding beyond Python with a new multilingual slice featuring real-world software engineering tasks across 5 languages. Ope

Hey r/LocalLLaMA, I’m a retired engineer with a background in distributed computing, currently running a 1-person startup. Like many people here, I’d love to experiment with 2T+ MoE models locally. Th

I'm a software engineer who mainly builds softwaes/applications, and I'm starting to work on machine learning projects. Since ML workloads often require GPUs, I know services like Google Colab and Kag

The news about Anthropic settling a class action lawsuit for 1.5B over training data isn't just a legal headache for them, it's a massive warning sign for engineering teams relying entirely on closed

Some fixes & boost(pp) for SYCL/Intel. Merged PRs: [SYCL] Flash Attention with XMX engine via oneDNN graph API (SDPA) on KV f16 for Xe2 ; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at

This post likely covers the process of creating a custom drone simulation environment using MuJoCo, a physics engine commonly used in machine learning research. The project involves leveraging MuJoCo'

Peptide and hormone replacement therapy companies have been systematically posting to r/biohackers to manipulate AI-generated search answers on ChatGPT and Google through a practice called AI-engine o

Dreamverse is an AI video generation engine that produces 30 seconds of 1080p video in 4.5 seconds on a single GPU , making it significantly faster than existing systems like Sora. It provides a creat

Witchcraft is a Rust reimplementation of Stanford's XTR-Warp semantic search engine that uses a single-file SQLite database for storage, enabling client-side deployment. The system operates completely

Ollama's MLX engine runs on NVIDIA GPUs via CUDA v13 on Windows and Linux. Users have reported that Ollama crashes on RTX 3060 with cuda_v13 in versions 0.13.0 through 0.15.6, while deleting the cuda_

This Reddit post from r/StableDiffusion asks the community for recommendations on which prompt engineering techniques and AI image generation models (specifically comparing z-image and z-image turbo v

A Reddit post discussing performance issues with GLM-5.1 when running through Ollama Cloud. GLM-5.1 is Z.AI's next-generation flagship model for agentic engineering, with significantly stronger coding

A Reddit post from the r/ChatGPT community sharing a user-discovered technique for improving the quality of AI-generated images in ChatGPT, likely involving prompt engineering strategies or workflow a

Moss is a YC-backed high-performance runtime for real-time semantic search that delivers sub-10ms lookups, instant index updates, and zero infrastructure overhead, running where the agent lives — clou

OllamaGPT is a self-hosted, ChatGPT-style web interface that runs locally using Ollama as its backend LLM engine, requiring no API keys or external services. It is designed for developers and power us

Ollama is a local inference engine designed for running pre-built LLMs on your own machine, and it does not include fine-tuning capabilities — this distinction is correct. Unsloth (and its Unsloth Stu

A Reddit thread from r/ChatGPT exploring the user experience phenomenon where GPT responses can feel cognitively draining or over-engineered even when technically accurate — touching on issues like ve

GLM-5.1 is Z.AI's next-generation flagship model for agentic engineering, featuring significantly stronger coding capabilities than its predecessor and achieving state-of-the-art performance on SWE-Be

TurboOCR is a high-performance OCR project that combines PaddleOCR with NVIDIA TensorRT, implemented in C++ and CUDA, achieving throughput of 270–1,200 images per second using FP16 half-precision infe

GLM-5.1 is Z.ai's next-generation flagship model for agentic engineering, built on a 754-billion parameter Mixture-of-Experts architecture with 40 billion active parameters per token, a 200,000-tok...

Local AI tools such as Ollama, LM Studio, and Jan all rely on the same underlying inference engine (llama.cpp) and support compatible model formats (primarily GGUF), meaning a single downloaded mod...

I've made a few posts recently, just talking about the hardware I've got and the model I use. Pair of P100s, Unsloth Qwen3.6 35b A3B UD Q4 K XL. There's been a ton of talk of model and hardware specif

This table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res

recently saw a bunch of open decision models pop out of nowhere in the last two weeks (laya, liquid's d1, cloudflare's clef-flash, interfaze's lev), so I wanted to see how far apart they actually are