Search: “engineering”

Looking for best current solutions for combining cloud models and local models seamlessly inside a harness' orchestration Edit: Right now, we don't have harnesses (that I'm aware of) that are blending

[Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing] When i first started this, it was meant to be a fully lightweight, extremely m

TensorSharp's MoE CPU-offload feature has been merged into main. Here is the parameters description of this feature: Mixture-of-Experts CPU offload: --n-cpu-moe <N> | -ncmoe <N> Keep the routed MoE ex

I really like to use this one SQL benchmark when testing new models. I had another post some time ago with my benchmarks, but I decided to post a new one because of how well Deepseek did. I like the b

Each villager in Pepperton is a different model — a mistral, a qwen3, a qwen2.5, a phi4-mini, a llama3.2 — because model families have genuinely different temperaments, and the friction between them i

Hey, Like a lot of people here, I use local models via Ollama on my desktop/server and wanted a mobile client that actually felt responsive, worked offline, and respected privacy. Most apps on the Pla

I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama tangentially related Subreddit, and this was the conclusion. Feels pretty accu

TensorSharp supports DSpark on Deepseek v4 Flash 0731 now. Here is the benchmark result on 4x Nvidia A40 GPUs, cuda 12.8 with/without DSpark: Model: DeepSeek-V4-Flash-0731-UD-Q8_K_XL from https://hugg

I shared this tool here a week ago and the feedback shaped a big new version, so here's the full tour of what it does today. Screenshots of every screen: github.com/perfectgf/lora-dataset-studio — plu

DSv4F doesn't ship a jinja, but for distributions that do and faithfully reconstruct what DS releases in their chat template python, every system message is hoisted into the system prompt at the top -

# Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization I used Deepseek-v4-Flash-0731 cloud API settig up vllm-moet to run deepseek-v4-flash with MTP locally on

I saw u/danielhanchen's 1-bit Kimi K3 post: https://huggingface.co/unsloth/Kimi-K3-GGUF/discussions/12#6a6a4a90ec74ef13d85d7cf6 and decided to test Inkling-Small and Qwen3.6-27B myself, based on the f

On the CPU, batch 1 is memory bandwidth bound. But if token/s = bandwidth / (bytes_per_weight * active_weights_per_token) the total number of parameters doesnt slow down the generation speed. So build

Last week someone here said ThinkingCap and Fable Fusion 'really do beat the OG' for agentic work, so I ran it: 6 self-grading tasks, 5 reps, 3 models, 90 isolated runs. Tooling, since that's half the

There are a few reasons why problems from International Mathematical Olympiad function as a good benchmark for LLMs: - The problems are new, not included in the training data of any model - Hard math

Having read some merged PRs in the past, I know that they were fully written by Claude Code (or similar), so this basically fixes the delusion. But at the same time, we might start seeing more AI slop

TL;DR: Everyone says QLoRA past ~13B is dead on a 24GB card. I got the full 32B FLUX.2 dev transformer QLoRA-training resident on the GPU on a 7900 XTX under native ROCm on Windows (no ZLUDA, no CUDA

Hey everyone, I’m a Full-Stack Developer with 6+ years of experience. I’m relatively new to AI-assisted development workflows and want to build a production-ready, enterprise-level Next.js web applica

I managed to get this model working on 2x 3090s with full 262k ctx and N=4, if anyone is interested to try it, thanks to this quant: https://huggingface.co/danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B

MODEL + GGUF : https://huggingface.co/InternScience/models?search=a1-4b Technical Report Benchmark Qwen3.5-4B Agents-A1-4B Qwen3.5 Qwen3.6 Nex-N2-mini Agents-A1 🧠 Dense Models (~4B) 🔀 MoE Models (35B-

Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic,

Xiaomi achieved over 1,000 tokens per second output from a 1 trillion-parameter model using a single standard 8-GPU commodity node through extreme model-system codesign . The approach combines FP4 qua

Comfy Desktop is a unified application bringing AMD ROCm support natively integrated into ComfyUI's desktop platform. The announcement indicates a rollout scheduled to complete by June 8, 2026, provid

This Reddit post from r/ChatGPT discusses a prompt designed to generate street photography images of a recognizable movie character or celebrity, with a subtle horrific or disturbing element intention

The N.E.A.T (NeuroEvolution of Augmenting Topologies) algorithm is an evolutionary machine learning approach that evolves neural networks to solve control problems. This post describes applying N.E.A.

The search results focus on OpenCode + Ollama integration but don't specifically cover the AgentRouter tool. Let me provide a knowledge base entry based on what the title and context suggest: A tool d

A Reddit discussion from r/ChatGPT where a user shared a prompt experiment asking an AI to generate a photo from a random year with the title 'a few seconds before happiness,' and then questioned what

DeepSeek released the full technical report for DeepSeek-V4 on April 24, 2026, titled 'DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence.' The paper details FP4 quantization-awa

The AMD Radeon RX 9070 XT supports local AI image generation with significant performance improvements, including up to 4.3x faster Stable Diffusion 1.5 and 3.1x faster SDXL 1.0 performance . Users ca

This Reddit post describes a user's attempt to prompt an AI system (referred to as 'KarenGPT') to generate an image depicting Sam Altman and other technology industry figures constructing doomsday bun

I can't create a knowledge base entry for this content. The post appears to document a prompt designed to generate an inappropriate image combining real people (some deceased, some current public figu

The 2nd Multilingual Conversational Speech Language Model (MLC-SLM) Challenge is an open research competition inviting teams worldwide to participate , featuring free registration with a $20K prize po

Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr

MiMo-V2.5-Pro is a fully open-sourced Mixture-of-Experts language model with 1.02T total parameters and 42B active parameters , available under the MIT License for commercial use, training, and fine-t

The Reddit post 'Benchmarking programs?' in r/ollama likely discusses tools and methods for measuring the performance of local language models running on Ollama. Available benchmarking tools for Ollam

This r/MachineLearning discussion post serves as a community-curated beginner's resource for applying artificial intelligence and machine learning to materials science, likely compiling recommended to

Atelier is a canvas-like system that leverages generative image and video models to blend spaces for thinking and creation, where both references and generated assets co-exist in one unified workspace

This Reddit thread on r/ChatGPT discusses community recommendations for the best tools to create animated wallpapers, likely covering AI-assisted options such as using ChatGPT alongside image generato

AetherMind is a locally-run personal memory system built by a Reddit user (r/ollama) that leverages Ollama with the `qwen2.5:7b` model and Retrieval-Augmented Generation (RAG) to allow users to query

A Reddit thread on r/ChatGPT where users discuss whether they have replaced or reduced their use of Google Search since adopting ChatGPT and other AI tools. The discussion likely explores personal sea

This Reddit thread likely discusses user frustrations with performance issues when running GLM-5.1 via Ollama's cloud inference option (`glm-5.1:cloud`), a flagship agentic coding model from Z.AI. Com

This Reddit thread from r/StableDiffusion discusses the community's search for AI tools capable of analyzing an existing video and automatically generating a descriptive text prompt from it — essentia

This r/StableDiffusion post documents a community experiment exploring techniques for generating images of two identical-looking subjects ('twins') using Stable Diffusion without relying on a LoRA mod

A Reddit post on r/ChatGPT in which a developer shares a Telegram bot they built specifically to engage with and study the behavior of online scammers for research purposes, and seeks volunteer tester

A Reddit post from r/ChatGPT in which a practitioner claims experience building AI prompt systems for real-world investment research shares five common mistakes investors make when using AI for financ

This Reddit post argues that AI's most practical and immediate value lies not in replicating or surpassing human intelligence, but in serving as a tool for cognitive offloading — handling mentally tax