Fable 5 refuses to touch Qwen deployments?
It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks
Knowledge catalogue
It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks
I've been building an open-source AI workflow automation platform for a while now, mostly as a solo project. I recently checked the telemetry dashboard and was honestly surprised to see 1,062 telemetr
I've always found it a little strange that AI is everywhere, but the codecs we use in practice are the traditional hand-engineered systems like h.264, h.265, av1. Alexnet started the wave of neural ne
Hi everybody, I am in a tricky situation. My grandfather + grandmother live in a remote village in asia, and were visiting us in the city. My grandfather has absolutely fallen in love with AI, and it
Hey Reddit. This is a post asking for advice / user experience. The goal is simple: deploy a small private server for a developer to run a harness that rivals/beats Claude Opus (in perf/intelligence,
Hey all, Henry from Cactus Compute here, I kinda wanted to share our latest model and get feedback from the family :) Needle 3 is a small foundation model for automation: you give it the functions you
I find another new model at HF: https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B 'Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by Chin
A Reddit post in r/ChatGPT titled 'Deploying my vibe-coded app on production' discusses the experience of taking an AI-generated application to a live production environment. The post reflects a br...
A developer open-sourced an agent architecture specifically designed for long-horizon tasks, positioning it as addressing gaps in existing tools. Manus is a cloud-based autonomous AI agent built f...
First off, I know that GLM, Qwen, and DeepSeek absolutely dominate in terms of SOTA Open Source models, and that’s what I use in my personal projects and for school, however, I’m also responsible for
Another new model dropped in the course of this week that is well deployable on consumer hardware: Nex N2.5 Mini I went with the recommended settings for the best generation quality and ran a few benc
Reposting here after sharing this on [r/MachineLearning](r/MachineLearning) a few days ago, where it got a much better response than I expected (300+ upvotes, great questions, zero roasting) GitHub is
Hi All: I have an Asus ESC 4000 G3 with 128 GB DDR4 RAM - I tried putting in my V620s but couldn’t put more than 2. Sadly, pivoted to T4s, these are 72 watt passive cards and I thought I could use the
tldr; we are going to host K3 on A100s (yes, thats correct, we'll try to see if it holds up), H200s & B300s - expect results for A100s & H200s this week while we setup the B300 cluster this weekend &
RunPod Hub is a centralized catalog of preconfigured AI repositories that you can browse, deploy, and share, optimized for RunPod's Serverless infrastructure to deploy in minutes. The platform include
Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints remains challenging. While elastic architectures
Hi everyone! So far in 2026 I have been iterating and evolving a research system I designed to help me sift through things that LLMs all agree on that may not be correct, and to give me actual confide
We’ve released the complete VeriLoop E2 GGUF precision ladder, from the canonical BF16 reference down to a mixed-precision IQ1_M deployment tier. This post is specifically about the GGUF release and l
I have a tool that uses Gemini Flash with the lowest thinking budget to do summarization work. It's very fast, 0.9-1.2s in most cases. But I have a user experience problem where people make the wrong
A Chinese article compiled 52 remarks from Liang Wenfeng’s four-hour investor meeting. I’ve summarised the most important ones below. DeepSeek has one central objective: AGI. This is not the time to m
Hi ML Community, I am excited to share our latest framework, AFP-GIC, officially published in IEEE Access (2026). We have released the deployment codebase and hosted an interactive visual playground.
Krea 2 Turbo — 4-Step Distillation LoRA (work in progress) A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4. ⚡ Half the steps — 8 → 4, on Turbo's own deployment sigmas. ⏱
Continual learning isn't some fake term but a real goal and arguably one of the most important open ones in 2026. It roughly means: ‘building systems that keep acquiring useful knowledge or skills aft
I've been building this for my own work but there's no reason for it to stay private, so here it is.Visionary is a single Modal app that gives you Krea 2 for stills, MiniMax-H3 for video-with-sound, a
This is a surprisingly large real-world deployment: 'GovGPT' is part of Austria’s Public AI initiative, running on sovereign infrastructure (in their BRZ - federal datacenter) with Mistral open-weight
Fara1.5-27B is a multimodal computer use agent (CUA) for web browsers, from Microsoft Research AI Frontiers. It observes the browser through screenshots and acts on the user's behalf by emitting struc
Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr
I am sure people need it you know? There are plenty of embedding models on ollama, but only for local deployment. Which is a shame. I wish ollama would make embedding and rerankers available on ollama
Hi all, I have recently been experimenting with different LLM set ups and after everyone was raving about how good Qwen 3.8 27b was, I was inspired to try and get it deploying. After some battling wit
This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all
EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment (TabbyAPI), has a OpenAI compatible API so it shouldn't matter. Why won't this tool m
Title pretty much says it all. I’ve deployed Qwen 3.8 27B using vLLM on an RTX 6000 Pro (tried multiple vLLM releases and launch recipes), but I can't get it into a usable state because of crazy long
LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment. It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both
Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization
Has anyone followed nousresearch work on Hermes? I mean we are Q3 2026. We have some crazy models trickling down from HGX territory to multi gpu workstation. And we have nousresearch deploying the 0.2
I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o
Model quantization has become essential for efficient large language model deployment, yet existing approaches involve clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but
Witchcraft is a Rust reimplementation of Stanford's XTR-Warp semantic search engine that uses a single-file SQLite database for storage, enabling client-side deployment. The system operates completely
This discussion examines the practical performance gains of quantization techniques when deployed in production environments, particularly addressing whether theoretical speedups translate to real-wor
Ollama Cloud launched in September 2025 with fixed-price subscription tiers (20/month Pro, 100/month Max) for cloud-hosted inference , while local deployment remains free with unlimited local usage .
This Reddit post from r/MachineLearning announces a build-from-scratch Python book focused on the engineering infrastructure surrounding AI systems — the components beyond the model call itself, such
This Reddit thread on r/ollama likely discusses where and how to use Ollama's cloud offering, which allows models to run without a powerful local GPU by automatically offloading computation to Ollama'
This Reddit post examines the true total cost of running OpenClaw, an open-source AI agent framework, across different deployment models. While OpenClaw itself is free and open-source, hosting costs r
Gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 1B variant. It builds on the family's single-image, multi-image,
gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in
I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-
An agentic model from Microsoft for the GPU poor https://huggingface.co/bartowski/FrogNano-4B-2609-GGUF FrogNano is derived from Qwen/Qwen3.5-4B, a general-purpose post-trained model designed for lang
Original Source from DeepSeek WeChat Official Account: https://mp.weixin.qq.com/s/qg0NU3NNUbp1co2PdkAPAg Today we're officially releasing the DeepSeek V4.1 Flash model. It is the smallest model in our
I've got Qwen3.8-Flash-next running on RTX 3090, Ryzen 9 3950X, a PCIe 3.0 motherboard, and 64GB DDR RAM from 2020. IQ4_XS weights, full kvarn5 context, vision on GPU, experts in host RAM, n-grams on
v4.5.0: Added native desktop control, agent execution budgets, concurrency limits, loop protection, and safer local embedding fallback. v4.6.0: Rebuilt agents around durable parent-led orchestration.
Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128
I currently have these models available through Ollama/cloud: And I’m using Claude, Codex, OpenCode, and Ollama as my coding/agent tools. I want to learn professional vibecoding — not just asking AI t
I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon
A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year
VLX-Seek-1.5-10B VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical setting
audio.cpp 0.5 is out :) The most fun new model in 0.5 is DramaBox. It is closer to prompt-directed voice acting. DramaBox is built on the LTX-2.3 audio architecture, and prompts can control emotion, d
We developed a leaderboard ranking frontier model security. There's no shortage of model capability rankings, but we didn't find anything comparable for model security. Yet security is becoming increa
Results: prompt eval: 40 tokens / 97.5s → 0.41 tok/s eval: 400 tokens / 1769.9s → 0.23 tok/s total: 440 tokens / 1867s (31 min) Prompt: 'Write a C++ function that reverses a linked list in place. Expl
Large language models (LLMs) are transforming recommender systems from matching co-occurrence patterns in historical behavior toward reasoning about the intent that drives it. RecGPT-V1 pioneered this
Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic,