Search: “deployment”

I've always found it a little strange that AI is everywhere, but the codecs we use in practice are the traditional hand-engineered systems like h.264, h.265, av1. Alexnet started the wave of neural ne

Hi everybody, I am in a tricky situation. My grandfather + grandmother live in a remote village in asia, and were visiting us in the city. My grandfather has absolutely fallen in love with AI, and it

Hey Reddit. This is a post asking for advice / user experience. The goal is simple: deploy a small private server for a developer to run a harness that rivals/beats Claude Opus (in perf/intelligence,

A Reddit post in r/ChatGPT titled 'Deploying my vibe-coded app on production' discusses the experience of taking an AI-generated application to a live production environment. The post reflects a br...

First off, I know that GLM, Qwen, and DeepSeek absolutely dominate in terms of SOTA Open Source models, and that’s what I use in my personal projects and for school, however, I’m also responsible for

Reposting here after sharing this on [r/MachineLearning](r/MachineLearning) a few days ago, where it got a much better response than I expected (300+ upvotes, great questions, zero roasting) GitHub is

Hi All: I have an Asus ESC 4000 G3 with 128 GB DDR4 RAM - I tried putting in my V620s but couldn’t put more than 2. Sadly, pivoted to T4s, these are 72 watt passive cards and I thought I could use the

tldr; we are going to host K3 on A100s (yes, thats correct, we'll try to see if it holds up), H200s & B300s - expect results for A100s & H200s this week while we setup the B300 cluster this weekend &

A developer open-sourced an agent architecture specifically designed for long-horizon tasks, positioning it as addressing gaps in existing tools. Manus is a cloud-based autonomous AI agent built f...

RunPod Hub is a centralized catalog of preconfigured AI repositories that you can browse, deploy, and share, optimized for RunPod's Serverless infrastructure to deploy in minutes. The platform include

Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints remains challenging. While elastic architectures

We’ve released the complete VeriLoop E2 GGUF precision ladder, from the canonical BF16 reference down to a mixed-precision IQ1_M deployment tier. This post is specifically about the GGUF release and l

A Chinese article compiled 52 remarks from Liang Wenfeng’s four-hour investor meeting. I’ve summarised the most important ones below. DeepSeek has one central objective: AGI. This is not the time to m

Hi ML Community, I am excited to share our latest framework, AFP-GIC, officially published in IEEE Access (2026). We have released the deployment codebase and hosted an interactive visual playground.

Krea 2 Turbo — 4-Step Distillation LoRA (work in progress) A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4. ⚡ Half the steps — 8 → 4, on Turbo's own deployment sigmas. ⏱

I've been building this for my own work but there's no reason for it to stay private, so here it is.Visionary is a single Modal app that gives you Krea 2 for stills, MiniMax-H3 for video-with-sound, a

This is a surprisingly large real-world deployment: 'GovGPT' is part of Austria’s Public AI initiative, running on sovereign infrastructure (in their BRZ - federal datacenter) with Mistral open-weight

Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr

I am sure people need it you know? There are plenty of embedding models on ollama, but only for local deployment. Which is a shame. I wish ollama would make embedding and rerankers available on ollama

Hey all, Henry from Cactus Compute here, I kinda wanted to share our latest model and get feedback from the family :) Needle 3 is a small foundation model for automation: you give it the functions you

Another new model dropped in the course of this week that is well deployable on consumer hardware: Nex N2.5 Mini I went with the recommended settings for the best generation quality and ran a few benc

Hi all, I have recently been experimenting with different LLM set ups and after everyone was raving about how good Qwen 3.8 27b was, I was inspired to try and get it deploying. After some battling wit

This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all

EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment (TabbyAPI), has a OpenAI compatible API so it shouldn't matter. Why won't this tool m

Title pretty much says it all. I’ve deployed Qwen 3.8 27B using vLLM on an RTX 6000 Pro (tried multiple vLLM releases and launch recipes), but I can't get it into a usable state because of crazy long

LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment. It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both

Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization

Has anyone followed nousresearch work on Hermes? I mean we are Q3 2026. We have some crazy models trickling down from HGX territory to multi gpu workstation. And we have nousresearch deploying the 0.2

I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o

Model quantization has become essential for efficient large language model deployment, yet existing approaches involve clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but

Witchcraft is a Rust reimplementation of Stanford's XTR-Warp semantic search engine that uses a single-file SQLite database for storage, enabling client-side deployment. The system operates completely

This discussion examines the practical performance gains of quantization techniques when deployed in production environments, particularly addressing whether theoretical speedups translate to real-wor

Ollama Cloud launched in September 2025 with fixed-price subscription tiers (20/month Pro, 100/month Max) for cloud-hosted inference , while local deployment remains free with unlimited local usage .

This Reddit post from r/MachineLearning announces a build-from-scratch Python book focused on the engineering infrastructure surrounding AI systems — the components beyond the model call itself, such

This Reddit post examines the true total cost of running OpenClaw, an open-source AI agent framework, across different deployment models. While OpenClaw itself is free and open-source, hosting costs r

Gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 1B variant. It builds on the family's single-image, multi-image,

gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in

Hi everyone! So far in 2026 I have been iterating and evolving a research system I designed to help me sift through things that LLMs all agree on that may not be correct, and to give me actual confide

I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-

An agentic model from Microsoft for the GPU poor https://huggingface.co/bartowski/FrogNano-4B-2609-GGUF FrogNano is derived from Qwen/Qwen3.5-4B, a general-purpose post-trained model designed for lang

I have a tool that uses Gemini Flash with the lowest thinking budget to do summarization work. It's very fast, 0.9-1.2s in most cases. But I have a user experience problem where people make the wrong

I find another new model at HF: https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B 'Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by Chin

Original Source from DeepSeek WeChat Official Account: https://mp.weixin.qq.com/s/qg0NU3NNUbp1co2PdkAPAg Today we're officially releasing the DeepSeek V4.1 Flash model. It is the smallest model in our

Continual learning isn't some fake term but a real goal and arguably one of the most important open ones in 2026. It roughly means: ‘building systems that keep acquiring useful knowledge or skills aft

I've got Qwen3.8-Flash-next running on RTX 3090, Ryzen 9 3950X, a PCIe 3.0 motherboard, and 64GB DDR RAM from 2020. IQ4_XS weights, full kvarn5 context, vision on GPU, experts in host RAM, n-grams on

v4.5.0: Added native desktop control, agent execution budgets, concurrency limits, loop protection, and safer local embedding fallback. v4.6.0: Rebuilt agents around durable parent-led orchestration.