All entries

Looking for tools to build/run an engine to do STT-LLM-TTS basically. I know how to run all of them separately, and if need be I can build one myself, but I was curious if there is already an open sou

I'm currently building my system around 3060s, but I might be able to get a 4060 for a nice deal. At first it seemed like a no brainer, but turns out the 4060 has lower memory bandwidth. In a system t

Spoke with many providers who lurk in this sub, plus met folks who work in inference. For a 10k monthly revenue, a provider was able to retain only 200 dollars in profit due to GPU costs. Their custom

Like many of you, I have seen many posts and tweets in the last weeks complaining about Artificial Analysis being 'broken', 'meaningless', and 'bought out.' People who say this have done no research a

OpenAI has decided to fully shut down a protein design project I'm working on for a client. Needless to say, open weight models are the only way forward. submitted by /u/Terminator857 [link] [comments

Title says it all. https://github.com/gjabdelnoor/Day1DeepseekV4.1-CPU The goal is pretty simple, I like having infinite slow tokens from the bioinformatics machine in the lab to run overnight or over

The downside of uncensoring a model is that it is known to potentially damage it, but CyberTiel is an even more capable software engineer than its censored TielCoder base, while allowing offensive sec

Here we go again, DeepSeek is back again with a new model V4-1 Flash A multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens Mark

People keep on getting confused about this, so I looked at the safetensors on hf. To be precise, the main model about 551.566B parameters with 40 layers. The FFN experts total to 543.582B parameters,

I like to benchmark new models that come out on motion videos. So here's a test I did for deepseek v4.1 flash. And I have to say flash has probably graduated from being a Luna class model to nearly an

Original Source from DeepSeek WeChat Official Account: https://mp.weixin.qq.com/s/qg0NU3NNUbp1co2PdkAPAg Today we're officially releasing the DeepSeek V4.1 Flash model. It is the smallest model in our

I believe I may currently hold the record for memory constrained inference for Qwen3.8–Flash-Next on Apple Silicon — needing only about 21GB of allocations. Introducing Cherenkov, an inference engine

Hey y'all! We've released a new model in our lineup: GigaChat-3.5 Reasoning. It's a 432B-A28B MoE with Gated DeltaNet for long-context efficiency. We trained domain experts (code, math, general, etc.)

https://preview.redd.it/4c6otbeg1qoh1.png?width=1139&format=png&auto=webp&s=2d127bd3bc83d5e6ba65c3a4bd909529f34e9a3c Total beginner here, i've just downloaded Ollama and Gemma4:26b, but when it answer

So I was playingw ith embedding models I saw that when you upgrade from model A to B, you face a very big backfilling cost Ie, suppose you have a 1b vectors from model A, and then you want to use mode

First off, I'm super thankful for the value Ollama has given me Got me setup with my first local LLM setup (early OpenClaw days amirite!?) Then got me into their 20 plan and what really sold me here w

I’ve been working on perfecting my setup ever since 3.8 hit, but no longer pay for subscriptions. I had a coding interview today and setup openrouter ahead of time as another option. I figured I could

After noticing that it is ranked among MUCH larger frontier models in the EQ-Bench Creative Writing benchmark and the Hemingway-bench, I decided to give it a try and was very impressed. I didn't do ve

Spent about two weeks convinced I had a prompting problem. My local Qwen3 agent would just stop. No error, no crash, finish_reason came back as 'stop' like the model had decided it was done. Except it

are very strict... if 20 calls to kimi are more than my 5 hour limit why even bother? im gonna cancel my sub for sure https://preview.redd.it/gbqkci93gnoh1.png?width=1080&format=png&auto=webp&s=6fac5a

Both good and bad things have come from a subreddit that was lot more niche than for example r/flashlight rapidly transforming into the largest online forum about an increasingly core part of the infr

so i saw that openui.com released OUI-1, a model fine-tuned on DiffusionGemma. the training dataset uses OpenUI-Lang, a custom DSL (domain-specific language), instead of plain HTML, Markdown, or React

Github Repo. Blog post. 💡 TL;DR (from the Github Readme) Spend less without making the agent do less useful work. SoL-Pi is a standalone extension for Pi that packages four reusable efficiency mechani

Part 4 of the same box. Part 1 was 17 -> 25-29 t/s with the expert cache PR, part 2 was 37-41 t/s after switching to UD-Q4_K_XL and stacking MTP on the cache, part 3 was the top-k fallback that was so

In v0.10 we achieved significant speedups over v0.9. The video shows the maximum speedups we measured for specific groups of models. Among LLMs, the biggest gain came from Qwen3 0.6B, which runs over

I am just sharing my config for Qwen 3.8 27b that fits on a 5060TI, what is cool about this is that you can even get vision! and a 85K context (I have 1.5gb of headroom for more context or a better qu

I initially saw CED as just an efficiency improvement, but the more I read about it, the more it feels like an inference architecture leap. The encoder/decoder split has some pretty interesting implic

Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up s

Long time lurker, but I'm finally upgrading to 128GB VRAM, and I'm trying to figure out what to run. I had been leaning towards Qwen3.8 Flash-Next at ~Q4, and I generally prefer to not run anything be

mentria.ai is a browser inference engine I've been building solo, from scratch in WebGPU/WGSL. This week it crossed a milestone I had been chasing for a while: a 27B one-bit model answering at up to 3

I fine-tuned Qwen3-TTS with inline transcript control tags rather in lieu of a separate instruction parameter and thought the community here might be interested in the result. https://huggingface.co/S

It seems to use a 96-bit LPDDR5X memory bus, instead of the previous 64-bit wide busses. Considering it's on 2nm, that's expensive silicon. That should result in around 115 GB/s memory bandwidth. A20

In july I posted here about aletha codex and I got a lot of support, and a decent amount of people started using it. I also got a lot of criticism too, which really helped me funnel this version 1.5 t

Running the full deepseek-ai/DeepSeek-V4-Flash-Vision-Exp on consumer Ampere — 10-12x RTX 3090, SM86-compatible vLLM build. 285B MoE, FP4 experts + FP8 attention, 157 GB weights. Highlights: - **60+ t

im new to using ai so i may be missing something. I set up ollama but dont want any of my data to go to any of the model providers, for certain models i appear to need an api key to download the model

Just a reminder for those out there itching to get into local llms - don't let FOMO or 'gear acquisition syndrom' take over. No matter the hobby, it's so easy to get stuck in a trap where we buy more

I shared ThoughtDAG here a while ago. I've now recorded how I use it: select a phrase in an answer, explore it on a branch, and connect the parts I want in the next question. I'm the developer. Though

I have been a dwarfstar fan for awhile and I really liked glm 5.3 flash but needed it to be materially faster to feel good using it. In the screenshot you can see the outcome of using the model with a

I'm using two different models. One Gemma 4 and the other qwen 3.5. Both of them are responding to my input either by silence ('thought for a moment.... then nothing') or nonsense: <|turn>user <|turn>

I’ve been experimenting with explaining AI/LLM concepts in a way that doesn’t assume too much technical background. This video is about KV Cache and why longer context windows require more memory duri

(hopefully this is okay to here - it seems like audio models and image / video modeals is allowed but yeah this is a bit different) So I've been doing independent audio research for a while now. The u

A few posts tagged with 'new model' present models that are finetunes. My opinion : I'd rather have the 'new model' tag reserved for new 'major' releases, like a new Qwen model, Deepseek V4 -> Deepsee

From the email 'Early access to new models During the program, approved users can try two next-generation Xiaomi MiMo models for a limited time: MiMo-X-Pro-Preview and MiMo-X-Flash-Preview. The MiMo-X

I was curious about how AIs talk to each other and how a conversation changes as it goes on, so I built AI Arena Lite. It connects to Ollama and lets you assign models to different agent roles, follow

I’ve been using Qwen a lot recently and I’m honestly very impressed with it. For some of my tasks, Qwen seems to understand what I’m actually trying to achieve better than ChatGPT/GPT-5.6. I currently

Its chat template has this line: {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %} xhigh is the most expensive of its three settings (low / medium / xhigh). If you never set one