Search: “models”

Hardware: rtx 5080 16 gb vram; 64 gb ram ddr5 6000hz; ssd with unlimited memory; ryzen 7 9800 x3d. OS: Windows 11 Software: I’d prefer llama.cpp, but it’s not a strict requirement; I’ll use whatever y

People who moved from older version to this new one, how much improvements do you see on inference? Ex: llama.cpp performance? This new version comes with Linux Kernel 7.0. Hope AMD cards gonna enjoy

I kept running into the same thing. Change a system prompt, get a different answer, and I couldn't tell if that was my change or just the sampler. The loop made it worse. Edit the Modelfile in another

hi i make tool for this called llmog it's purpose to make llms free to - auto annotation datasets - reclassification existing yolo datasets running totally local using llama cpp or vllm or use externa

As in actual physical heat. How are people coping with the heat that running a decent inference rig pumps out? I've got dual 5060 Ti GPUs and a relatively modest CPU (Intel 14 Core Ultra 5 245KF Deskt

I dont know about others, but Nvidia seem to close the lid on older GPUs since they want to push their new technology. Llama and team has been the to go places for older GPUs like V100s. Knowing how N

Their last version 7.14 was released just a month ago. llama.cpp PR(waiting for approval) for Version 10.0 https://github.com/ggml-org/llama.cpp/pull/27803 Hope this version comes with more boost & im

Post rédigé avec l'aide de GLM5.3 flash Contexte : formateur professionnel d'adultes en France, j'utilise des agents LLM au quotidien et le français qu'ils produisent est mon critère n°1, avant les be

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks: Stronger Coding: GLM-5.3 is the mo

Everything has changed in two months: DS4 0731 flash was the start of a wave that is taking open weights to paradise. It is easy to think that Qwen 4 and GLM 6 will be on par with Mythos. submitted by

I installed the Claude Desktop via the ollama Mac app. Unfortunately, no connectors or plugins can be installed or none are listed. Is that normal or is there no possibility there submitted by /u/amne

HuggingFace releases microduck a 10 inch open-source biped with 15 actuators and sensors (camera, speaker, LiDAR, NFC, bluetooth, wifi, ...) that you train yourself with reinforcement learning, $400.

I was using UD-Q3_K_XL until now with more than 140000 context. Quality wise it's very good, very few erroneous tool calls. Then I saw many others here reporting good results with IQ3_XXS, so I gave i

Run it locally with one command (downloads the model, opens a small web UI): uvx --from sopro soprotts serve A year ago I posted Sopro V1 on this subreddit, a small TTS I trained for ~$250 as a side p

I've been building this for my own work but there's no reason for it to stay private, so here it is.Visionary is a single Modal app that gives you Krea 2 for stills, MiniMax-H3 for video-with-sound, a

With this move Nvidia is not only acquiring the HuggingFace platform, but they might also effectively acquire the copyright to the `llama.cpp` project, together with the entire team behind it. In Febr

I wanted to see just how capable Qwen3.8-27b is locally. I have a RTX 4090 and 96GB of RAM but the Q4 comfortably fits in the GPU with plenty of context, the few times I needed more than 130k context

https://x.com/romanchernin/status/2092488160680751437?s=20 - Multimodal (Vision) - 1M Tokens Context Window - DeepSWE ~63% EDIT: THE ORIGINAL POST IS GONE Roman Chernin @romanchernin The mysterious Ox

I was bored and handwrote a tiny 100-line bash script to let an agent search for and read articles from an offline wikipedia archive during a regular chat. It's not particularly useful, but it's defin

So here's the thing, almost everyone use NVIDIA to run their LLMs, we also do the same, a lot of people we've met use like RTX PRO 6000 or even H100, B300 It seems like everyone eyes is looking at NVI

.. didn't llama.cpp aka ggml get acquired by Huggingface not too long ago? How would this sale affect llama.cpp and ggml? What are possible risks, and did ggerganov share his opinions or potential nex

Krea 2 Turbo — 4-Step Distillation LoRA (work in progress) A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4. This is an update release, following up from my previous post

Portable rig for Qwen3.8-27B-BF16 200K+ token prompts. My work Panasonic Toughbook + the T1 + power brick + headphones all fit in my lunchbox. Need the BF16 for huge context highly sensitive document

good morning, has anyone already used qwen 3.8 with the tesla v100 or sxm2 or pcie? any information is useful to me context token/s thanks for the help. submitted by /u/Full-Chicken2110 [link] [commen

My best bet is the embedding-offloaded linear where the 51b n-grams track semantics and context, like 3.8 with its loss of real world knowledge bolted back on. Otherwise? Sparse full-attention with de

https://preview.redd.it/usib0obv1blh1.png?width=1171&format=png&auto=webp&s=fa7b6e9155fcc461a2d397f9cf75aece4c72235c https://preview.redd.it/w453mok02blh1.png?width=2613&format=png&auto=webp&s=693182d

hello everyone. I'm training PPO variants on different multi-agent tasks from the VMAS library (Independent PPO / Graph PPO and such, see HetGPPO by Bettini et al.). I noticed that for every architect

It worked pretty well, digging through and analyzing some local files. Claude code regularly stops at some point and fails to continue while DSH worked for 2 h, recognized that it could benefit from r

similar to video upscaler LLMs, what's the bleeding edge right now on something similar for audio music? the idea is you feed it in with a low bitrate MONO old soundtrack, and the LLM then recreates i

TL;DR: I'm planning to spend around 100 on cloud GPUs to benchmark Qwen3.8-27B with a focus on questions that actually matter when running it locally: different quant levels/providers, 8-bit vs 16-bit

For the GPU poor among us: I'm curious what results you're getting with low quants of Qwen 27B 3.8. My main inference hardware is limited (Mac mini M4 24 GB), but I'm getting great results with Unslot

With vision enabled I am able to run at 150k context on a single RTX 3090 and the results are just amazing. I was even able to write a gmail plugin for DeepSeek harness with locally hosted Qwen 3.8 27

A few weeks ago I shared Agent Quest, my open-source experiment that turns Claude Code and Codex sessions into heroes living inside a small 2D world. The original idea was mainly about making it easie

What I ran: 8x B300 on Modal, 56.79 per hour, vLLM, tensor parallel 8, native MXFP4 Cold boot ~27 min (1.56 TB load, JIT, 51 CUDA graph captures) TTFT 0.92 to 1.02 s, decode 92 tok/s steady, 83 tok/s

Hi all Looking at a PowerColor R9700 for Qwen3.8-27B, Q4, llama.cpp/Vulkan. AMD's own blog quotes 51.8 tok/s but doesn't say what context length that's at, or whether MTP=2 was holding up. Separately

Hey everyone! Last year, Mozilla released Orbit, an AI-powered browser summarizer hosted on a GCP server. After people started digging into the extension, they discovered things like backend endpoints

https://huggingface.co/jojohai/Qwen3.8-27B-MTP-graft Tested on Vulkan, the grafting saves RAM compared to using an external file. What I don't guarantee however is the quality of answers. The model is

It performs at full capacity without any issues anywhere, but only the Ollama DeepSeek seems particularly trashy. Why on earth has the Ollama DeepSeek unusable trash that goes into over reasoning and

If anyone still remembers GLM-4.5-Air from last year, you can now get a nice speedup by enabling MTP in llama.cpp. It is a 106B MoE with only 12B active parameters, which makes it interesting for mach

Has anyone else experimented with ubatch size when running GLM-5.2 locally? I was testing the 226 GiB GLM-5.2-UD-IQ2_XXS GGUF on 3x RTX PRO 6000 Blackwell GPUs and got a pretty interesting result. Wit

Hi there. I have a Lenovo Yoga laptop (16GB LPDDR5, AMD Ryzen 7 8840HS, integrated GPU). I have downloaded Ollama and installed Qwen 2.5coder:3b llm. I have tried OpenClaw, and Qwen Code but it seems

Using llama.cpp I seem to be unable to get my to GPUs working tougether correclty, so I need help somehow. Setup: 96GB RAM, one Blackwell 5000 (48GB) and one 3090 (24GB). I am trying to run the UD-Q3_

The preprint can be accessed via the following link: https://arxiv.org/abs/2608.12408 (q-bio.NC / cs.LG). And for the code: https://github.com/nilsleut/evaluation-resolution-rsa The following assertio

Released today: version 1.3 of energygraph Zero build dependencies, lightweight tool for live views of the power-consumption. Version 1.3 adds support for dGPUs from nvidia, intel, amd. Depending on v

Where did THESE things go for ChatGPT? Claude has them. I can access older versions of my conversations easily. But with ChatGPT, they just DISAPPEARED. I can no longer regenerate answers or following

I recently bought a 32GB V100 off eBay and thought I was all set with rudimentary tests showing zero issues. But then I started seeing weird VRAM-related errors in llama.cpp. I sic'ed Claude on it, to