Search: “people”

A few people here mentioned interest in a way to test their custom Pi setups, so I figured I’d drop this here: RoastMyHarness The basic idea is a small engine that sets up an environment to run DeepSW

I’m developing this https://swarm.dafonet.com If someone want to collaborate? I’m looking for people with capabilities to test it in different infraestructures. Best regards Gabriel Giani submitted by

Hey everyone! I'm curious to hear from people that use a combination of cloud-based frontier models and local ones for development. I'm planning to set something similar up and wanted to hear about ac

For many, Vram is precious, I see many people recommend using --no-mmproj-offload to save gpu vram but it is painfully slow. Especially if you are using it with agentic coding. If possible, add that s

I recently moved to Japan and ran into an issue where a lot of anime on Netflix only has Japanese subtitles here. I wanted something similar to the subtitle translation extensions that already exist,

People keep on getting confused about this, so I looked at the safetensors on hf. To be precise, the main model about 551.566B parameters with 40 layers. The FFN experts total to 543.582B parameters,

After noticing that it is ranked among MUCH larger frontier models in the EQ-Bench Creative Writing benchmark and the Hemingway-bench, I decided to give it a try and was very impressed. I didn't do ve

In july I posted here about aletha codex and I got a lot of support, and a decent amount of people started using it. I also got a lot of criticism too, which really helped me funnel this version 1.5 t

Just a reminder for those out there itching to get into local llms - don't let FOMO or 'gear acquisition syndrom' take over. No matter the hobby, it's so easy to get stuck in a trap where we buy more

Most of the new ~30B (and now ~10B, thankfully for my GPU) models we see score really high on benchmarks, but I feel like they don't push back on dumb ideas enough. I think most people don't being lik

What is Astra actually supposed to be better at, Because if the answer is “everything,” then I genuinely do not understand what kind of testing was done for writing-heavy use cases. My biggest issue i

Up front: I'm one of the people building the page-perception layer used here. We started by testing small local models. The result turned out to be more interesting than the original test. 12 small mo

I'm looking for more information to expand my creativity around this. I don't really have a realistic idea of what people actually do with local AI yet, I mostly just want to explore the possibilities

On September 1st, three young men from California used Gemini to plan a hiking route on Mount Shasta. The AI ​​told them to bring far less water and food than was actually needed for a hike of that di

I keep seeing these massive ultimate chatgpt prompt posts all over social media, like we’re still using Chat and other LLMs the same way we did when they first came out. Meanwhile the more I use AI, t

I feel that reproducibility is now a lost cause in machine learning research for three reasons: Many research is moving towards the physical AI territory, where you need expensive hardwares or even en

A month ago I posted about Gemma sometimes solving the reasoning problems inside my translation data instead of translating them. A few people suggested two very fixes, which is to use a proper transl

https://preview.redd.it/6e9xb57a42nh1.png?width=787&format=png&auto=webp&s=ffae7996bbf8ab00498cc62c733e7597dc550f24 I'm not sure how many people care about Android Studio, but I think it's cool that G

Not written by AI all mistakes mine. I saw people on the subreddit saying that 3.6 works better without thinking. It made me want to know for certain about which is better, 3.6 or 3.8 for low thinking

I've made a modification of llama.cpp MTP for people that want to run models like QWEN 27B on 16GB and similar setup, the focus is reducing the memory cost of MTP allowing more speed for less ctx cost

So, ideally for this thread we exclude the ones that everyone on here is already well aware of and discussing on here a lot, like N-gram, quantization improvements, MTP, D-flash, and D-spark, since th

I see this kind of approach quite often with local AI models. For example, when optimizing models for an RTX 3090, people sometimes convert them from FP8 to INT8 so they can run efficiently on Ampere

People who moved from older version to this new one, how much improvements do you see on inference? Ex: llama.cpp performance? This new version comes with Linux Kernel 7.0. Hope AMD cards gonna enjoy

We are considering to buy computer, server to run decent model on premises. I would like to run big open source model with more that 70b parameters roughly. I read the people used to run on Apple Stud

Hey, An LLM writing into durable memory is an unreliable author. It invents, mislabels, restates, and contradicts. Once one of those mistakes is saved, the agent can carry it into every conversation t

As in actual physical heat. How are people coping with the heat that running a decent inference rig pumps out? I've got dual 5060 Ti GPUs and a relatively modest CPU (Intel 14 Core Ultra 5 245KF Deskt

Ever since Qwen 3.8 Flash Next dropped, there's a misconception going around that N-gram tables will let people run 1T+ parameter models on a single server with 980B parameters offloaded to SSD. I'm h

A lot of people are skeptical Nvidia will keep huggingface intact now that they will buy huggingface. There's a lot of doom and gloom about not having any alternatives, removing nsfw models, saying th

So here's the thing, almost everyone use NVIDIA to run their LLMs, we also do the same, a lot of people we've met use like RTX PRO 6000 or even H100, B300 It seems like everyone eyes is looking at NVI

I'm curious what other people are using for local LLM coding / agentic coding with only 8 GB of VRAM. My current setup is: Intel Core i7-11800H RTX 3070 Laptop, 8 GB VRAM 32 GB DDR4 RAM openSUSE Tumbl

Hey everyone! Last year, Mozilla released Orbit, an AI-powered browser summarizer hosted on a GCP server. After people started digging into the extension, they discovered things like backend endpoints

Hey everyone! Last year, Mozilla released Orbit, an AI-powered browser summarizer hosted on a GCP server. After people started digging into the extension, they discovered things like backend endpoints

I'm confused by something I've noted in the recent discourse since Kimi K3 has been released on subscription. Has anyone done the (I know Ollama should display this) math as well on how much K3 is usi

seems like others are now using lower precision models. On open code sub i see a lot of people saying model has gone dumber since price hike started. How it is behaving in Ollama cloud PRO? Same? subm

Hello everyone, tried so hard to optimize my config and finally I simply get up to 70 t/s with q6 variant. And wanted to share with you guys so that other people with the same setup can enjoy. Please

Hello all, I'm fairly new to the whole Llm space and could really use some guidance from people who've done similar work. Im building a pipeline to extract structured data from large-batch of invoices

Do you think today’s GPU-poor systems running Qwen 3.5 9B can outperform the ones we had with ChatGPT-4o? I mean, when 4o disappeared, people felt like they’d lost a great model, and now Qwen 3.5 9B Q

Hey! Like a lot of people here with consumer GPUs (RTX 4060 8GB in my case), I wanted to see if I could use local models for daily coding tasks instead of burning cloud credits on simple boilerplate.

I'm always testing an image prompt with a picture of a historic place in my hometown – a small but well known 250,000 people town in Germany. I'll just ask the model, in which City this photo has been

Over & over I see people post slop and excusing it as 'english is not my native language'. As if, if we could understand their language we'd see they wrote it all by hand Fine then, new rule: > if you

Some people told me that the difference in richness and layout between Glimmer and Qwen wasn't clear to them. This example makes it super clear. I'm aware that comparing Glimmer 30B (a dense model) wi

Since lot's of people were sceptical or whatever, heres how to uncensor / jailbreak V4 flash and proof. No it is not lead on whatever, first prompt, first try, every time. Put this in System message:

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon

Something happened today that I think most people are going to miss because there are two separate stories and neither one is getting the full picture. OpenAI expanded Daybreak. If you haven't heard o

leaked Super specs have the 5070 Ti and 5080 going 16GB to 24GB and the 5070 to 18GB, thanks to the new 3GB GDDR7 modules. 24GB on a Ti-class card is actually the number people here have been waiting

I'm guessing that many people have been waiting for this comparison. For clarity, both models are running at full FP16 KV-cache. Due to VRAM limitations, Muse Glimmer is running full 262,144 context,