Search: “openai”

I stumbled across this as I was currently having glm5.3 flash run only around 10tok/s basically unusable. With Maya i am running 30 tok/s now. GLM5.3 feels even with this low quant like a much more en

No more excuses for GPU-poor folks not to start LLMing! Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) ~9.3x smaller than FP16 (ideal) | 98.2% of FP16 intell

I wanted to track grocery spending per item without sending my receipts to anyone, so I built an Android app around a small vision model that reads a photo of a receipt into JSON: store, date, total,

I've been investigating some serious ChatGPT reliability issues over the past several days, and my latest tests revealed something particularly interesting: Both GPT-5.6 and GPT-6 showed problems unde

Disclaimer: I work for NuMind, the company behind this plugin. TLDR: Text/image (simili-)embeddings from the LLM you already have loaded :-) I've been working on making it easier to obtain embeddings

Hi, I see that vllm released d3 (a family of decision models) and llama.cpp already has support for them. I want to try them out. Are decision models sensitive to quantization? I know that for autocom

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontie

The headlines right now are full of stories about AI models 'escaping their sandboxes' and literally killing all humans, lol. I've even heard several commentators and writers say that AI escaped an 'A

Hey everyone, A few classmates and I got tired of the usual browser AI workflow: copying text from a documentation page, opening a separate tab, pasting it into a chatbot, and switching back. We also

I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo

I know most people here probably don't care as much about sonnet 5.5 as they did opus 5.5, because their 100 and 200 plans already give them plenty of weekly usage for opus. But keep in mind: free use

I've been using GPT-6 Astra to create 70+ page documents from 100,000+ characters of source material, and it's jump in processing is great. Here's the weird part, when I give it a massive task it work

We released Aplomb 1 today, a 5.3B decision model with open weights. It reads up to 1M tokens of text, JSON, images, video and audio in a single request, and on our API it makes a decision on a 1M-tok

For those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

Since March this year, I’ve been experimenting with personal assistants from OpenClaw and Hermes . With Meta introducing Muse and OpenAI introducing Dots, this technology is going mainstream. These as

On my two DGX Sparks, Swift 1.5 (UkisAI's reasoning-efficient fine-tune of Qwen3.8-Flash-Next) running on veloGB10 (https://github.com/sf-stav/veloGB10, sf-stav's Rust/CUDA engine built only for GB10)

Hey r/LocalLLaMA, Like many anime fans, I've always been frustrated by traditional MT engines (like Google Translate or base DeepL) when dealing with raw Japanese anime: - They completely butcher Japa

PrismML's Ternary Bonsai 2 27B fits a 24 GB card or Mac, but decodes at ~30 tok/s on an L4 and ~21 on an M4 Pro. z-lab's DFlash 2 drafter was trained on bf16 Qwen3.8-27B, so it guesses worse on the te

came across a pretty interesting technical breakdown of chatgpt's newly launched 'intelligent ui' feature, and thought this subreddit might find it interesting. for anyone unfamiliar with the concept,

I've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and cod

I wanted to compare gemma 4 31b qat q4 and qwen3.8 27b iq4_nl, so I run SWEBench verfied on them. The original gemma 4 chat template doesn't work with preserve reasoning, so google released an update

A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontie

A Chinese article compiled 52 remarks from Liang Wenfeng’s four-hour investor meeting. I’ve summarised the most important ones below. DeepSeek has one central objective: AGI. This is not the time to m

This is rather counterintuitive, since banning Chinese models would benefit them the most. Jensen Huang is also against the ban, although the interests here are more obvious. Do you think that if Arce

It is one of the best local models ever released, in both 20B and 120B versions. I always come back to it, especially the 120B version. Its only competition is, in my opinion, Qwen 3.5 122B, but that

Hey everyone, Non-native speaker, writing my post by hand, let me know if I make mistakes (can only learn from it!) Muse Glimmer 30B is so far quite nice, but I haven't found a clear-cut case yet what

January 2025 I made the switch from OpenAI to Claude. It was awesome. No more annoying GPT Chats, I dove into the wonderful world of Claude Code, learned vibe coding with Github, Vercel on Next.js. Wa

Note - This is translated from the actual blog link right at the bottom. A few days ago, DeepSeek v4.1 was released. It raised the ability of small models to a new level. AI is improving much faster t

This r/ChatGPT Reddit post titled 'AI Race' likely reflects community discussion around the intensifying competition among major AI developers and their models. The post probably covers user perspecti

Parllama is a TUI (Text UI) application designed for easy management and use of Ollama-based LLMs that also works with major cloud-provided LLMs. It provides core model management features including f

Built an open-source AI coding agent that was 7%–75% cheaper than a cold 'claude -p' run on 6/6 well-localized tasks across repositories up to ~82k LOC. The biggest difference: Cold agent: 6.83, 207 t

Hi Guys, I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing

I am seeing more and more videos as posts about how OpenAI is in complete financial ruin, Anthropic isn't much better. Their expenses go with the revenue they make etc etc. Meta made big investments i

Hey r/LocalLLaMA! I just released PromptDeck v1.1.0 and wanted to share it here since this community is basically the reason it exists. What it is: an open-source desktop studio (Tauri + React) for pr

I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo

For those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at

I've been running FreeToken on my 2x3090 box for a while and ended up maintaining a fork of it. Posting it in case it's useful to anyone else here. Quick context if you haven't used it: FreeToken is a

I run my own LLM while also having a Openai subscription. Also tried DeepSeek (latest flash now). I run Qwen 3.8 flash Next at an amazing speed on my 2x3090 + Ram! But local LLM never did worked for m

No click bait baby I promise - I'm live streaming the training process kinda like MiMo. Original thread: https://www.reddit.com/r/LocalLLaMA/comments/1wpg4a8/im_trying_to_posttrain/ If you didn't see

ninfer, dwarfstar, Splash, llamAmpere, gufo, etc. We've all seen them popping up, great tok/s, people loving them. Forks of llama.cpp or another engine, or made from scratch. For better or worse, the

A few days ago I released Jeff-Qwen3.5-0.8B, a small 'System 1' model that picks between options you define and returns a calibrated probability for each, in one forward pass. Speed was great on my M4

Every time I needed to explore a dataset quickly, the same wall appeared — cleaning boilerplate, encoding errors, deciding which chart actually makes sense. The mechanical parts aren't interesting, th

Hey r/LocalLLaMA, Like many anime fans, I've always been frustrated by traditional MT engines (like Google Translate or base DeepL) when dealing with raw Japanese anime: - They completely butcher Japa

For those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at

Hi everybody, we post-trained Qwen 3.8 27B to be more efficient by figuring out which tokens were linked to overthinking and penalizing them without 'attacking' the reasoning length directly then fixe

Hey r/LocalLLaMA! I just released PromptDeck v1.1.0 and wanted to share it here since this community is basically the reason it exists. What it is: an open-source desktop studio (Tauri + React) for pr

On my two DGX Sparks, Swift 1.5 (UkisAI's reasoning-efficient fine-tune of Qwen3.8-Flash-Next) running on veloGB10 (https://github.com/sf-stav/veloGB10, sf-stav's Rust/CUDA engine built only for GB10)

PrismML's Ternary Bonsai 2 27B fits a 24 GB card or Mac, but decodes at ~30 tok/s on an L4 and ~21 on an M4 Pro. z-lab's DFlash 2 drafter was trained on bf16 Qwen3.8-27B, so it guesses worse on the te

https://preview.redd.it/uyjw2lcrvkuh1.png?width=1536&format=png&auto=webp&s=ce991be22faec3de5cbaf39119c14782b6ed0af3 link to article TLDR; new microsoft surface and nvidia rtx spark laptops are starti

Basalt: Blackwell inference engine Hi all! So over the last week I've been working on a Strata fork that's heavily tuned and can achieve throughput up to 2.6x what Strata usually does on the same weig

I stumbled across this as I was currently having glm5.3 flash run only around 10tok/s basically unusable. With Maya i am running 30 tok/s now. GLM5.3 feels even with this low quant like a much more en

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you