Search: “chat”

Hi, I see that vllm released d3 (a family of decision models) and llama.cpp already has support for them. I want to try them out. Are decision models sensitive to quantization? I know that for autocom

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

tl;dr: running Qwen3.8FN Q4 locally is cheaper for me than every cloud model except Luna and the newer Haiku at mid-peak and peak electricity pricing where I live, and becomes the absolute cheapest mo

I started Flama in 2018 to take a trained model to a production API with as little ceremony as possible. With 2.0 LLMs became first-class alongside classic models, and together with Miguel Durán-Olive

After much testing and talking to actual users across all three proactive AI agents: Muse - best one for your parents, heavy IG/FB users, people who like or need easy setups, people who don’t want to

[UPDATE / RETRACTION: The '100% GTO' claim had a prompt-leakage bug] We patched the repo to pass strictly neutral states, here are the real, honest numbers: • Latency: Still ~2ms (ONNX + Absorbing Mar

I built BitRun because I wanted to run 1-bit ternary LLMs on ordinary laptops without melting my battery, spinning up a dedicated GPU, or wrestling with complex build environments. Right now, BitRun g

I've been using ChatGPT Voice for years, particularly during my commute to work. Until recently, I've almost exclusively used Standard Voice Mode, but a recent issue has made it practically unusable.

arXiv:2610.11351v1 Announce Type: new Abstract: Large language models (LLMs) increasingly act as agents with little human oversight, so potential mistakes they make can go unnoticed. Users then depend

New Show Announcement: High Taste Testers Good food and good ai convos Pilot episode 🔜 http://youtube.com/@latentspacepod ft: @davis7, @dexhorthy @AdjectiveAllison, @altryne, @benhylak on DevDay/Dots,

Hey everyone, Sharing an educational tool called AskCyph Health Explorer to explore how single-letter DNA mutations lead to disease, how they alter 3D anatomy, inheritance probabilities, and available

arXiv:2610.13623v1 Announce Type: cross Abstract: Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and busine

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

If an agent saves an engineer 2 hours to implement a feature but collectively costs the team 3 hours in alignment and review, the ROI is negative. Isolated chats are suboptimal in team environments be

Banger paper from Google DeepMind on co-evolving an agent's harness and its weights. (bookmark it) If you already log your harness changes and results, those logs can become training data for the mode

This continues the small-model browser experiments from the previous posts. Two standard, off-the-shelf models ran locally on the same Samsung Galaxy S21: Gemma 4 E2B, Q3_K_M as the reasoner Qwen3.5-0

Quartermaster (my local inference manager: one server that swaps llama.cpp, image, audio and video models in and out of VRAM) can now help you server and manage decision models: Auto-discovered. Drop

llama.cpp now has a Windows app 🦙 Llama Windows makes it easy to run models locally, with optimizations for NVIDIA RTX Spark. 📥 Browse and download models from Hugging Face 💬 Chat with local models ri

Brilliant paper on agentic continual learning from Meta and MIT. (bookmark it) Agent memory harnesses usually only write forward. A lesson goes into the bank and is never checked against whether it he

I've been working on an inference engine called jitllm. The idea came from my background in operating systems and JIT compilers: generate kernels for whatever hardware is there, and treat layers a bit

Interesting new research from Meta Superintelligence Labs It discusses how reward hacking affects coordination in multi-agent coding systems. Coding agents on a team share code, memory, and plans, so

arXiv:2610.17321v1 Announce Type: cross Abstract: As demand for LLM inference grows and consumer GPUs grow more capable, edge deployment is becoming increasingly attractive. WebGPU is compelling in th

chat: replace output string with parts (#30389) chat: replace output string with parts reserve first Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5563975

Giving my Dot a dedicated email address turned out to be surprisingly painless & solved a few problems around how to relate to a personal agent. I don't want it using my email or drafting messages on

Each villager in Pepperton is a different model — a mistral, a qwen3, a qwen2.5, a phi4-mini, a llama3.2 — because model families have genuinely different temperaments, and the friction between them i

I released LLM 0.32 this morning, the most significant new version of LLM since the initial launch of the project. The new version includes support for visible reasoning traces, server-side provider t

A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I al

January 2025 I made the switch from OpenAI to Claude. It was awesome. No more annoying GPT Chats, I dove into the wonderful world of Claude Code, learned vibe coding with Github, Vercel on Next.js. Wa

Note - This is translated from the actual blog link right at the bottom. A few days ago, DeepSeek v4.1 was released. It raised the ability of small models to a new level. AI is improving much faster t

arXiv:2609.16517v1 Announce Type: cross Abstract: Resume screeners must infer job-relevant competence from resumes whose presentation can vary substantially in wording, structure, stylistic polish, an

arXiv:2609.26147v1 Announce Type: new Abstract: Modern LLMs are deployed as families of post-trained variants (base, instruct, chat, code) derived from a shared set of pre-trained weights. We present

EXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without tra

Automated lint: 14 errors, 54840 warnings, 3 info

We’re building Copilot as a new OS for work that spans every model, every form factor, and every task. Today, we’re announcing our biggest update to Copilot to date, bringing four things together: · A

Automated lint: 17 errors, 59669 warnings, 3 info

xAI's Grok 4.7 is now available on Amazon Bedrock: a frontier model for coding, long-running agents, and knowledge work. It offers a 500K token context window and four configurable reasoning effort le

Automated lint: 19 errors, 64915 warnings, 3 info

New Show Announcement: High Taste Testers Good food and good ai convos Pilot episode 🔜 http://youtube.com/@latentspacepod ft: @davis7, @dexhorthy @AdjectiveAllison, @altryne, @benhylak on DevDay/Dots,

If an agent saves an engineer 2 hours to implement a feature but collectively costs the team 3 hours in alignment and review, the ROI is negative. Isolated chats are suboptimal in team environments be

I built a @Grok @Bot that watches SPCX on its own — fully automated, no manual work. I’ve been running it for over a month and fine-tuned it several times. This version is very good. Every weekday bef

For the last few months, I ran DeepSeek v4.0 flash (NVFP4). First 0731, then visionexp because it was a free improvement. I got around 65 tps decode and almost 2k prefill, and ran 4-5 agents in parall

Learn to optimize your own harness, folks. You can squeeze much more performance and value from a harness. This work claims that a self-modified harness matched Codex for 4.03. Specifically, a coding

Huge release from EverMind AI. Raven is an open-source multi-agent system that builds a separate harness for each model and domain, evolves those harnesses from their failures, and coordinates them th

Disclosure: This unit was provided free of charge by Orange Pi for testing. No editorial review, no preconditions, no script. All data, bottlenecks, and thermal behavior are reported directly from har

Hi all, having searched google and asked several AIs without success, maybe s.o. can help. I have 2 x dgx spark in a cluster. deepseek-flash is running successful. Now I want to test qwen3.8-flash-nex

It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu

Last month I created a portable memory format as a .txt file as a small project. I named it as: Memory Journal, in a shape of a skill card. It's essentially a large prompt wrapped in a text file. Init

Learn to build custom harnesses, folks! Here is an impressive paper showing the impact of a good custom harness on long-context work. GLM-5.3-Flash goes from 4.4% to 61.1% exact match on OfficeQA Pro

Basalt: Blackwell inference engine Hi all! So over the last week I've been working on a Strata fork that's heavily tuned and can achieve throughput up to 2.6x what Strata usually does on the same weig

I stumbled across this as I was currently having glm5.3 flash run only around 10tok/s basically unusable. With Maya i am running 30 tok/s now. GLM5.3 feels even with this low quant like a much more en

Recommended NVIDIA paper on memory-efficient post-training RL. A very interesting technique for scaling RL for LLMs. The main takeaway is that you can update all of a model's weights with RL and still

Learn how to deploy decision models with llama.cpp on Amazon SageMaker AI. This technical how-to serves OpenAI-compatible chat completions with the AWS Deep Learning Container for llama.cpp, then exte

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

tl;dr: running Qwen3.8FN Q4 locally is cheaper for me than every cloud model except Luna and the newer Haiku at mid-peak and peak electricity pricing where I live, and becomes the absolute cheapest mo

llama.cpp now has a Windows app 🦙 Llama Windows makes it easy to run models locally, with optimizations for NVIDIA RTX Spark. 📥 Browse and download models from Hugging Face 💬 Chat with local models ri

New impressive work from Microsoft and collaborators. And it's one of my favorite recent topics: RL environments. They show that Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 by trainin