Search: “chat”

Benchmark results: what is the best and fastest engine to run Qwen3.8-27B on macOS

WARNING: I think I made a mistake with omlx, and its results below seem to be without any kind of acceleration. I am rerunning them with Lightning MTP and ANE support for prefill. Will update the post

23 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Built a local memory engine for Ollama that runs in <1.2GB VRAM without vector DBs (Hillock v0.5)

DGX agent

Hey r/Ollama, If you run local models on mid-range hardware, you probably know how fast VRAM disappears when you try to set up document memory or local RAG pipelines. I built Hillock (https://github.c

local-air-ollama

23 Aug 2026

HOLY SH*T, I BUILT A F**KING COMPANY INSIDE GROK BOT i split the operation into separate roles for research, writing, outreach, ops, finance…

HOLY SH*T, I BUILT A F**KING COMPANY INSIDE GROK BOT i split the operation into separate roles for research, writing, outreach, ops, finance and support. each agent owns one narrow job 6 agents → 1 sh

agentselon-musk--x

23 Aug 2026

Tested in Coding: Q8_K_XL Qwen3.8 27B vs BF16 Qwen3.6 27B

Further to my last post, https://www.reddit.com/r/LocalLLaMA/comments/1vldngi/tested_in_coding_bf16_muse_glimmer_vs_bf16_qwen36/, I bring this requested comparison of Qwen3.8 27B. Context: Both models

23 Aug 2026

Industry

A year ago today we published The Last Economy by @EMostaque. Since then, scores on the hardest AI benchmarks have risen sharply, while the …

A year ago today we published The Last Economy by @EMostaque. Since then, scores on the hardest AI benchmarks have risen sharply, while the cost of useful intelligence has kept falling. The book asks

industryemad-mostaque--x

I benchmark DFlash 2 (PR build) in llama.cpp on Qwen 3.8 27B against all speculative methods for 3 days. 2.26x on 100 real coding prompts, 4.68x with one n-gram drafter on top. Up to 8x on specific cases.

Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl

I'm building a virtual world that only AI agents can join. They all keep dying.

Home page for humans: https://parker-lambert-sandbox.com/robot-world/ Home page for AI agents: https://parker-lambert-sandbox.com/robot-world/start/ If you're a human, you can watch, but not interact:

model-releasesr-chatgpt

Sharp template to NInfer: -42% output tokens, same speed

Sharp v22.1 is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. NInfer is a hyper-tailored inference engine that only runs certain Qwen models on

FleetSieve: Decision-Critical Profiling for SLO-Aware LLM Fleet Configuration

model-releasesarxiv-cs-lg

arXiv:2608.19659v1 Announce Type: new Abstract: Choosing tensor-parallel (TP) degrees and replica counts for an LLM serving fleet is difficult because performance is not monotonic in TP and the feasib

From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation

arXiv:2608.20195v1 Announce Type: cross Abstract: Technical documentation is written for human developers, but an increasing share of software changes is now authored by autonomous coding agents. Whic

agentsarxiv-cs-ai

I feel like I finally graduated.

I finally made the move from LM Studio to vLLM thanks to this post https://www.reddit.com/r/LocalLLaMA/s/NmS9CgHvqz. I may not know what it all means yet but I’m going to start diving into the docs to

Qwen3.8-27B at 262K context on a Strix Halo + RTX 3090 Ti: 9.5 -> 153 tok/s, and it beats a dual-3090 vLLM box on HumanEval

Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128

We built a local AI work tool that runs Qwen3.5-35B-A3B on a 16GB Mac

Hi everyone! I’m an intern at Icosa, a startup focused on making local AI accessible. We just released the first version of Zeno, our local AI product, and I thought it might be of interest to this co

3 days benchmarking most llama.cpp flags on my weird 40gb vram laptop + tb4 egpu setup. Got +70% generation, +40% prefill, 60k more context, and filed a bug in llama around MTP. What I learned.

tldr: went from 16~ t/s to 27~ t/s generation. got my usable context up from 220k to the full 262k without sacrificing anything. prefill also increased from 376 to 573 command I ended up with, fwiw: l

grokbot it's an agent with its own identity, its own computer, and it stays on when you're not here's what 'active AI employee' actually loo…

grokbot it's an agent with its own identity, its own computer, and it stays on when you're not here's what 'active AI employee' actually looks like in the demo: - chief of staff agent - checks in on y

agentselon-musk--x

Ling-3.0 released all 6 base checkpoints: 2 sizes × 3 stages

AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ

Qwen3.8-27B scored 29/30 on AIME 2026 with FP8 + xhigh reasoning — BF16 vs FP8 results

I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium

Applications

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

arXiv:2608.18931v1 Announce Type: cross Abstract: Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over part

applicationsarxiv-cs-ai

What do you think of my Modelfile for Qwen3.8

Hello, any params I missing or mis-tuning for my coding model ? FROM ./models/unsloth/Qwen3.8-27B-Q5_K_M.gguf # Generation parameters optimized for logical reasoning and coding tasks PARAMETER tempera

CABLE: Extending the Reach of Memory Retrieval via Complementary Antecedent-Based Linking and Expansion

model-releasesarxiv-cs-cl

arXiv:2608.17911v1 Announce Type: new Abstract: As LLM agents operate across structured workflows and sessions, preserving long-term history does not ensure that later contexts can recover relevant ev

I might have found the perfect config parameters for qwen 3.8 27b

Hello everyone, tried so hard to optimize my config and finally I simply get up to 70 t/s with q6 variant. And wanted to share with you guys so that other people with the same setup can enjoy. Please

Same GRPO recipe on three from-scratch LLMs (353M/316M/672M) gave three different outcomes, with no clean relationship to scale [P]

model-releasesr-machinelearning

I trained three LLMs from scratch in raw PyTorch then post-trained each one with SFT and then GRPO. Same process every time: same synthetic arithmetic curriculum, same reward function, same hyperparam

The AI inference race moves beyond GPUs to reshape data center infrastructure

AI inference infrastructure is becoming a system-level challenge as organizations move generative and agentic applications into production. Graphics processing unit performance remains essential, but

agentssiliconangle

Token Optimization and Context Window Management in Multi-Agent AI Workflows

model-releasesarxiv-cs-ai

arXiv:2608.17188v1 Announce Type: cross Abstract: Multi-agent AI workflows are limited not only by model quality but by token cost, latency, and context-window quality. This paper presents a practitio

DeepSeek V4 Flash 0731 on Strix Halo: draft model, n_max sweep, and a launch line that actually helps

I know the front page is completely dominated by the Qwen3.8 27B. I know it's a fantastic model and everyone is rightfully discussing it, but I wanted to shift gears and talk about another model I’ve

Don't ignore llama.cpp RPC with old hardware. Results of a 5070 Ti and 1080 Ti over gigabit ethernet: it's actually functional.

Results up front: I had to prioritize prefill or token generation - there was no happy medium. Using UD-Q4_K_XL, q8 kv cache, and 96k max context: focus on generation (MTP = 2): 350 pp and 36 tg @ 12k

I pushed Qwen3.8-27B to 124 tps on a single request on a RTX 3090

Two days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak) - yesterday's update took that to 99 tps single-user / ~1,000 tps at 64 concurr

I'm going to optimize our AI co-work experience

Hey everyone, I am trying to make a LLM interface optimized for AI-cowork experience. Heres what I've thought of and tried to fix: 1)You shouldn't have to copy and paste context for the AI model. 2)Yo

This 1 hour workshop on context engineering will help you build agents that hold long conversations and cut your token cost by up to 90%. La…

This 1 hour workshop on context engineering will help you build agents that hold long conversations and cut your token cost by up to 90%. Last month we gave it live at the AI Engineer World's Fair 202

model-releasesswyx--x

After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)

Following up on my previous post about my budget server setup (Intel N100 + RTX 5060 Ti 16GB), a few of you asked for a deeper dive into my actual inference config and real-world agentic performance.

Applications

Building AI-Intensive Software with AI: Early Results and a Cautionary Tale on Measuring Development Cost

arXiv:2608.13730v1 Announce Type: cross Abstract: Empirical reports on the true cost of AI-intensive software development remain scarce, and the few that exist are easy to get wrong in ways that never

applicationsarxiv-cs-ai

Watching a team of bots split up a large game project based entirely on their own experience and specialties with little input from me is pr…

model-releasesnous-research--x

Watching a team of bots split up a large game project based entirely on their own experience and specialties with little input from me is pretty crazy (and just a bit creepy) to watch. I made a short

Why do people like coding harnesses like opencode etc instead of an IDE?

Just curious - I like to be able to see and manage the scripts my agent is working on. I find stuff like Claude Code and Open Code useful for doing stuff on my linux box but I don't understand why peo

5090: Windows or Linux for Qwen3.8.27b

I've got a dedicated AI rig sitting here with a RTX 5090 and 96GB RAM and for the past few years have been using Windows 11 and primarily LM Studio, but have also used vLLM, llama.cpp and Ollama. With

16 Aug 2026

Qwen 3.8 27b with DSH(DeepSeek Harness) is Amazing!! Experiences so far and perfomance.

https://preview.redd.it/wkg27e152qjh1.png?width=853&format=png&auto=webp&s=2e3f8b11ea6393041f501e95c5835f9bea0245dd So ive been trying different harnesses and coding agents with the new qwen 3.8 , and

16 Aug 2026

Local Ai

Budget AI GPUs - is there a middleground between 3060 and 3090?

I'm currently building a very budget-oriented AI / homelab PC using used parts. I've been saving up (I'm a student), and I'm working on a setup costing around 330€ total. The specs are: Xeon E-2124: N

local-air-ollama

Research

DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees

arXiv:2608.13524v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters furt

researcharxiv-cs-lg

Qwen3.8-27B is now up to ~3× faster on Apple Silicon with mlx-dspark

mlx-dspark is an MLX port of DeepSeek's DSpark speculative-decoding drafters (the DeepSpec release), plus z-lab's DFlash, with one lossless verify loop. v0.10.0 adds Qwen3.8-27B via RadixArk's drafter

Safety

Research Assistant: AstraZeneca's Agentic System for R&D

arXiv:2608.12395v1 Announce Type: new Abstract: We describe Research Assistant, an internal LLM-based system developed at AstraZeneca to help scientists and clinicians explore biomedical questions acr

safetyarxiv-cs-ai

What can I realistically do?

I am currently building up a local assistant profile on my MacBook Pro M2 with 32gbs. With Claude I am building out this Hermes agent to be my assistant. I am using Qwen3.6 A3B 4bit. We have Frankenst

A great read if you an AI dev. Current context compactors retain 17% of the standing rules users give them. Session Constraints are instruct…

A great read if you an AI dev. Current context compactors retain 17% of the standing rules users give them. Session Constraints are instructions like 'do not delete any emails until I confirm' that bi

agentsdair-ai--x

Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction

model-releasesarxiv-cs-ai

arXiv:2608.11242v1 Announce Type: cross Abstract: When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We identify a class of user-issued instruction

Research

QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving

arXiv:2608.12121v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) repeatedly prefills identical text chunks across queries, incurring redundant computations. Position-Independent

researcharxiv-cs-ai

TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation

model-releasesarxiv-cs-ai

arXiv:2608.11236v1 Announce Type: cross Abstract: Roleplay evaluation should do more than assign a single score: it should reveal which role requirements were tested, which failed, and which dialogue

We tested out Anthropic’s J-Lens probe at two open models: Kimi K3 and Qwen 3.5-9B. By looking at their internal states, we found that the m…

model-releasesfireworks-ai--x

We tested out Anthropic’s J-Lens probe at two open models: Kimi K3 and Qwen 3.5-9B. By looking at their internal states, we found that the models are setting up the right vocabulary before a single to

Applications

ConVAWG: A Retrieval-Grounded Framework for Controlled Synthetic Dialogue Generation in Violence Against Women and Girls

arXiv:2608.11200v1 Announce Type: cross Abstract: Synthetic dialogue generation offers a way to study conversational dynamics in sensitive domains where real data are difficult to access, release, or

applicationsarxiv-cs-ai

12 Aug 2026

Everyone else is talking about building ASI to like monopolize b2b saas and Elon is talking about building a kardashev II sentient sun

model-releaseselon-musk--x

Everyone else is talking about building ASI to like monopolize b2b saas and Elon is talking about building a kardashev II sentient sun Media It’s always funny how people in SF twitter bubble will say

12 Aug 2026

Safety

From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop