Search: “chat”

Qwen3.8-27B is amazing, but it’s slow. A stronger 35B-A3B Mixture of Experts-coder that can run and solve real codebase issues fast (even on constrained hardware) is a valuable addition to the arsenal

Natasha Singer / New York Times: A look at the playbook tech giants like Google, Microsoft, and OpenAI use to shape American schools to their benefit, and the pushback against unproven AI tools — One

Adding a verifier improves every agentic AI system. But frontier verifiers are expensive - so we trained our own. Matching quality, at a fraction of the cost. Thread below 🧵 1/ Your agent already foun

WARNING: I think I made a mistake with omlx, and its results below seem to be without any kind of acceleration. I am rerunning them with Lightning MTP and ANE support for prefill. Will update the post

Hey r/Ollama, If you run local models on mid-range hardware, you probably know how fast VRAM disappears when you try to set up document memory or local RAG pipelines. I built Hillock (https://github.c

HOLY SH*T, I BUILT A F**KING COMPANY INSIDE GROK BOT i split the operation into separate roles for research, writing, outreach, ops, finance and support. each agent owns one narrow job 6 agents → 1 sh

Further to my last post, https://www.reddit.com/r/LocalLLaMA/comments/1vldngi/tested_in_coding_bf16_muse_glimmer_vs_bf16_qwen36/, I bring this requested comparison of Qwen3.8 27B. Context: Both models

A year ago today we published The Last Economy by @EMostaque. Since then, scores on the hardest AI benchmarks have risen sharply, while the cost of useful intelligence has kept falling. The book asks

Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl

Home page for humans: https://parker-lambert-sandbox.com/robot-world/ Home page for AI agents: https://parker-lambert-sandbox.com/robot-world/start/ If you're a human, you can watch, but not interact:

Sharp v22.1 is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. NInfer is a hyper-tailored inference engine that only runs certain Qwen models on

arXiv:2608.19659v1 Announce Type: new Abstract: Choosing tensor-parallel (TP) degrees and replica counts for an LLM serving fleet is difficult because performance is not monotonic in TP and the feasib

arXiv:2608.20195v1 Announce Type: cross Abstract: Technical documentation is written for human developers, but an increasing share of software changes is now authored by autonomous coding agents. Whic

I finally made the move from LM Studio to vLLM thanks to this post https://www.reddit.com/r/LocalLLaMA/s/NmS9CgHvqz. I may not know what it all means yet but I’m going to start diving into the docs to

Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128

Hi everyone! I’m an intern at Icosa, a startup focused on making local AI accessible. We just released the first version of Zeno, our local AI product, and I thought it might be of interest to this co

tldr: went from 16~ t/s to 27~ t/s generation. got my usable context up from 220k to the full 262k without sacrificing anything. prefill also increased from 376 to 573 command I ended up with, fwiw: l

grokbot it's an agent with its own identity, its own computer, and it stays on when you're not here's what 'active AI employee' actually looks like in the demo: - chief of staff agent - checks in on y

AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ

I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium

arXiv:2608.18931v1 Announce Type: cross Abstract: Test-time scaling (TTS) improves language model outputs by spending additional inference compute - generating multiple candidates, searching over part

Hello, any params I missing or mis-tuning for my coding model ? FROM ./models/unsloth/Qwen3.8-27B-Q5_K_M.gguf # Generation parameters optimized for logical reasoning and coding tasks PARAMETER tempera

arXiv:2608.17911v1 Announce Type: new Abstract: As LLM agents operate across structured workflows and sessions, preserving long-term history does not ensure that later contexts can recover relevant ev

Hello everyone, tried so hard to optimize my config and finally I simply get up to 70 t/s with q6 variant. And wanted to share with you guys so that other people with the same setup can enjoy. Please

I trained three LLMs from scratch in raw PyTorch then post-trained each one with SFT and then GRPO. Same process every time: same synthetic arithmetic curriculum, same reward function, same hyperparam

AI inference infrastructure is becoming a system-level challenge as organizations move generative and agentic applications into production. Graphics processing unit performance remains essential, but

arXiv:2608.17188v1 Announce Type: cross Abstract: Multi-agent AI workflows are limited not only by model quality but by token cost, latency, and context-window quality. This paper presents a practitio

I know the front page is completely dominated by the Qwen3.8 27B. I know it's a fantastic model and everyone is rightfully discussing it, but I wanted to shift gears and talk about another model I’ve

Results up front: I had to prioritize prefill or token generation - there was no happy medium. Using UD-Q4_K_XL, q8 kv cache, and 96k max context: focus on generation (MTP = 2): 350 pp and 36 tg @ 12k

Two days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak) - yesterday's update took that to 99 tps single-user / ~1,000 tps at 64 concurr

Hey everyone, I am trying to make a LLM interface optimized for AI-cowork experience. Heres what I've thought of and tried to fix: 1)You shouldn't have to copy and paste context for the AI model. 2)Yo

This 1 hour workshop on context engineering will help you build agents that hold long conversations and cut your token cost by up to 90%. Last month we gave it live at the AI Engineer World's Fair 202

Following up on my previous post about my budget server setup (Intel N100 + RTX 5060 Ti 16GB), a few of you asked for a deeper dive into my actual inference config and real-world agentic performance.

arXiv:2608.13730v1 Announce Type: cross Abstract: Empirical reports on the true cost of AI-intensive software development remain scarce, and the few that exist are easy to get wrong in ways that never

Watching a team of bots split up a large game project based entirely on their own experience and specialties with little input from me is pretty crazy (and just a bit creepy) to watch. I made a short

Just curious - I like to be able to see and manage the scripts my agent is working on. I find stuff like Claude Code and Open Code useful for doing stuff on my linux box but I don't understand why peo

I've got a dedicated AI rig sitting here with a RTX 5090 and 96GB RAM and for the past few years have been using Windows 11 and primarily LM Studio, but have also used vLLM, llama.cpp and Ollama. With

https://preview.redd.it/wkg27e152qjh1.png?width=853&format=png&auto=webp&s=2e3f8b11ea6393041f501e95c5835f9bea0245dd So ive been trying different harnesses and coding agents with the new qwen 3.8 , and

I'm currently building a very budget-oriented AI / homelab PC using used parts. I've been saving up (I'm a student), and I'm working on a setup costing around 330€ total. The specs are: Xeon E-2124: N

arXiv:2608.13524v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters furt

mlx-dspark is an MLX port of DeepSeek's DSpark speculative-decoding drafters (the DeepSpec release), plus z-lab's DFlash, with one lossless verify loop. v0.10.0 adds Qwen3.8-27B via RadixArk's drafter

arXiv:2608.12395v1 Announce Type: new Abstract: We describe Research Assistant, an internal LLM-based system developed at AstraZeneca to help scientists and clinicians explore biomedical questions acr

I am currently building up a local assistant profile on my MacBook Pro M2 with 32gbs. With Claude I am building out this Hermes agent to be my assistant. I am using Qwen3.6 A3B 4bit. We have Frankenst

A great read if you an AI dev. Current context compactors retain 17% of the standing rules users give them. Session Constraints are instructions like 'do not delete any emails until I confirm' that bi

arXiv:2608.11242v1 Announce Type: cross Abstract: When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We identify a class of user-issued instruction

arXiv:2608.12121v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) repeatedly prefills identical text chunks across queries, incurring redundant computations. Position-Independent