All entries

server: report the isolate working directory from get_info (#26773) server: report the isolate working directory from get_info Without an explicit cwd, get_info fell back to the server process working

I’m planning a dedicated home AI server, mainly for local LLM inference, agents/tool use, Docker services, and eventually larger MoE models with CPU offload. My plan is to start with 2x 16GB GPUs = 32

Over the past few months I have been building a CPU-first inference engine from scratch in pure C99 (no Python, no CUDA, no BLAS, just GCC and make). The focus has been running 1.58-bit ternary models

I was wondering what a minimal coding agent implementation would look like that can be used like Claude Code or Codex Not feature-by-feature of course but basically stripping everything out that is no

CPUs and the rise of neurosymbolic AI by @garymarcus (with implications for what Artificial Intelligence implies about natural intelligence -- Gary's & my interest ever since we worked together 37 yea

Currently serving 200 tps+ output speed for DeepSeek-V4-Flash on Ollama's cloud with zero data retention (ZDR). Have an amazing weekend 🫡 DeepSeek-V4-Flash-0731 is now fully rolled out as the new defa

I’m running DSV4F 0731 on dual spark, and honestly… wow. It’s an absolute workhorse, and the benchmarks are real. Everyday tasks with Hermes agent? Effortless. Coding tasks with OpenCode? I’m genuinel

Don't miss the bit where OpenAI first found out they were responsible for the Hugging Face attack when they reached out to HF to get one of their credentials revoked and HF told them it had already be

Disclaimer - no LLM was used to write this post/note As larger post about my setup will come later, want to give heads-up to folks who use VLLM and >= 2 GPUs. So I have pretty meaty server (8 channel

Hey everyone, I could use some advice on setting up speculative decoding correctly with llama-server. My Hardware: GPUs: RTX 4090 + RTX 6000 Pro (120GB total VRAM) RAM: 32GB I am currently testing the

The global buildout of AI infrastructure reached a new milestone today — Firebird, an emerging AI cloud, launched the CIS region’s largest AI factory in Armenia, establishing a new AI computing hub po

Artificial intelligence can be technologically transformative and still produce a capital bubble. Those two ideas are not in conflict. The bubble bursting does not require AI to fail. It only requires

I discovered recently that Google uses their own TPUs, like tiny ASIC cards like the toy ones that existed for bitcoin. And while it sounds inefficient the fact they use thousands of them because...th

HUD mode Hermes stops being a window you switch to and becomes a layer over the app you're both working in. Or keep it around as a little buddy agent. Ask it random things, drag it anywhere, it's your

I’m building DesktopLab, an open-source local-first control plane for development agents. I recorded the setup boundary that most agent demos skip: DesktopLab detects the host, proposes the supported

I use auto mode for everything and now that will be the default in Claude. Anthropic had to decide whether to prioritize maximization of human control or minimization of risk, and it chose the latter.

What IDE are you using. its another problem area for me . I usually use VSCode , but with local llms I have not found an extension which works optimally VSCode CoPilot chat with Ollama: CoPilot bloats

Imagine image 2.0, non-agentic yet, more to come in a week or two 💙 Announcing Imagine Image 2.0, our next generation image model with precision editing, crisp text rendering, improved factuality, and

(I am not a native speaker, written by myself, so please bear with me) I really want to like DeepSeek-V4-Flash-0731. But it has serious flaws that don't align with the high score on intelligence bench

Firstly a big thanks to the poster 'hellohazine', he basically only removed the multi-lingual fat of the model and just kept the English language intact. It is the exact model, and the rest of the mod

LiteParse can now extract structured data from your PDF in milliseconds: ✅ checkbox states ✅ annotations ✅ vector graphics ✅ word-level bounding boxes It is the most comprehensive, accurate (and fast)

seems to be about as good as a vega 56 with 16Gb of VRAM, is it worth it? (don’t want to deal with NVIDIA drivers on Linux, already have an rx6650xt and might simply use vulkan for llamacpp inference)

This PR should be ready for testing now. I tested with a very small (8B params) sub-model extracted from the original one. Appreciate if someone can test with the bigger model. GGUF(for testing) from

Running across 2 clusters using llama.cpp over RPC too. Both clusters are not enough to hold everything in memory, so main cluster still partially offloads to run. Goal will be to get all the GPUs in

Neat example here of the agents communicating purely through file names, including adding base64-encoded attachments and using 'zz' prefixes to ensure their new message sorts to the bottom of the list

My comment on Now we have a timeline of the OpenAI accidental attack against Hugging Face — Hacker News.I think one of the most interesting details here might be tucked away in that first bulletin poi

I am wondering if anyone can give opinion on if Ollama Cloud pro or max plans are worth it. Id be looking to use it with Kimi K3, Qwen 3.8 and Deepseek v4flash for now. Wondering if it would be better

@OpenAI oo claude code has this now!!! need to try https://x.com/ClaudeDevs/status/2085817074816070014 New in Claude Code: your sessions can now message each other. Instead of having to re-explain you

OpenAI Group PBC today disclosed that one of its unreleased large language models may pose a significant cybersecurity risk. The algorithm, which is known as Astra, was first detailed last week. OpenA

Basically what the title says. For me, it would always crash and burn trying to use tensor split. Apparently, there's some bug where GPU memory gets corrupted with the default microbatch (512) or high

Hi everyone, I'm a systems analysis student researching a problem a lot of you probably know well: how much you actually trust the published VRAM/RAM requirements for open-source models before trying

I compared Qwen 35B-A3B MoE against Qwen 27B dense on a series of local coding-maintenance tasks. On my R9700/llama.cpp setup, the MoE model generated about 3.9× faster (~116 vs ~30 tok/s), but the co

I've been tuning my new Radeon AI Pro R9700, and figured that this would be useful information for people who are trying to optimise their setups. I'm pretty happy with these results and looking forwa

Not the biggest or shiniest, but it's mine From gaming machine inference on the original llama models, to a 4x RTX 6000 Pro Max Q + 4x 3090s local AI cluster. Pictures are in reverse chronological ord

MiniMax-H3 in ComfyUI 0.30.0, RTX 4080 16 GB (224 W cap). int8 DiT + int8 Qwen3-VL-32B text encoder. Turbo LoRA @ 0.9, euler + simple, 6 steps, no CFG. MiniMaxH3ReferenceToVideo with 3 reference image

Looking for V100 users to share your config and it's performance. GPU: Tesla V100 PCIE 32Gb Qwen3.6 27B Q4_K_M + Q8_0 MTP 128K context length Pi coding agent llama.cpp model preset: [*] spec-default =

François Chollet commented that claims the demise of Google were greatly exaggerated, cautioning against undervaluation. According to a Polymarket report, Sergey Brin is expected to take direct oversi

Researchers from TogetherAI analyzed DeepSeek V4 Flash and GPT‑5.6 Luna on the DeepSWE benchmark. The study found that employing a DeepSeek‑first cascade with test‑suite verification solved more tasks

Emad posted that Stable Diffusion v1.4 reached the end of its training cycle roughly four years before the post was published. Greg Brockman added that GPT‑4 similarly completed training around the sa

When you move a model into production, you want the quality you evaluated to carry through the serving stack. @Kimi_Moonshot benchmarked Kimi K3 across major inference providers, and Together AI ranke

You may have been told to watch this video about the OpenAI AI hack. You really should, even if you don't usually care about tech stuff. If nothing else, click this link to the 18 minutes in & see how

Build a 100% offline fast Retrieval Augmented Generation (RAG) system that runs without an internet connection, without cloud APIs, without OpenAI/Ollama Published a video where you can build a fully

Follow-up to my original Spectrum MiniMax H3 post: https://www.reddit.com/r/StableDiffusion/comments/1vf1ze3/spectrum_acceleration_for_minimax_h3_in_comfyui/ In that first post, I released the MiniMax

arXiv:2510.06612v2 Announce Type: replace Abstract: Speech-driven talking face synthesis (TFS) focuses on generating lifelike facial animations from speech input. Current TFS models perform well in En

arXiv:2608.05471v1 Announce Type: cross Abstract: Prenatal ultrasound imaging is key for assessing fetal health, but AI progress is limited by scarce, privacy-restricted, and hard-to-annotate datasets

arXiv:2511.13480v2 Announce Type: replace-cross Abstract: This study focuses on understanding the complex dynamics between humans and AI systems by analyzing user reviews. While previous research has