Search: “benchmark”

Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic,

What it is: ATH-MaaS/OvisOCR2 - a 0.8B document-parsing VLM post-trained from Qwen3.5-0.8B (SFT + RL + OPD), Apache 2.0, runs on vLLM 0.22.1. One prompt per page image -> complete markdown (HTML table

Introduction RxBrain (Hy-Embodied-RxBrain-1.0) is a unified multimodal foundation model for embodied cognition — a single model that couples language reasoning with visual imagination to deliver three

This Reddit thread from r/ollama features a user seeking community recommendations for motherboard, CPU, and RAM components to build a system optimized for running Ollama locally without a dedicated G

No agent maintained moral reasoning consistency across scenarios. Findings from a structured study with 11 agents on classic ethical dilemmas [R]

This r/ollama post describes a technique where an LLM is used to automatically tune its own llama.cpp runtime flags — such as parameters related to GPU offloading, KV cache quantization, batch sizes,

This r/MachineLearning discussion thread centers on a hands-on workshop covering context engineering — the process of designing and optimizing the information an AI agent sends to and receives from a

**llm-server v2** is a community-developed local LLM server tool that introduces AI-driven self-tuning capabilities, automatically identifying and applying optimal performance flags for **llama.cpp**

First, kudos to Strata. It inspired this project, and while I was building it Strata got much faster and added NVFP4 support, which was my main reason for starting. By then I was too far in to stop, s

https://preview.redd.it/zyoo32u7hnvh1.png?width=1400&format=png&auto=webp&s=f0dc824a831d10abc6b824afabb98c64e7fa9d40 from Mistral: Timestral is a family of time series foundation models for zero-shot

Going to try these out to see if they have the same looping behaviors that other Bonsai models have had. Still requires their fork of llama.cpp (branch prism) so dont expect it to work with vanilla ll

https://preview.redd.it/kudj7xii5ivh1.png?width=1508&format=png&auto=webp&s=07712e81310b81aceb2cf8f4d8a7c2eadeb0c304 Just built a new PC, full parts list is here. I am using Deadcode's Radiance infere

I built BitRun because I wanted to run 1-bit ternary LLMs on ordinary laptops without melting my battery, spinning up a dedicated GPU, or wrestling with complex build environments. Right now, BitRun g

All seems a bit pointless now. Either pump out AI papers and neglect your own development, or take it slow and fall behind in the publishing race. I already have two papers made largely without LLMs (

As strata is somewhat new, I assume that some people din't yet learn how to finetune it for best performance on their machine. I find documentation a bit hard to navigate (use your agents for this!),

[UPDATE / RETRACTION: The '100% GTO' claim had a prompt-leakage bug] We patched the repo to pass strictly neutral states, here are the real, honest numbers: • Latency: Still ~2ms (ONNX + Absorbing Mar

Here is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo

I built a desktop app harness that keeps local meeting notes and can use them to autocomplete my typing as well as complement my dictation with additional context via reading my screen or using the pr

This post is to let people know about a workshop/tutorial on Machine Learning that will run on the (blue) math discord server: discord.gg/maths. The place is nice, people are cool and helpful, and as

https://preview.redd.it/p6i9qifyy8vh1.png?width=2069&format=png&auto=webp&s=2b1e3c7c4fc95f12f0f7d417a4d21410aebfbd35 The 3090 Founders edition has a reputation for hot VRAM. Since LLM token generation

Hi, I see that vllm released d3 (a family of decision models) and llama.cpp already has support for them. I want to try them out. Are decision models sensitive to quantization? I know that for autocom

This is an update to my previous post which can be found here: https://www.reddit.com/r/LocalLLaMA/s/n7sgRZZc4z Since my last post I have continued working on the conversion process and also convertin

I came across this blog post from OrcaRouter: https://www.orcarouter.ai/blog/qwen-3-8-next-vs-qwen-3-8 Not only is the blog post nonsensical in some parts, it's also completely without any value. It b

https://preview.redd.it/3ekngxk224vh1.png?width=1080&format=png&auto=webp&s=9bfe5c92033e17d5b2fb4529f1320c99455a138b I built a GPU compiler and wrote an article about it, focusing on concepts shared b

Paper: Looped Diffusion Transformer (arXiv 2609.40305, 30 Sep). Instead of making a text-to-image DiT bigger, it runs a shared group of transformer blocks several times inside each denoising step. The

EXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without tra

If you build on open-weight reasoning models, you have probably wanted a 'how sure is the model about this answer?' signal that does not cost 5x sampling, a judge model, or a labeled dataset. Hence, w

Just wanted to share a build I am doing for a client. $18k. Specs: Threadripper 9970x 128gb DDR5 ECC 5600 4x AMD Radeon R9700 AI Pro 32gb(128gb total VRAM) 1600W PSU(GPUs are undervolted and under 210

https://preview.redd.it/r2ledkyuntuh1.png?width=1684&format=png&auto=webp&s=9ed4518b2d0f20f44835ebbfac605581da186a57 AesCode generates information-rich visual artifacts such as slides, posters, and da

Alright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less v

I ran a small test for Dory to see whether a local model could update one fact in a synthetic notes file without losing any of the others. I asked Gemma and Qwen to rewrite the file, return a patch, o

Hey everyone! 👋 Feeding multi-gigabyte files (SQL dumps, videos, audio, PDFs, and spreadsheets) directly into LLMs often causes context window overflow, GPU VRAM bottlenecks, massive token bills, and

Disclosure: I work at H2O.ai. We released H2O-Lightning-4B, an open-weight (Apache-2.0) model for the 'decisions API' style of inference that Jev made popular: you send a state plus typed questions (p

Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints remains challenging. While elastic architectures

Hello all! Thank you to everyone that contributed their feedback and notes for LlamAmpere the last time I posted. I've continued to chip away at improvements for the 3090 crowd -- this release is mode

Figure 1b from our paper Disclosure: I'm one of the authors (Microsoft). The paper, code, dataset are public and ThinkingBox is on Hugging Face OpenEnv as well. Raw evaluation trajectories are not rel

I've been building ANCHOR, a web automation agent that runs entirely on Ollama. You describe a task in plain language ('Register Maria Silva, IT department, contractor, and save'), point it at a site,

There are way too many forks to keep track of, so I've gotten lost. As far as I can tell, Radiance VLLM is best for models that fit in GPUs while some form of llama.cpp is probably best for MOE RAM-sp

Disclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is

The other week I posted about Jev vs. Kev compared and since then, OpenAI released the decisions endpoint, Cloudflare released Clef and many here asked about Laya as well. This time we compared six po

Hey everyone, Jovan from UkisAI (Swift Qwen) here! For those who don't know us, UkisAI is a small lab making tiny frontier LLMs, tools and datasets (+doing it open-source!). I'm one of the guys runnin

We test three state-of-the-art multi-modal large language models (GPT-4o, Gemini-1.5-Pro, Gemini-1.5-Flash) and find that they fail dramatically when generalization requires that the rules of the game

https://preview.redd.it/owhrvbbiu2uh1.png?width=4096&format=png&auto=webp&s=55d816753e7adee0e1391c9f581052b4bde44da7 d1-omni-600M d1-omni-600M is a 600M parameter decision model built on LFM2.5-Encode

https://preview.redd.it/pr5bl6rcezth1.png?width=834&format=png&auto=webp&s=dca7440c56e169b53a586182368829f848805c9a They released FastH3 V2 three weeks ago: https://www.reddit.com/r/StableDiffusion/co

Disclosure first: I work at Vstorm and I build AgenticOS, an open-source (Apache-2.0), self-hosted platform for building AI agents in a browser. This post was drafted with an LLM's help and edited by

I don't know if anyone else has ran into these issues but when using Qwen Flash Next, my confidence in it at iq4_xs is high but not 100%. I notice it not following instructions, hallucinating more oft