Knowledge catalogue
Alright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less v
Based on my experiments and feedback from claude, it seems extremely unlikely and even if I get it working, not practical because of minimum context. But just asking here as a last effort. I'm able to
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud
As i use a rather tiny setup 12gb vram gpu and 32bg ram, i was wondering: What setup do you use, which harness? and how do you customize it to run qwen 3.8 flash next swift with a high context length
webAI released TwIL LM3 Pro on September 30. I haven't seen it posted here yet, so I went through the model card, and their comparison chart is attached. It's a 3.66B model built on IBM's Granite 4.2
Hi, Because sometimes you need both speed AND quality, I made my own Qwen 3.8 27B Swift 1.5 quant. It's an 8 bits AWQ + GPTQ quant. Made with love and precision (read the model card) on my 4 RTX 3090,
https://preview.redd.it/9k5fw4j47puh1.png?width=545&format=png&auto=webp&s=ff0a6ff6fb69ae9a9bcd579bd612c553fc3e2a00 https://preview.redd.it/xcwzlmo77puh1.png?width=664&format=png&auto=webp&s=c8a637e2e
sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54540406 macOS/iOS: macO
opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5457
cmake(s390x): disable z17 target for unsupported compilers (#30297) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54655455 macOS/iOS: macOS Apple Silicon
model : support MiniCPM-V 4.7 (#29416) mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_ca
server: leave a busy slot untouched when a request pins it (#30295) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a
Banger paper from Meta on AI research agents. (bookmark it) If you build AI-scientist systems, this one is worth your time. The result: A 27B open model beats the strongest open autoresearch baseline
Basalt: Blackwell inference engine Hi all! So over the last week I've been working on a Strata fork that's heavily tuned and can achieve throughput up to 2.6x what Strata usually does on the same weig
I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared
hello, I currently use swift qwen3.8 gsq rco in iq3 xxs in strata. The model is only used for hermes agent. I have 64GB of ram + 32GB R9700 where the model runs on. But I also have an rx7800xt with 16
Around 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S Then, just after that, I introduced Laya, the
For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a
Finetuned Qwen3.5-4B into a decision model like Jev using @UnslothAI. - 4-bit QLoRA, bf16, single A100, 15 min. - Train/val loss flatten after 2 epochs so most of the learning happens fast. - Trained
A conversational AI plush toy where nothing leaves my network. Everything runs on my LAN: STT: Whisper LLM: Hermes 3 Llama 3.1 8B (Q8_0) TTS: Kokoro Memory: per-person (LangGraph). It knows who it's t
honestly pretty cool to have this on my new tab page just merged in some PRs by others too! you now get Claude API credits with your MAX plans ($100, or 200 matching your plan) every month, use this t
I wanted to compare gemma 4 31b qat q4 and qwen3.8 27b iq4_nl, so I run SWEBench verfied on them. The original gemma 4 chat template doesn't work with preserve reasoning, so google released an update
I was looking into finalizing my daily driver and it was not clear which one to chose. I have a 5060ti 16GB so I picked the following contenders, which seem to be working at a reasonable speed on my h
Hi All: I have an Asus ESC 4000 G3 with 128 GB DDR4 RAM - I tried putting in my V620s but couldn’t put more than 2. Sadly, pivoted to T4s, these are 72 watt passive cards and I thought I could use the
**Summary** * Daniel Dale (@ddale8) comments that the common structure of a claim/fact‑check—“CLAIM: (outline a claim)/FACT: (present unrelated facts)”—is not how reliable fact checks operate. * He e
I ran a small test for Dory to see whether a local model could update one fact in a synthetic notes file without losing any of the others. I asked Gemma and Qwen to rewrite the file, return a patch, o
Spent the past month tweaking and experimenting with many different numbers to achieve 30tps. Hardware: -Rx6700xt 12gb vram (AMD) -2x16 ddr4 3200 ram -r5 5600x -llama.cpp vulkan sdk -window11 (no wsl
Introducing Anatomy Describe a machine in one line. Claude brings it to life. - Interactive isometric wireframes you can rotate, inspect and explore - WebGL shaders for fire, water, plasma and more -
Mistral OCR is on OpenDocRouter! For its price, it is decent over tables and reading order. It's also blazing fast at p50 1.8s per page We're constantly bringing in new models since OpenDocRouter laun
Hey everyone! 👋 Feeding multi-gigabyte files (SQL dumps, videos, audio, PDFs, and spreadsheets) directly into LLMs often causes context window overflow, GPU VRAM bottlenecks, massive token bills, and
DigUp is a free Mac app that runs Google DeepMind’s new EmbeddingGemma 2 locally over your own files. The model puts text, images, audio and video in one space, so you describe what you remember and l
Running HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM.
Strata vs Infernix on the same model, RTX 5090 — A/B at 262k and 512k context Hi folks, another inference engine post for your feed. I was blown away by Strata, but saw someone else post about Inferni
I've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and cod
Financial Times: Sources: Nvidia is in talks to acquire or invest more into US open-weights AI startup Reflection AI; the deal may be an acquihire to avoid antitrust scrutiny — Donald Trump's administ
Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted
The ultimate LLM benchmark: Chilean Spanish. 80% of AI models fail to recognize Chilean Spanish and insist on speaking like an Argentine 80% de modelos de IA no logra reconocer el español chileno e in
We @Sequoia are thrilled to back @TypesafeAI @CompleteSkeptic. ChatGPT put AI front and center in the consumer mind. Claude Code transformed the day-to-day work of the developer/knowledge worker. Jev
arXiv:2603.01376v2 Announce Type: replace Abstract: Sparse plus Low-Rank (mathbf{S} + mathbf{LR}) decomposition of Large Language Models (LLMs) has emerged as a promising direction in model compressio
arXiv:2610.11716v1 Announce Type: cross Abstract: We describe a 4-tensor attention model that predicts the next semantic state of a scene, for video generation and robot planning. A window of states h
arXiv:2610.11696v1 Announce Type: new Abstract: Puzzles are widely used to evaluate the reasoning capabilities of artificial intelligence (AI) systems for sequential decision making, yet approaches or
arXiv:2610.10607v1 Announce Type: new Abstract: Immersive VR180 video is increasingly produced with professional stereo fisheye cameras, yet public VR180 research resources are mostly collected from o
arXiv:2610.12183v1 Announce Type: cross Abstract: Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited. Recent large la
arXiv:2607.17575v3 Announce Type: replace Abstract: We propose ARBITER, a novel LLM guardrail framework that introduces two key ideas: (i) dual-hypothesis reasoning, a reasoning method for LLM guardra
arXiv:2610.10862v1 Announce Type: cross Abstract: Modeling heterogeneous materials remains a challenge for physics-informed neural networks such as the deep energy method (DEM). The DEM and its varian
I shipped a new feature for my blog today: the Newsletters page, which offers an index of all of the newsletters I've sent out, both my free weekly Substack and my monthly sponsors-only updates. I bui
arXiv:2610.10580v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly being integrated into hardware verification to automate specification interpretation, verification-artif
arXiv:2610.11149v1 Announce Type: new Abstract: Video anomaly detection (VAD) is a fundamental and safety-critical task in computer vision. Recent generative approaches detect anomalies from a distrib
A week of free access to Qwen3.8-Max, Qwen3.8-Flash, and Wan3.0 on @gmi_cloudis live. Explore what you can build with us now. 🙌 We're extending FREE Qwen 3.8 Max + Flash for 7 more days! And we've inc
arXiv:2403.18791v4 Announce Type: replace Abstract: Estimating the pose of objects from images is a crucial task of 3D scene understanding, and recent approaches have shown promising results on very l
arXiv:2602.15397v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models leveraging the native autoregressive paradigm of Vision-Language Models (VLMs) have demonstrated superior
arXiv:2610.12240v1 Announce Type: cross Abstract: Time-series foundation models (TSFMs) have achieved strong forecasting performance across domains. However, most adaptation methods remain static. Exi
arXiv:2610.11585v1 Announce Type: cross Abstract: Recent advances in large language model (LLM) pretraining highlight the role of high-quality training data in improving performance. While model-based
arXiv:2610.12265v1 Announce Type: new Abstract: Accurate workload forecasting is critical for elastic resource provisioning in web-scale cloud services, where distribution shifts driven by viral conte
arXiv:2610.10628v1 Announce Type: cross Abstract: Existing LLM-based approaches for software Requirements Engineering (RE) typically rely on basic prompting strategies or rudimentary agent collaborati
arXiv:2610.11050v1 Announce Type: new Abstract: Computer-use agents are capable of completing complex tasks, increasing the use of automatic judges to determine success, either for training or for eva
arXiv:2610.12002v1 Announce Type: cross Abstract: Test-time training (TTT) adapts an LLM's parameters using signals derived from test inputs, and can make striking improvements in pre-specified settin
arXiv:2610.09944v2 Announce Type: replace Abstract: An essential control of AI agents is their ability to manage runtime. This ability requires a sense of time-awareness, to predict and estimate wall-
arXiv:2610.10848v1 Announce Type: new Abstract: Accurate evaluation is central to selecting which LLM to deploy, yet testing a candidate on live traffic exposes real users to an unvetted model. Teams
arXiv:2610.10541v1 Announce Type: new Abstract: Knowledge Graph (KG) quality depends not only on downstream graph validation, but also on the quality of tabular metadata used before integration. In me