Search: “applications”

I have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor

Hey guys, Looking for an alternative for Claude Cowork: Project Work / Documents Integrations like Notion, Gmail, etc. Tools like Websearch, PDF creation, etc. Came over Eigent (https://github.com/eig

Running things locally already means you care more about control and reproducibility than most. Worth extending that to your prompts too. There's a live workshop on Oct 3 covering exactly that. Serj S

I was reading a paper that surveyed the field of neural architecture search, where it said within 5 years, around 3000+ new models were proposed. The amount of compute and resources spent on this is a

I know Qwen is a key player in the local LLM space and has consistently introduced truly impactful technologies—like n-gram in Qwen-Next and the recent Qwen 3.8 27B, which is an amazing local model. H

Hi! I'm the author of MoEspresso, which is my way of putting my own ideas about inference engines to the test. A lot of the fun has been trying different design choices, measuring what happens, and fi

Distributed Training and Inference both involves having a fundamental understanding of how distributed systems work in general Distributed Parallelism Tensor Parallelism Pipeline Parallelism Model Par

Hey guys, I came across the Forrester function recently while studying mathematics, and I started wondering if there’s more to it than just being a mathematical function. I have a feeling I’ve seen it

This post is written by a human and I'd appreciate it if you treated it as such. Thanks. So, I've been noticing a pretty clear interest in developing as good a coding and agentic tool-calling model as

Someone recommended that I try the ByteShape Qwen 3.8 27B IQ3-XXS GGUF after seeing my previous testing of the GSQ quant. So I did. And the result was… surprisingly bad. For context, I'm running: RTX

I’m building a personal local coding-agent project called Cody, using local 9B/14B models through Ollama. This is still a work in progress. It has not passed its full graduation process yet, and the l

A few days ago, I came across a post/comment saying that you can get better data privacy by using local Ollama models in a RAG pipeline instead of relying on cloud-hosted models/APIs. The explanation

OpenAI’s decision to retire custom GPTs appears to rest on the assumption that Plugins, Skills, Projects, Apps, and Workspace Agents collectively replace their functionality. For many Small and Medium

I'm still using LLM's the 2022/23 way of doing everything manually from the chat box. I copy and paste everything manually into the box and ask the AI to explain or do something, such as pasting code

Hi everyone, The NeurIPS results aren’t out yet, but I’m expecting my paper to be rejected, so I’m considering submitting it to a journal instead. My scores were 2/3/3 (3/4/4). I feel like I would pro

MiniMax has released the source code for MiniMax Code’s terminal agent: https://github.com/MiniMax-AI/minimax-code https://preview.redd.it/st9izmm8zaqh1.png?width=3016&format=png&auto=webp&s=5853bc316

I'm starting a PhD where I will be doing Graph ML (somewhere along the lines of graph signal processing/ graph deep learning.) My eventual goal is research scientist at big tech, or whichever company

Pushed out this new update, hopefully decreases the instances of crashes. I torture tested this one for ~12 hours after my fixes and found no instability. Q4 K XL needs more fine tuning, which I will

Hello LocalLLaMA Fam! I built this project because I, like many of us here, enjoy local AI and have a nice setup, but still find myself reaching for Cloud-based AI due to their convenience. My models

I’ve been thinking about an alternative way of doing low-level AI perception, and I’m curious whether anyone here is already working on something similar. The basic idea is to use waves and physical s

Hi, everyone, I’ve just released VoxGen, a lightweight native inference engine for VoxCPM2, written in Rust and using Vulkan compute instead of Python/PyTorch/CUDA. Why VoxGen? The main reason I start

Happy Friday! Ever had issues getting the anatomy right on your t2v generation? Just add a slider with your H3 Prompt! What have you all been building on your local AI studios? 384x448, int8, 20 steps

In the past, I have use llama.cpp, but I read that the exl3 quantization format should give better precision, so I have tried exllamav3/tabbyAPI. It was able to write the shown simple HTML game withou

I’ve been experimenting with a custom inference/runtime research project called CRANE V2, mostly because I wanted to answer a stupid question: How far can you push absurdly large MoE models on an ordi

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detai

Standard constrained RL assumes consequences are immediate and attributable to the current action. This breaks down whenever violations are delayed and stochastic, which is most real-world settings yo

I haven't had a lot of luck in the reliability of the output of any models so far for local coding use, and find myself relying on Claude's limited free sessions to assist with coding. My coding appli

Most YouTube benchmarks only show trivial tasks like generating landing pages or simple Three.js games. Is a local model like Qwen 3.8 27B actually capable of real-world systems programming—such as bu

Further to my last post, https://www.reddit.com/r/LocalLLaMA/comments/1vldngi/tested_in_coding_bf16_muse_glimmer_vs_bf16_qwen36/, I bring this requested comparison of Qwen3.8 27B. Context: Both models

Component Validated configuration Motherboard ASRock Rack SPC621D8U-2T/OVH CPU Xeon Gold 6330 (Get gold/platinum if interested in Optane Pmem gimmicks) GPU fabric Two Broadcom/PLX PEX88096 islands, ei

None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research. Two key highlights: - They use WSM to replace LR decay with

Following up on my previous post about my budget server setup (Intel N100 + RTX 5060 Ti 16GB), a few of you asked for a deeper dive into my actual inference config and real-world agentic performance.

North Micro Vision Instruct is a 2.4B-parameter open-weight vision-language model with native-resolution image support, released under the Apache 2.0 license. It is designed as a compact foundation fo

LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment. It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon

GitHub: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader Release v1.0.39: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader/releases/tag/v1.0.39 Also available through ComfyUI Manag

Hello all, I am wondering if a transition from a Ph.D. in electrical engineering (Quantum optics/photonics) to a job in ML is a reasonable aspiration. Personally, I have extensive software development

I'm guessing that many people have been waiting for this comparison. For clarity, both models are running at full FP16 KV-cache. Due to VRAM limitations, Muse Glimmer is running full 262,144 context,

Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardwa

VLX-Seek-1.5-10B VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical setting

NVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blu

I don't own a mac Right now, i want to buy one but its main job will be to host a Ollama or similar software to give me usable AI models in my network, since I don't want to pay for Cloude, GitHub cop

Questions & Responses(in BOLD) below. Favorite question(s) moved to end of the thread with combined responses(removed duplicates). Be optimistic folks. I'm sure we're getting other models too apart fr

TLDR below 👇🏼 I’ve seen a lot of hype around Qwen 3.6 35B and 3.5 120B lately, especially regarding coding and tool-use capabilities. On this subreddit it is the defacto recommended model for everyone

audio.cpp 0.5 is out :) The most fun new model in 0.5 is DramaBox. It is closer to prompt-directed voice acting. DramaBox is built on the LTX-2.3 audio architecture, and prompts can control emotion, d

Just saw this posted by Fal.ai and then by Hailuo themselves, the next video model will be open weight released! Here's the blurb and link to to the feature post: Today, we're launching MiniMax H3, a