Has anyone been working on a solid setup for DSV4F on x2+ R9700s?

I'm hoping that one of you guys has been working on an inference engine or has somehow found improvements to running DSV4F on RDNA4 multi-GPU setups. I am currently building a custom inference engine in Rust using HIP but its still in the early stages. I'm using vulkanforge and antirez' work on ds4 as inspiration, and likely will be adopting a custom quant like what antirez did. The only issue with it is that it's entirely built out for my setup and has things placed on my system in certain areas to work fast. Currently I have 2x R9700s, Ryzen 5 9600x, 128GB DDR5. My second card is still on PCIE 4 x4 so its majorly bottlenecked. Planning to only put the hot experts on that card since bandwidth between would be minimal, then use the RAM for the cold/missed experts with a design to xfer cold experts to the GPU and swap out the least used ones after multiple misses during cooldown periods between processing. From testing my best case scenario is around 80 tok/s on tg with DFlash but my hope is at least 60 tg. submitted by /u/Public_Umpire_1099 [link] [comments]

Source: r/LocalLLaMA | 2026-08-04