Utilize a nvidia gpu and amd gpu together for 2 different ai models?

We run a local model instance in our company that the dev we hired built for us. We're a trade business and we want to further use our on hand hardware for it. The specs given we have is a 5090 gpu with 64gb of ram and a ryzen 9600 cpu, its am5 thats what i know? We have a older gen AMD gpu on hand, 12 gb of vram, that came with a msi prebuilt back in 2018 we used for our receptionist back then. Since we use llama.cpp, can we continue loading our custom tuned model on the 5090, and load up a seperate weaker gemma model or something else, approx. 4B model, on the AMD gpu? Our setup would be this: 1 PC/Server, and it would contain both GPUs on 1 motherboard, 5090 serving our main tuned Qwen 27B model, and the weaker AMD gpu serving a weaker 4B model. the 4b model's purpose would be for completely simple automations that run 1 to 5 times a day where it summarizes a paragraph or two into layman terms, and the tooling our dev built handles the rest. Currently the 5090 is able to handle this easily and more, but for this specific task, we want to be able to offloaded to the weaker models. As currently when our tuned Qwen instance runs, the simple automation needs to wait for the bigger task to finish, which can take some time. So to avoid that, we want to offload the simple task to the AMD gpu. Would this be doable? submitted by /u/Curious-Pen5547 [link] [comments]

[[audiocpp-release-05-dramabox-expressive-tts-confucius4-cross|[audio.cpp] Release 0.5: DramaBox expressive TTS, Confucius4 cross-lingual voice transfer, plus 7 more models and ROCm/HIP]]
5700, 48GB RAM and a 3090 24Gb. Best OS and framework/model?
How well do multiple GPUs scale for LLM inference? (Trying to understand the basics)

Source: r/LocalLLaMA | 2026-08-05

Related