When Is NVLink Worth It?

Summary

Nvidia doesn’t support NVLink on consumer GPUs anymore. The last one that had it available was the RTX 3090. Right now in April 2026, NVLink bridges are pretty expensive, $200-$400 depending on the exact size and whether you’re able to snipe one at retail. Since I have two 3090s, I wanted to test how much benefit NVLink actually provides for AI workloads. The Tests I ran a few tests to measure the effect that adding NVLink would have on AI-related workloads: Model Inference with layer split and tensor parallelism. I used llama-bench with Llama 3.3 70B and Gemma 4 31B. Layer split does very little inter-GPU communication, while tensor parallel inference must sync at every layer. Model Training using DDP for TinyLlama 1.1B and FSDP for Qwen2.5 3B. DDP only syncs once to compute gradients at each step, while FSDP requires constant data transfers. With these, I felt I had a good mix of tests that would be affected to different degrees by inter-GPU bandwidth. Hardware CPU AMD Epyc 7532 (32 Cores, 128 PCIe lanes) Motherboard Supermicro H12SSL-i, 5 PCIe 4.0 x16 Slots RAM 8 Channels, 32GB DDR4-3200, 256GB total GPU 2x EVGA RTX 3090, installed directly in slots 3 & 6, both running at full x16 bandwidth. OS Ubuntu 24.04, Nvidia driver 595 Initial Results The performance on all layer split tests was unaffected. For model inference with tensor parallelism, NVLink boosted prompt processing by about 30%, while token generation speed was exactly the same. Token generation speed being the same on tensor parallel runs was unexpected. I thought there would be a large difference since it needs to sync activations frequently. Although token generation speed is usually the headline number, I still care a lot about prompt processing speed since most of my usage patterns, such as coding agents, use very long context windows. The uplift for FSDP training is huge, a nearly 3x improvement. DDP training was not affected at all. However, I consider this data to be somewhat misleading, as the n...