Linux Patches Introduce "KNOD" For In-Kernel Network Offloading Directly To AMD GPUs

Summary

Some extremely cool patches were posted to the Linux kernel mailing list on Sunday. The patches for "KNOD" allow for in-kernel network offloading to GPUs with an initial focus on AMD GPU support. What makes this all the more nifty is that it doesn't depend upon any user-space libraries like AMD ROCm but is all handled in-kernel with driving the GPU directly. After initially presenting his concept last year at the Linux Plumbers Conference in Japan, Taehee Yoo yesterday posted the KNOD patches to the Linux kernel mailing list under a Request For (RFC) banner. KNOD is for in-kernel network offload device that is entirely kernel driven for accelerated packet processing without the need for ROCm or any other user-space components. The kernel code itself is managing the GPU queues, JIT compiling the per-packet program to GPU machine code, and dispatching the work.The network adapter DMAs the received packets straight into the GPU memory. The concept also allows for offloading existing XDP programs and IPsec SAs directly to the GPU in a transparent manner. "Line-rate packet processing that does non-trivial per-packet work - an XDP program doing L4 load balancing, or IPsec crypto - is bound by the host CPU: each core handles one packet at a time, so scaling means spending more cores. A GPU is the opposite shape - thousands of lanes running the same small program over many packets at once (SIMT) - which happens to match the per-packet-program model of XDP. knod moves that per-packet compute off the host CPU and onto a GPU. The NIC DMAs received packets directly into GPU memory, the GPU runs the program across a batch of packets in parallel, and only the result comes back: a verdict for every packet, plus the packet itself for the ones destined to the host. The CPU no longer pays the per-packet program cost, and throughput scales with GPU occupancy rather than core count. Crucially this happens entirely inside the kernel. GPU packet processing today generally launches work ...