NVIDIA Introduces Reshard Collective to Optimize RL Weight Refit

NVIDIA has unveiled the Reshard collective, a novel communication method designed to optimize weight refit in reinforcement learning (RL) workflows. Weight refit involves transferring updated policy model parameters from training GPUs to inference GPUs, a process critical for generating fresh training trajectories. The Reshard collective aims to address inefficiencies in current methods, particularly as model sizes continue to scale. Traditional weight refit methods rely on a gather-and-broadcast approach, where training GPUs reconstruct complete model parameters before broadcasting them to inference GPUs. While straightforward, this process results in redundant communication, as inference GPUs receive entire tensors even when only specific shards are required. For large models such as Kimi K3 (2.8 trillion parameters) and Qwen 3.8 2.4T A95B (2.4 trillion parameters), this inefficiency is magnified.

BeCrypto News shows a short excerpt and links to the publisher. The full report belongs to Blockchain.News.