NVIDIA Co-Design Enhances AI Factory Memory Efficiency and Throughput

NVIDIA has unveiled a co-design strategy aimed at maximizing the efficiency of high-bandwidth memory (HBM) in AI factories. By optimizing how memory is utilized, the company has achieved significant throughput gains, with the NVIDIA Vera Rubin NVL72 delivering up to 30x the throughput per megawatt compared to the GB300 NVL72, despite both GPUs maintaining the same 288 GB HBM capacity. This improvement highlights the importance of maximizing the useful work each byte of memory performs, rather than merely increasing capacity. One of the key innovations driving these gains is NVFP4, a 4-bit precision format that combines block-level and tensor-level scaling to store more intelligence in fewer bytes.

BeCrypto News shows a short excerpt and links to the publisher. The full report belongs to Blockchain.News.