NVIDIA Introduces NVHBM Memory Technology, Increasing Bandwidth by 30% and Reducing Power Consumption by 15%

BlockBeats News, August 27th, NVIDIA released NVHBM technology for the next-generation AI infrastructure. NVHBM is a custom HBM base die jointly designed and validated with a major memory chip supplier, which will be provided to hyperscalers and AI-native enterprises developing custom AI accelerators (XPU) through NVLink Fusion.

Compared to standard HBM4e, NVHBM can provide up to 30% additional memory bandwidth per stack, reduce up to 15% of HBM power consumption, and free up to 25% of die area for XPU. NVIDIA stated that these improvements can help custom accelerators support larger AI models, quicker read KV cache, and enhance training and large-scale inference efficiency.

By redesigning the physical memory interface, NVHBM moves the memory controller into the 3D HBM stack and integrates a custom PHY, reducing the PHY and associated area by up to 67% and simplifying intermediate layer routing. The freed-up die area can be used to increase matrix engines, caches, networks, and other workload-specific features.

At the rack level, NVLink Fusion chips can connect the custom XPU to the NVLink network and link to the CPU through NVLink-C2C. NVIDIA stated that after combining NVHBM with NVLink Fusion, the end-to-end performance of each XPU can increase by about 30%, allowing custom XPU to form a heterogeneous computing system with NVIDIA GPUs. The official commercial release date has not been disclosed in the article.