**NVIDIA Unveils Upgraded GH200 Grace CPU and Hopper GPU**
NVIDIA CEO Jensen Huang made a significant announcement during his keynote address at the SIGGRAPH conference. The company introduced an enhanced GH200 (Grace CPU and Hopper GPU) and a dual GH200 system. These advancements aim to address the memory bandwidth and capacity limitations faced by large language models (LLMs) such as GPT3/4 and ChatGPT. By increasing memory capacity and bandwidth per GPU by 70% and 50% respectively, NVIDIA aims to improve the performance and reduce the costs associated with deploying these massive models.
**The Dual GH200 Board – A Boost for ChatGPT-like Model Inferencing**
One of the highlights of NVIDIA’s announcement is the introduction of the dual GH200 board, which combines two Grace Hopper superchips connected by NVLink on a single board. This configuration also includes fast LPDDR5 memory, which is more cost-effective and energy-efficient compared to traditional x86 servers. With the ability to scale up to 256 GPUs over NVLink, this platform significantly reduces the number of GPUs required for inference by approximately 60%.
The implications of this innovation extend beyond cost savings. By offering faster inference processing, reduced energy consumption, and improved performance, NVIDIA anticipates increased demand for its Arm-powered Grace CPU. Customers can now achieve comparable results with 10 GH200s instead of two dual x86 CPUs and 16 expensive GPUs. This shift not only benefits customers but also positions NVIDIA to replace high-end Intel Xeon CPUs with their Grace CPUs, which could have a substantial impact on the industry.
**Closing the Gap with AMD and Expanding Grace CPU Adoption**
NVIDIA’s latest advancements in memory capacity and bandwidth aim to solve a significant pain point in LLMs and enable the company to close the competitive gap with AMD. By allowing data center operators to deploy fewer GPUs while maintaining the same memory capacity, NVIDIA is addressing a critical industry need. This development is expected to drive increased adoption of the Grace Hopper superchip and the Arm-powered Grace CPU while posing a potential challenge for x86 vendors.
**Conclusion**
NVIDIA’s announcement of the upgraded GH200 Grace CPU and Hopper GPU, along with the introduction of the dual GH200 system, marks a significant milestone in the AI landscape. The increased memory capacity and bandwidth provided by these advancements offer a viable solution to the challenges faced by large language models. Not only does this innovation improve performance and reduce costs, but it also positions NVIDIA to compete more effectively with AMD while driving the adoption of the Grace CPU. With these developments, NVIDIA is poised to reshape the AI industry and deliver tangible benefits to data center operators and customers.
GIPHY App Key not set. Please check settings