**NVIDIA H100 Tensor Core GPU Dominates MLPerf Training v3.0 Benchmarks**
**Introduction**
The performance of a system across various AI workloads can be measured through its MLPerf benchmark numbers. As the field of AI evolves, MLPerf also evolves to reflect industry changes. The latest MLPerf Training v3.0 benchmark suite introduces new tests for recommendation engines and large language model (LLM) training.
**NVIDIA H100 Dominates Every Benchmark**
The MLPerf LLM benchmark, based on OpenAI’s GPT-3 LLM trained with 175 billion parameters, is a computationally expensive task. NVIDIA designed the H100 Tensor Core GPU specifically for these workloads, making it a popular accelerator for training large language models. The H100 features a new transformer engine that accelerates transformer model training and inference, resulting in faster training times.
In the latest MLPerf benchmark results, the NVIDIA H100 dominated almost every category and was the only GPU used in the new LLM benchmarks. Out of the 90 systems tested, 82 used NVIDIA accelerators, with nearly half of all results based on the H100 Tensor Core GPU. The H100 set records on every workload in the MLPerf training and inference benchmarks, while NVIDIA’s A100 and L4 GPUs also delivered impressive results.
**LLM at Scale: NVIDIA + Inflection AI + CoreWeave**
While per-accelerator results provide insights, real-world production workloads are often built using multiple accelerators in a clustered system. NVIDIA, in collaboration with Inflection AI, developed a large-scale GPU cluster based on the H100 Tensor Core GPU, hosted and tested by CoreWeave. This cluster combines 3,584 H100 accelerators with 896 4th generation Intel Xeon Platinum 8462Y+ processors. The results achieved by this system set new records on every workload tested, showcasing the non-linear scalability potential of the NVIDIA H100 GPU.
Comparatively, Intel also reported LLM benchmark results, combining Intel Xeon Platinum 8380 processors with Intel Habana Gaudi2 accelerators. However, the training times for Intel’s systems were significantly higher than those achieved with the NVIDIA H100 GPU.
**Analyst’s Take**
The dominance of NVIDIA within the AI ecosystem is evident from the MLPerf results, where almost every submitted result was based on an NVIDIA accelerator. NVIDIA’s strong presence in the industry is not only due to its accelerator technology but also due to the reliance of the AI community on its software. NVIDIA provides comprehensive AI tools and solutions, from low-level libraries to full-stack solutions, and continues to invest in enterprise-level tools for workload and model management. This software investment sets NVIDIA apart and ensures its continued leadership in the AI industry.
The MLPerf results also highlight the power and efficiency of running AI training workloads in the cloud. Building a training cluster with multiple accelerators can be complex and expensive. The estimated cost of an H100 accelerator is between $30,000-$40,000, but CoreWeave offers rental services at $2.23/hour. This pricing model makes AI training accessible and cost-effective, especially considering the limited availability of H100-based instances from public cloud providers.
In conclusion, NVIDIA’s dominance in the AI ecosystem, coupled with the impressive performance of the H100 Tensor Core GPU, is revolutionizing the way we approach technology and data. NVIDIA’s presence in the data center continues to expand, and the company is positioned as a key enabler for the future of AI.
GIPHY App Key not set. Please check settings