Intel unveils enterprise-focused 'Gaudi' AI processor

Intel introduced its latest artificial intelligence processing chip, "Gaudi 3," on Tuesday, closely following Nvidia's unveiling of its Blackwell GPU just two weeks prior.

Gaudi 3 marks the third iteration of Intel's specialized chip designed for both artificial intelligence training and inference tasks. Intel integrated this chip family into its portfolio through the acquisition of Habana Labs, a Tel Aviv-based startup, for $2 billion in 2019.

When it comes to inference tasks, where a trained neural network provides responses to real-world queries, Gaudi 3 exhibits a 50% performance improvement over its predecessor, the H100.

In recent benchmark assessments, the Gaudi chip series has demonstrated notable proficiency when pitted against Nvidia's offerings. Notably, in the latest MLPerf competition hosted by MLCommons, an industry alliance, the previous generation Gaudi 2 chip stood as the sole data center chip competing with Nvidia's H100 in utilizing Meta's Llama 2—a substantial language model comprising 70 billion parameters.

Gaudi 3 is comprised of 64 individual tensor cores on its die, which enhance matrix multiplications crucial for AI processing, supported by eight discrete "matrix math engines." With 96 megabytes of fast on-chip SRAM cache memory and an additional 128 gigabytes of external "HBM3e" memory—regarded as the fastest DRAM in the industry—composed of multiple memory-chip die stacked adjacent to the processor.

In terms of performance metrics, Gaudi 3 can achieve 1.84 teraFLOPs (trillion floating-point operations per second) when handling 8-bit floating-point mathematical operations, a widely recognized measure of chip performance.