From the perspective of chip composition, AI servers are mainly CPU+acceleration chips, usually equipped with GPU, FPGA, ASIC and other acceleration chips. The combination of CPU and acceleration chips can meet the needs of high-throughput interconnection.
1.CPU
As the computing and control core of the computer system, it is the final execution unit for information processing and program execution. Its advantage is that it has a large amount of cache and complex logic control units, and it is good at logic control and serial operations; its disadvantage is that it has a small amount of calculation and is not good at complex algorithm operations and processing parallel repeated operations. Therefore, CPU can be used for inference/prediction in deep learning.
Currently, server CPUs are developing towards multi-core to meet the needs of increasing processing power and speed. For example, AMD EPYC 9004 has up to 96 cores. However, system performance should not only consider the number of CPU cores, but also the operating system, scheduling algorithm, applications and drivers.
2.GPU
GPU is highly suitable for AI model construction. Due to its parallel computing capabilities and compatibility with training and inference, GPU is currently widely used in acceleration chips. Taking NVIDIA A100 as an example, during training
In the process, GPUs help solve problems at high speed: 2048 A100 GPUs can handle training workloads such as BERT at scale in one minute. During inference, Multi-Instance GPU (MIG) technology allows multiple networks to run simultaneously on a single A100, optimizing utilization of computing resources. Building on the other inference performance gains of the A100, structural sparsity support alone can bring up to a twofold performance improvement. On advanced conversational AI models such as BERT, the A100 can increase inference throughput up to 249 times that of the CPU.
Currently, ChatGPT has triggered a wave of GPU applications. Among them, Baidu is about to launch ERNIE Bot. Apple has introduced the M2 series chips (M2 pro and M2 max) designed with AI accelerators and will be installed on new computers. As ChatGPT usage surges, OpenAI needs stronger computing power to respond to millions of users, thus increasing demand for Nvidia GPUs.
AMD plans to launch TSMC's 4nm process "Phoenix" series chips that compete with Apple's M2 series chips, and the "Alveo V70" AI chip designed using the Chiplet process. Both chips are planned to be launched on the market this year, targeting the consumer electronics market and AI reasoning respectively.
3.FPGA
FPGA has the characteristics of high programmability flexibility, short development cycle, on-site reprogrammability, low latency, and convenient parallel computing. It can empower large models through deep learning + distributed cluster data transmission.
4.ASIC
Compared with general-purpose integrated circuits, ASIC has the advantages of smaller size, lower power consumption, improved reliability, improved performance, enhanced confidentiality, and reduced cost during mass production, and can further optimize performance and power consumption. With the development of machine learning, edge computing, and autonomous driving, a large number of data processing tasks are generated, and the requirements for chip computing efficiency, computing power, and energy consumption ratio are getting higher and higher. ASIC has attracted widespread attention through its combination with CPU, and leading domestic and foreign manufacturers have made arrangements to prepare for the arrival of the AI era.
Among them, Google's latest TPU v4 cluster is called Pod, which contains 4096 v4 chips and can provide more than 1 exaflops of floating point performance. NVIDIA GPU+CUDA is mainly aimed at large-scale data-intensive HPC and AI applications; the Grace-based system is closely integrated with NVIDIA GPU, and the performance is 10 times higher than the NVIDIA DGX system. Baidu's Kunlun 2nd generation AI chip uses the world's leading 7nm process and is equipped with the self-developed second-generation XPU architecture, which improves performance by 2-3 times compared to the first generation; Kunlun 3rd generation will be mass-produced in early 2024.
5. Optical module
Currently, the demand for model computing power in the AI era has far exceeded the growth rate of Moore's Law. Especially after the era of deep learning and large models, it is expected to double in 5-6 months. However, data transmission rate has become a computational bottleneck that is easily overlooked. With the growth of data transmission volume, the demand for optical modules, as the carrier of equipment interconnection in the data center, has also increased.
