Why is GPU getting important in AI era?

The stock price of nVIDIA has skyrocketed in AI era. Although IT related stocks have experienced the same phenomenon under COVID-19, the stock price rally of nVIDIA is dramatic. Why did stock price of nVIDIA increase? I will find the answer with GPU, the core product of nVIDIA, and the emergence of AI era.

What is GPU contributing to the most portion of nVIDIA profit? GPU (graphic processing unit) is literally the processor especially dedicated for graphic applications like game and movie edit. GPU complements the limits of CPU,  general-purpose processor. Graphic processing, generally, calculates color (texture) of numerous pixels constituting of a frame. For example, 4K resolution constitutes of about 8 millions of pixels. To play high-resolution games or movies, computing devices are required to calculate enormous pixels in simultaneously. However, CPU includes at most about 20 cores and each core can operate tens of instructions concurrently at ideal situation. Since CPU is aimed to operate diverse instructions with flexibility, it has limitations on graphic processing. To complement CPU, GPU contains lots of simple processing units. Since most operations of graphic application are multiplications and additions, GPU with lots of simple processing units capable of calculating them generates more frames or pixels than CPU within a given time. The below figure represents the differences between CPU and GPU.

Comparison between CPU and GPU (source: blogs.nvidia.com)

Why does the demand of GPU increase in AI era? AI operation requires the parallel computing constituting of simple additions and multiplications, which is similar to graphic applications. If we take a look at equations in AI models, most calculations are additions and multiplications of matrices (vectors). Aforementioned AI models are vision recognition ones like CNN and speech recognition models like RNN. Multiplication of matrices are done by multiplying each elements of different matrices and adding the results of multiplications. Therefore, the AI-preferred hardware which can execute lots of multiplications and additions at once is not CPU but GPU. Since it takes years to develop and manufacture new computing processors with expertise and costs, most companies have used GPU for AI application except Google who developed TPU. In this reason, the demand of GPU has increased.

Will nVIDIA keep her the dominant position? The answer depends on the AI ASIC companies. AI ASIC (application-specific integrated circuit) is dedicated for AI operations like TPU by Google. AI ASIC companies like Habana acquired by Intel and Graphcore have argued that they can enhance the efficiency of AI operation with their ASICs. The criteria of efficiency is the amount of operations per Watt. Since the original purpose of GPU is graphic operations, it has intrinsic inefficiency when it is applied on AI applications. To overcome the inefficiency, their new solutions, AI ASIC, are necessary. It will be interesting to see the competition between GPU and AI ASIC in AI hardware industry for next couple of years.