Machine learning is a subset of artificial intelligence function that provides the system with the ability to learn from data without being programmed explicitly. Today AI requires a lot of resources to train and produce accurate results. However, there will be more human friendly AI systems in the near future.

Machine learning is basically a mathematical and probabilistic model which requires tons of computations. It is very trivial for humans to do those tasks, but computational machines can perform similar tasks very easily.

Consumer hardware may not be able to do extensive computations very quickly as a model may require to calculate and update millions of parameters in run-time for a single iterative model like deep neural networks. Thus, there is a scope for the hardware which works well with extensive calculation. The machine learning flow includes the four steps for preparing a machine learning model

There are four steps for preparing a machine learning model, pre-processing input data, Training the deep learning model, Storing the trained deep learning model and Deployment of the model. Among all these, training the machine learning model is the most computationally intensive task.

We need to make the training model faster, this can be accomplished simply by performing all the operations at the same time, instead of taking them one after the other. This is where the GPU comes into the picture, with several thousand cores designed to compute with almost 100% efficiency. Turns out these processors are suited to perform the computation of neural networks as well.

The fight between CPUs and GPUs favors the latter because of the large amount of cores of GPUs offsetting the 2–3x faster speed of CPU clocks – ~3500 (GPU) vs ~16 (CPU). The GPU cores are a streamlined version of the more complex CPU cores, but having so many of them enables GPUs to have a higher level of parallelism and thus better performance.

CPUs are designed to run almost any calculation, that is why they are called general-purpose computers. In order to achieve this generality, CPUs store values in registers, while a program tells the Arithmetic Logic Units (ALUs) which registers to read, perform an operation (such as an addition, multiplication or logical AND) and which register to use for output storage, which in turn contains lots of sequencing of these read/operate/write operations. Due to this support for generality (registers, ALUs and programmed control), CPUs cost more in terms of power and chip area.

There are alternatives to the GPUs such as FPGAs and ASIC, as all devices do not contain the amount of power required to run a GPU (~450W, including CPU and motherboard). TPU (Tensor Processing unit) is another example of machine learning specific ASIC, which is designed to accelerate computation of linear algebra and specializes in performing fast and bulky matrix multiplications.