Several Types of Artificial Neural Networks Architecture that You Should Know
Definition of neural networks, several types of neural networks architecture, and their usage
Before I go into detail about several types of neural network architecture, It would be helpful if we first understood deep learning.
In general, deep learning is a subfield of machine learning (ML) in artificial intelligence(AI that deals with algorithms inspired by the biological structure and functioning of a brain to aid machines with intelligence) (Moolayil, 2019).
When should we use deep learning? Deep learning is used to solve complex problems that cannot be solved by machine learning approaches such as classifying audio, video, images, etc. Deep learning is designed using a neural network architecture that is trained using a backpropagation algorithm. Until now, there are many applications of deep learning that are a part of our daily lives, such as self-driving cars, machine translation, speech recognition, face detection, etc.
Keep in mind here that I will explain as simply as possible and will not include the mathematical calculations behind the neural network architecture.
1. Multilayer Perceptron (MLP)
The first neural network architecture that you should know is the Multilayer Perceptron (MLP). Multilayer perceptron aka classical neural networks is a neural network architecture designed to estimate any continuous functions and can solve problems that are non-linear separable. Multilayer perceptron consists of 3 main layers, namely the input layer, hidden layer, and output layer.
The input layer will receive the input signal for processing and forward it to the hidden layer. The hidden layer is the main computing engine of the MLP that receives input from the input layer and forwards it to the output layer or vice versa during backpropagation. While the output layer performs predictions or classifications based on the input data that is passed from the hidden layers.
The multilayer perceptron is widely used in several cases, such as pattern classification, prediction, recognition, and approximation.
2. Convolutional Neural Networks (CNN)
The second neural network architecture you should know is Convolutional Neural Networks (CNN). Convolutional neural networks are neural network architectures designed to solve computer vision problems such as image classification or image detection. This neural network architecture consists of 3 main layers, namely the convolutional layer, pooling layer, and fully connected layer.
The convolutional layer is the most important building block of CNN architecture, which consists of several layers depending on the complexity of the problem to be solved. The first layer of the convolution layer is not connected directly to every pixel in the input image, but is only connected to the receptive features of the input image. The first layer of the convolutional layer is devoted to capturing basic (low-level features) such as colors, lines, edges, etc. Furthermore, there is a pooling layer that works by reducing the size of the input image from the convolutional layer to make computations more efficient. Finally, there is a Fully connected layer, which is used to connect every neuron in one layer to all neurons in another layer.
There are many applications of convolutional neural networks that are a part of our daily lives, such as object detection in a self-driving car, social media face recognition, image analysis in the medical field, etc.
3. Recurrent Neural Networks (RNN)
The third neural network architecture you should know is Recurrent Neural Networks (RNN). Recurrent neural network is a neural network architecture that works very well in handling sequential data such as natural language processing and time series data. In general, the structure of recurrent neural network is almost the same as multilayer perceptron, the difference is that there are recurrent layers.
The recurrent layer accepts input, produces output, and sends the output back to itself. However, in processing long sequences, the recurrent neural network may encounter unstable gradient problems. To solve this problem, we have to replace the neuron cells in the recurrent neural networks with Long Short-Term Memory(LSTM) cells and Gated Recurrent Unit(GRU) cells that can handle this problem.
There are many applications of recurrent neural networks that are a part of our daily lives, such as machine translation, sentiment analysis, text summarization, document generation, chatbots, etc.
4. Generative Adversarial Networks (GAN)
Lastly, the neural network architecture you should know is Generative Adversarial Networks (GAN). Generative adversarial networks are neural network architectures that can generate new data with the same characteristics as the data provided during training. This neural network architecture has two networks placed opposite each other, namely generator and discriminator. Each network has a different role.
The generator will learn and generate a realistic fake image from a random distribution as input (typically Gaussian), while the discriminator will take a fake image from the generator or a real image from the training set as input, and guess whether the input image is real or fake. As the model training iteration goes on, the generator will generate a more realistic image that can fool the discriminator. You can visit this website to see the images generated by the model generative adversarial networks.
There are many applications of generative adversarial networks that are a part of our daily lives, such as generating realistic photographs, generating new human poses, generating images based on sentence illustration, etc.
Conclusion
Each architecture of the neural network has different uses. MLP is used to solve classic problems that cannot be solved by machine learning, CNN is used to solve computer vision problems, RNN is used to solve sequential data problems, and GAN is used to produce realistic and high-resolution images.
Reference:
[1] Abirami, S., & Chitra, P. (2020). Energy-efficient edge based real-time healthcare support system. The Digital Twin Paradigm for Smarter Systems and Environments: The Industry Use Cases, 339–368. doi:10.1016/bs.adcom.2019.09.007.
[2] Géron, A., (2019). Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems. O’Reilly Media.
[3] Menzies, T., Kocagüneli, E., Minku, L., Peters, F., & Turhan, B. (2015). Using Goals in Model-Based Reasoning. Sharing Data and Models in Software Engineering, 321–353. doi:10.1016/b978–0–12–417295–1.00024–2.
[4] Moolayil, J. (2019). Learn Keras for Deep Neural Networks: A Fast-Track Approach to Modern Deep Learning with Python. Apress.
