Neural networks are powerful deep learning models capable of synthesizing vast amounts of data in seconds. There are many different types of neural networks that help us accomplish various everyday tasks, from recommending movies or music to assisting us in online shopping.
Similar to how airplanes were inspired by birds, neural networks (NNs) are also inspired by biological neural networks. While the principles are the same, the processes and structures can differ significantly. Below, I will introduce the 6 types of neural networks that every data scientist must understand.
6 Basic Types of Neural Networks
Now that we understand how neural networks work, let’s take a look at the various types and functions of neural networks used in deep learning:
-
Perceptron -
Multilayer Perceptron -
Convolutional Neural Network -
Recurrent Neural Network -
Long Short-Term Memory Network -
Generative Adversarial Network
1. Perceptron
The perceptron is the simplest neural network structure. This model is also known as a single-layer neural network and consists of only two layers:
-
Input Layer -
Output Layer
There are no hidden layers. The perceptron takes inputs and computes the weighted input for each input node. This weighted input generates an output through an activation function.
Due to its simple architecture, it cannot be used for complex tasks.
2. Multilayer Perceptron
The Multilayer Perceptron (MLP) typically represents a fully connected network. In other words, every neuron in one layer is connected to all neurons in the adjacent layer. Therefore, MLP has a higher processing capability than the perceptron. However, the “fully connected” nature of these networks makes them prone to overfitting the data. Typical methods to reduce overfitting include early stopping, adding dropout layers, and adding regularization terms.
3. Convolutional Neural Network
Humans use neurons in the eyes to recognize objects, which can detect edges, shapes, depth, and motion. Convolutional Neural Networks (CNNs) are one of the most important types of neural networks in computer vision, inspired by the visual cortex of the eyes, used for visual tasks such as object detection. The convolutional layer of a CNN distinguishes it from other neural networks. This layer performs a dot product, i.e., component-wise multiplication followed by addition.
In the initial stages of a CNN, the filters are random and do not provide any useful results. Using a loss function, the filters are adjusted through multiple iterations, allowing the network to improve in achieving its tasks (e.g., detecting object edges). Although CNNs typically require a large amount of training data, they are widely applicable to various image and even language tasks.
CNNs are inspired by the visual cortex, thus they are widely used in applications involving computer vision. These applications include facial recognition, face detection, object recognition, handwritten character recognition, and tumor detection in medical diagnostics.
4. Recurrent Neural Network
When we read a specific chapter, we do not try to understand it in isolation but relate it to previous chapters. Similarly, like natural neural networks, machine learning models need to leverage previously learned text to understand current text.
In traditional machine learning models, this is impossible because we cannot store the previous stages of the model. However, Recurrent Neural Networks (often referred to as RNNs) are a type of neural network that allows us to do this, making them very useful for applications that require the use of past data. Let’s take a closer look at RNNs below.
Recurrent Neural Networks are networks designed to interpret temporal or sequential information. RNNs use other data points in the sequence to make better predictions. They achieve this by influencing the output through the input and reusing activations from previous or subsequent nodes in the sequence.
RNNs are commonly used for sequential applications such as time series forecasting, signal processing, and handwritten character recognition. Additionally, RNNs are widely applied in music generation, image captioning, and predicting stock market fluctuations.
5. Long Short-Term Memory Network
In RNNs, we can only retain information from the most recent stages. But for problems like language translation, we need to retain more. This is where LSTM networks excel.
To learn long-term dependencies, our neural networks need memory capabilities. LSTMs are a special case of RNNs that can achieve this. They have the same chain structure as RNNs but with different repeating module structures. This repeating module structure allows the network to retain a large amount of previous stage values.
I have mentioned the powerful capabilities of LSTM networks for language translation systems, but they have a wide range of applications. Some of these applications include sequence-to-sequence modeling tasks such as anomaly detection, speech recognition, text summarization, and video classification.
6. Generative Adversarial Network
Given training data, Generative Adversarial Networks (or simply GANs) learn to generate new data using the same statistics as the training data. For example, if we train a GAN model on photos, the trained model will be able to generate new photos similar to the input photos.
GANs consist of two parts: a generator and a discriminator. The generator model creates new data, while the discriminator tries to determine real data from the generated data. As the generator and discriminator improve in their respective tasks, the generated data thus gets better until (ideally) the quality is nearly the same as the training data.
GANs are commonly used to create cartoon images or faces for games and animated movies. Additionally, GANs can help generate synthetic data from a small amount of data to improve machine learning models.
Editor / Zhang Zhihong
Reviewer / Fan Ruiqiang
Rechecker / Zhang Zhihong
Click below
Follow us
Read original article