A Comprehensive Guide to Getting Started with Neural Networks

The term “neural” is the adjective form of “neuron,” and “network” refers to a graph-like structure; therefore, an “artificial neural network” is a computational system that attempts to mimic (or at least is inspired by) the neural connections in our nervous system. Artificial neural networks are also known as “neural networks” or “artificial neural systems.” It is common to abbreviate artificial neural networks as “ANN” or simply “NN”—both abbreviations will be used below.

Artificial neural networks (ANNs) consist of an input layer, one or more hidden layers, and an output layer. Each node or artificial neuron is connected to another node and has associated weights and thresholds. If the output of any single node exceeds a specified threshold, that node is activated and sends a signal to the next layer of the network. Otherwise, no signal is passed on to the next layer of the network.

Over time, neural networks learn and improve their accuracy based on training data. However, once these learning algorithms have been fine-tuned to enhance accuracy, they become powerful tools in computer science and artificial intelligence, enabling us to quickly classify and cluster data. Tasks like speech recognition or image recognition can take minutes instead of hours compared to manual identification by human experts. One of the most famous neural networks is Google’s search algorithm.

Information flows through a neural network in two ways. When it is learning (being trained) or operating normally (after being trained), information is input into the network through input units, triggering layers of hidden units that subsequently reach the output units. This common design is called a feedforward network. Not all units are always “firing.” Each unit receives input from its left-side units, with the input multiplied by the weights of the connections it passes through. Each unit sums all the inputs it receives in this way, and (in the simplest type of network) if the total exceeds a certain threshold, that unit “fires” and activates the units it is connected to (the units on its right).

For a neural network to learn, it must include feedback elements—just as children learn by being told whether what they did was right or wrong. In fact, we have always been using feedback. Think back to when you first learned to kick a soccer ball; when you shot, your brain observed the speed of the ball and the path it took, noting how close it was to the goal. The next time you shot, you would recall what you did wrong before and adjust your actions accordingly, hoping to kick the ball better. Thus, you use feedback to compare the desired outcome with the actual outcome, find the difference between the two, and use it to change what you do next (“I need to try harder”).

Neural networks learn things in exactly the same way, typically through a feedback process called backpropagation (sometimes abbreviated as “backprop”). This involves comparing the output produced by the network with the output it intended to produce and using the difference to modify the weights of the connections between units in the network—from the output units to the hidden units to the input units—in other words, backward. Over time, backpropagation allows the network to learn, reducing the difference between actual output and expected output to the point where both are completely aligned, thus allowing the network to compute exactly as intended.

Neural networks are sometimes described based on their depth, including how many layers they have between input and output, or the so-called hidden layers of the model. This is why the term neural network is almost synonymous with deep learning. They can also be described by the number of hidden nodes the model has or how many inputs and outputs each node has. Variants of classic neural networks allow information to be propagated in various forms of forward and backward propagation between layers.

Specific types of artificial neural networks include:

l Feed-forward neural networks: One of the simplest variants of neural networks. They pass information in one direction through various input nodes until it reaches the output node. The network may or may not have hidden node layers, making their functionality easier to explain. This type of ANN computational model is used in technologies such as facial recognition and computer vision.

l Recurrent neural networks: This network is more complex; they save the output of processing nodes and feed the results back into the model, which is how the model learns to predict layer results. Each node in an RNN model acts as a storage unit, continuing to compute and perform operations. The neural network starts with forward propagation just like a feedforward network but then continues to remember all the information it has processed for later use. If the network’s predictions are incorrect, the system self-learns and continues to move in the direction of correct predictions during backpropagation. This type of ANN is often used for text-to-speech conversion.

l Convolutional neural networks: One of the most popular models today. This neural network computational model uses a variant of multilayer perceptrons and includes one or more convolutional layers that can be fully connected or pooled. It has been used in many state-of-the-art AI applications, including facial recognition and natural language processing. Other uses include semantic detection, signal processing, and image classification.

l Deconvolutional neural networks: Utilize the reverse CNN model process. Their goal is to find lost features or signals that may initially have been deemed unimportant for CNN system tasks. This network model can be used for image synthesis and analysis.

l Modular neural networks: Contain multiple neural networks that work independently of each other. During computation, the networks do not communicate or interfere with each other’s activities. Therefore, complex or large computational processes can be performed more efficiently.

Advantages of artificial neural networks:

v The ability to generalize and infer unseen relationships from unseen data means that artificial neural networks can predict outputs for unseen data.

v Artificial neural networks can endure longer training times, depending on factors such as the number of weights in the network, the number of training examples considered, and the settings of various learning algorithm parameters.

v The learning method of ANNs is very robust against noise in the training data. Training examples may contain errors that do not affect the final output.

Disadvantages of artificial neural networks:

v The requirement for processors with parallel processing capabilities means that neural networks depend on hardware.

v The lack of rules to determine an appropriate network structure means that suitable artificial neural network architectures can only be found through trial and error and experience.

v The lack of explanation behind the exploration solutions is one of the biggest downsides of artificial neural networks. The inability to explain the reasons or methods behind the solutions leads to a lack of trust in the network.

Image recognition was one of the first fields where neural networks were successfully applied, but the use of this technology has expanded to more areas, including:

l Computer vision (including image and video domains)

l Speech domain (such as speech recognition)

l Natural language processing (mainly focused on text)

l Other aspects of life, such as intelligent driving and healthcare

It can be said that the ways people communicate in life are nothing more than text communication (emails, letters), voice communication (phone calls, instant voice), and forms such as images and videos. Therefore, the problems that neural networks (specifically deep learning) can solve indeed cover various aspects of life.