Introduction: Hello, readers of the “Beginner’s Data Learning” series! It has been a while. Google recently launched a neural network visualization teaching platform called “TensorFlow Playground”. You can now play with neural networks right in your browser! Isn’t that amazing? After trying it out with the beginner, you’ll definitely feel like, “Aha, this is what a neural network is, I get it now!”
Now, let me share the website with you: http://playground.tensorflow.org. Overall, it looks like this:
Let’s learn more about this neural network playground!
◆ ◆ ◆
Several Basic Questions
Beginner: I’ve forgotten what a neural network is; can you help me review it?
Answer: Sure! Those who want to review the previous “Beginner’s Data Learning” series can find the basic article on neural networks in the historical articles at the end. In simple terms, a neural network is a technology that learns from data through computer programs. It is based on our current understanding of how the human brain works (note that it is based on this understanding because the human brain is a very complex neural network, and artificial neural networks are actually a very simplified representation of our understanding of how the brain works). First, we need to create a bunch of interconnected neurons that can transmit information to each other. Then we want to use this network to solve a problem. We don’t expect the network to find the correct answer immediately; it can continuously self-adjust, strengthening the connections that lead to correct outputs and weakening those that lead to incorrect outputs. For more details about neural networks, please refer to Michael Nielsen’s “Neural Networks and Deep Learning” (http://neuralnetworksanddeeplearning.com/index.html). If you want a more technical introduction to neural networks, please refer to the book “Deep Learning” by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (http://www.deeplearningbook.org/).
Beginner: What is a neuron?
Answer: Let’s look at a very simplified image. Essentially, it looks like this.
Isn’t it simple and straightforward? The big circle in the image represents a neuron, and the f(·) inside it is called the activation function. Its input, as shown in the image, is the total sum of all inputs xi multiplied by the corresponding weights wji.
Beginner: What is an activation function?
Answer: An activation function, as the name suggests, is the function that activates that neuron. To clarify, for example, a step function (as shown in the example below) can be an activation function; once its input reaches a certain value, its output jumps to that value, achieving activation and completing the process of qualitative change.
Beginner: So how does the network “self-adjust continuously”?
Answer: Let’s look at a slightly more complex diagram. Do you see the “Error” in there? That represents a backpropagation process. When we say the network’s “self-adjustment”, we refer to correcting the next input based on the difference between this output and the correct output. So how do we correct it? We use the gradient descent algorithm. You can understand gradient descent like this: there are three slides in front of you, one at 30 degrees, one at 60 degrees, and the last one at 90 degrees. If you want to reach the ground the fastest, you would definitely choose the 90-degree slide. This selection process can be called the gradient descent algorithm.
Beginner: Cool, can I build a network according to my ideas?
Answer: Of course! On the TensorFlow Playground platform, we can choose different parameters and designs, and the training of the neural network will be visually displayed for each iteration. Google has put the open-source code on GitHub to make it easier for beginners to learn about neural networks. At the very top of the webpage, we will see a list of parameter options. Let’s explain these options one by one:
Activation is the activation function, defining the output of each neuron. Google provides four options. Users can try them one by one and clearly understand their differences through visualization.
Learning Rate is the learning rate, which determines the step size of each learning step. This is related to the gradient learning we discussed earlier. When optimizing using the gradient descent algorithm, a coefficient is multiplied in front of the gradient term, and this coefficient is called the learning rate. If the learning rate is too small, the function converges very slowly; if it is too large, it may fail to find the extreme value, or even the function may not converge.
Beginner: By the way, how should we choose the learning rate?
Answer: Good question! To explain this, we need to introduce another term called Loss. Loss refers to the difference between the predicted value and the actual value. The smaller the loss, the more accurate the model’s predictions, and the better the model. The Playground allows you to visually see the trend of Training Loss and Testing Loss after each iteration in the upper right corner. The ideal situation is that both Loss values gradually decrease, indicating that the model is becoming more accurate. If Training Loss decreases while Testing Loss increases, we may be overfitting. For different problems, there are suitable methods to calculate Loss, which you may often hear referred to as the loss function or cost function.
Returning to the learning rate selection issue, it generally requires continuous trial and error. One practical method for selecting the learning rate is to set it to 0.01 initially and then observe the trends of Testing and Training Loss. If Loss keeps decreasing, you can gradually increase the learning rate. If Loss increases or fluctuates, you need to decrease the learning rate. After some trials, you can roughly determine a suitable value for the learning rate. The trend graph of Loss after each iteration can be clearly seen in the OUTPUT section on the right side of the Playground platform.
Beginner: I get it! What kind of problems can this platform simulate?
Answer: Let’s go back to the parameter setting section.
Problem type includes two types of problems that neural networks can solve: 1) Classification, which involves discrete problems. Here, we can visually see that our problem is to build a model that successfully distinguishes between orange and blue points.
2) Regression, which involves continuous problems.
The left image above shows four example instances for classification problems provided by Google, while the right image shows two examples for regression problems.
On the left side of the page, there are additional parameters that can be set: the ratio of training to test data indicates how much of the total dataset is taken as training data. Training data refers to the data used to adjust the network, while test data refers to the data used to validate the network’s effectiveness. Training data + test data = total data volume. You can also set the noise level and the amount of data used in each training session. Noise can be set to determine whether to include noise. Batch size refers to how many data points to calculate in each iteration. There is also a “REGENERATE” button; pressing this button will regenerate a random dataset according to the parameters you have selected.
Beginner: So we can only choose a specific problem from the provided experiments and data, but we can freely adjust the parameters.
Answer: Yes. Next, in the middle section, we can choose the features we want to input into this neural network and the number of layers, as well as how many neurons are in each layer.
Beginner: Can you explain what FEATURES means?
Answer: This refers to whether we need to input features into the network during training. For example, when describing a person, you would use the features that best distinguish them from others. The features we refer to here have the same meaning; they are the things that can best describe the characteristics of the input data (these “things” can be represented by functions or parameter values). Now looking back at the selectable FEATURES in the screenshot, these data are all two-dimensional, where X1 represents the horizontal coordinate value and X2 represents the vertical coordinate value. Once you understand this, the other feature values become easy to comprehend.
Depending on the problem, we need to input different features. For complex classification problems like the one shown above, simply using X1, X2, X12, and X21 may not be sufficient to create a good model; in this case, we would consider cross-features like X1X2. We can also restrict the features and let the neural network learn useful features by continuously iterating and adjusting the weights of the neurons. In practical applications, many image recognition problems can be solved without manual feature extraction, directly operating on the raw images.
Beginner: What do the colors displayed in the network represent?
Answer: Orange represents negative values, while blue represents positive values. Here, we initialize the data with -1 (orange) or +1 (blue) (these data are represented by small circles). In the hidden layers, the color of the lines between layers is determined by the weights connecting the neurons. Blue represents positive weights, while orange represents negative weights. The input value of the neuron on the right of the connecting line is determined by the output value of the neuron on the left, the corresponding weight of the connecting line, and the activation function. In the output layer, the color of the points is determined by their respective values. The background color in the output layer represents the predicted value for a specific area, and the intensity of the background color indicates the confidence level of the prediction.
Beginner: This platform can do a lot! What library do they use?
I might want to learn more in-depth.
Answer: This library is a small program library developed by Google specifically for this visualization teaching tool (link here: https://github.com/tensorflow/playground/blob/master/nn.ts). If you want to do practical applications, consider using Google’s open-source TensorFlow library (https://www.tensorflow.org/).
Of course, to truly understand machine learning, merely reading the beginner series and playing with Google’s “TensorFlow” playground is far from enough. Do you want to thoroughly and comprehensively understand what deep learning is and its specific applications in engineering? The “Machine Learning Training” by Big Data Digest is currently open for registration, just long-press the QR code below or click “Read the original text” at the end to learn more.
Recommended Previous Articles, click the image to read
Beginner’s Data Learning: Understand Machine Learning in One Article
Beginner’s Data Learning: Understand NoSQL Databases in One Article
Beginner’s Data Learning: NoSQL Advanced
Beginner’s Data Learning: What is a Neural Network
[Translated] Eight Steps to Become a Data Scientist
