The BP neural network (Back Propagation) is a multi-layer network that propagates signals forward and errors backward. This network has very strong computational capabilities and can solve many nonlinear problems by analyzing the nonlinear mapping relationship between input and output variables through training samples.
1. Structure of BP Neural Network
The structure of the BP neural network includes the input layer, hidden layer, and output layer.
- Input Layer: Responsible for receiving external data, which is input in vector form. The number of neurons in the input layer depends on the number of features in the input data.
- Hidden Layer: Can have one or more layers, where hidden layer neurons are connected to the input layer and the next layer via weights. The hidden layer plays a role in feature extraction and nonlinear transformation, allowing it to learn complex internal patterns in the input data. The number of neurons in the hidden layer is not fixed and is usually adjusted based on experience and experimentation; it can also be optimized as a parameter.
- Output Layer: Outputs the predicted results, where the number of neurons in the output layer depends on the specific task. For example, in a binary classification problem, the output layer may have only one neuron, comparing the output value with a certain threshold to determine the category; in a multi-class classification problem, the number of output layer neurons equals the number of categories, with each neuron outputting the probability of the corresponding category.
The training process of the BP neural network mainly includes two stages: forward signal propagation and backward error propagation.
(1) The first stage is the forward propagation process
At the beginning of training, multiple sets of training samples are input into the network’s input layer. The data is processed through the input layer and hidden layer, and from the hidden layer to the output layer, ultimately obtaining the computed results from the output layer. The training sample data pairs include the input values of the samples and the expected output values. When the training samples are transmitted between different network layers, they need to be processed through the weights, thresholds, and transfer functions between the input layer and the hidden layer, as well as those between the hidden layer and the output layer. The output values of the input training samples are computed alongside the corresponding expected output values, and the total error of all samples is calculated. The convergence of the error function determines whether to proceed to the second stage;
(2) The second stage is the backward propagation process: this means returning the information along the original path, using gradient descent and the error functions of all training samples to update the connection weights and thresholds, minimizing the output error so that the output values are closer to the expected outputs.
Figure 1: Basic Network Diagram of BP Neural Network
Although the BP neural network has strong nonlinear mapping and learning capabilities, it is prone to getting stuck in local minima during the learning process due to the random selection of initial connection weights and thresholds, making it difficult to obtain the global optimal solution. Therefore, some optimization algorithms are employed to optimize the structure of the BP neural network, mainly optimizing the initial weights and thresholds of the neural network. Common optimization algorithms include genetic algorithms and ant colony algorithms.
Note:Formulas are difficult to edit, so screenshots were taken directly.
Copyright Statement: Reproduction is prohibited! Personal opinions for reference only; if there is infringement, it will be deleted immediately!
Follow the WeChat public account, and in the chat dialog box, click “Machine Learning” -> “BP Algorithm” to obtain the BP neural network algorithm for Matlab and Python versions for free.