Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

Basic Mathematical Principles

The main differences in artificial neural network models lie in the topology of the neural network connections, the characteristics of the neurons, and the scale of learning, etc. Based on the differences in the topology of neuron connections, we can roughly categorize neural network models into the following two types.

1. Feedforward Network

In a feedforward network, each neuron only receives input from the previous layer and outputs to the next layer, with no feedback in the neural network. Its topology can be represented as a directed acyclic graph. This network implements the transformation of signals from the input space to the output space, and its information processing capability lies in the finite composition of simple nonlinear functions.

The simplest feedforward network structure is a two-layer feedforward network, which consists only of an input layer and an output layer. Generally, a feedforward neural network with one input layer, one output layer, and multiple hidden layers is called a multilayer feedforward network.

2. Feedback Network

Another type of neural network model is called a feedback network, where there is feedback between neurons. Its topology can be represented as an undirected complete graph. Its information processing capability stems from state transformations, which can be handled using dynamical system theory. The stability of the system is closely related to associative memory functions.

Common Functions

When building a feedforward neural network using MATLAB, the following three functions are primarily used:

newff: Function to create a feedforward neural network;

train: Function to train a neural network;

sim: Function to simulate using the neural network.

Below is a brief introduction to the usage of these three functions.

01

Syntax of newff Function

Syntax: net=newff(P, T, [S1 S2…S(N-1)], {TF1 TF2…TFN}, BTF, BLF, PF, IPF, OPF, DDF)

P: Input matrix vector;

T: Target matrix vector;

[S1 S2…S(N-1)]: Number of neurons in each of the first N-1 layers of the neural network;

{TF1 TF2…TFN}: Activation functions of the neural network, default is ‘tansig’;

BTF: Training algorithm used for the learning rule, default is ‘trainlm’;

BLF: BP weight/bias learning function, default is ‘learngdm’;

PF: Performance function, default is ‘mse’;

IPF: Input processing function;

OPF: Output processing function;

DDF: Validation data partition function. Generally, the first seven parameters are set during use, while the last three parameters can use system default values.

Common Activation Functions

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

Only when you want to limit the output of the network (e.g., to restrict it between 0 and 1) should the S-shaped activation function be included in the output layer. Generally, the bipolar S-shaped activation function is used in the hidden layers, while the linear activation function is used in the output layer.

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)
Neural Network Algorithm in Intelligent Optimization (With Matlab Code)
Neural Network Algorithm in Intelligent Optimization (With Matlab Code)
Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

The train function is the neural network training function.

Syntax: [net, tr, Y1, E] = train(net, X, Y)

X: Network input matrix; Y: Network output matrix; tr: Training tracking information; Y1: Actual output of the network; E: Error matrix.

The sim function is the neural network simulation calculation function.

Syntax: Y = sim(net, X)

net: Trained neural network; X: Network input matrix; Y: Network output matrix.

MATLAB Simulation Example

Using Bayesian regularization algorithm to improve the generalization ability of the BP network. This is used to train the BP network to fit a sine sample data with added white noise.

Solution: The simulation process is as follows:

(1) Construct a 3-layer BP neural network: 1 input node, 3 hidden layer nodes, and the activation function of the hidden layer is ‘tansig’; 1 output node, and the activation function of the output layer is ‘logsig’.

(2) Train the BP network using the Bayesian regularization algorithm ‘trainbr’, with a target error goal=1×10-3, learning rate lr=0.05, maximum iteration epochs=500, fitting the sine sample data with added white noise, and the fitting data root mean square error is 0.0054. The fitted graph is shown below.

BP Network Fitting Sine Curve with Added White Noise

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

The MATLAB source code is as follows:

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

The table shows the monthly sales of a certain drug, using the BP neural network to predict the sales of the drug. The prediction method uses a rolling forecast, i.e., using the sales volume of the previous three months to predict the sales volume of the fourth month. For example, using the sales volumes of January, February, and March as input to predict the sales volume for April; using the sales volumes of February, March, and April as input to predict the sales volume for May. This process is repeated until the prediction accuracy requirement is met.

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

Solution: The simulation process is as follows:

(1) Construct a 3-layer BP neural network to predict the drug sales: 3 input nodes, 5 hidden layer nodes, and the activation function of the hidden layer is ‘tansig’; 1 output node, and the activation function of the output layer is ‘logsig’.

(2) Train the BP network using gradient descent momentum and adaptive lr algorithm ‘traingdx’, with a target error goal=1×10(-3), learning rate lr=0.05, maximum iteration epochs=1000. The comparison curve of actual and predicted sales values is shown below.

Comparison Curve of Actual and Predicted Drug Sales Values

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

The MATLAB source code is as follows:

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

The table shows the historical statistical data of highway transportation capacity in a certain area. Please establish the corresponding BP neural network prediction model and predict the corresponding highway passenger and freight volume based on the data from 2010 and 2011.

Solution: The simulation process is as follows:

(1) Construct a 3-layer BP neural network to predict the highway transportation capacity: 3 input nodes, 8 hidden layer nodes, and the activation function of the hidden layer is ‘tansig’; 2 output nodes, and the activation function of the output layer is ‘purelin’.

(2) Train the BP network using gradient descent momentum and adaptive lr algorithm ‘traingdx’, with a target error goal=1×10-3, learning rate lr=0.035, maximum iteration epochs=2000. The fitted curves for historical highway passenger volume and freight volume are shown in Figures 9.6 and 9.7, respectively. The predicted results are: 2010 highway passenger volume is 452.77 million people, highway freight volume is 222.90 million tons; 2011 highway passenger volume is 453.08 million people, highway freight volume is 222.96 million tons.

Historical Highway Passenger Volume Fitting Curve

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

Historical Highway Freight Volume Fitting Curve

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

The MATLAB source code is as follows:

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

Code Acquisition

Reply “Neural Network Algorithm” in the backend to obtain the code text for free.

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)
Today’s Benefit! Get it for Free
The organizer has compiled about 50 common algorithm packages
Including neural networks, gray correlation, interpolation and fitting, clustering analysis
Gray prediction, genetic algorithm, entropy weight method
Data envelopment algorithm, TOPSIS method and other model algorithms’ codes
Can be run directly

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

⌂ Part of the Screenshot

Reply in the public account backend“Algorithm Package”
to get the above materials for free

Neural Network Algorithm in Intelligent Optimization (With Matlab Code)

Scan to reply“Algorithm Package”
to get the above materials for free

Leave a Comment