Convolutional Neural Networks: Neural Networks with Representation Learning Capabilities

1 Algorithm Introduction

Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that include convolutional computations and have a deep structure. They have emerged in recent years as an efficient recognition method that has gained widespread attention. The design inspiration for CNNs comes from the hierarchical processing capabilities of the animal visual system, where lower layers of the neural network extract shallow features of images (such as edge information), while higher layers extract deeper features (such as a specific pixel block), ultimately obtaining the overall features of the image. CNNs can perform both supervised and unsupervised learning. The parameter sharing of convolutional kernels within the hidden layers and the sparsity of inter-layer connections allow CNNs to learn grid-like features, such as pixels and audio, with a small amount of computation, achieving stable results without additional feature engineering requirements.

In summary, CNNs are a type of neural network specifically designed for processing image data, capable of translation-invariant classification of input information according to their hierarchical structure, and are thus also referred to as “translation-invariant artificial neural networks.”

2 Algorithm Principles

CNNs extract local features of input data through convolution operations and form complex feature representations through multiple layers of convolutions and pooling operations, ultimately performing tasks such as classification or regression through fully connected layers.

The basic structure of a convolutional neural network typically consists of the following parts:

(1) Input Layer: Receives raw image data or other types of grid-structured data.

(2) Convolutional Layer: Extracts local features of input data through convolution operations. Each convolutional kernel in the convolutional layer can extract a specific feature, and multiple convolutional kernels can work in parallel to extract different types of features.

(3) Pooling Layer: Downsamples (or reduces dimensionality) the output of the convolutional layer to decrease the number of parameters and improve computational efficiency. Common pooling operations include max pooling and average pooling.

(4) Fully Connected Layer: Integrates the features extracted from the previous layers for tasks such as classification or regression. Each neuron in the fully connected layer is connected to all neurons in the previous layer.

This structure gives CNNs the characteristics of local connectivity, weight sharing, and translation invariance. Specifically: (1) Neurons in the convolutional layer are only connected to a local region of the input data (i.e., the local receptive field), which helps capture local features of the image; (2) The same convolutional kernel shares weights across all positions in the input data, significantly reducing the number of parameters in the network and lowering model complexity; (3) Regardless of where features appear in the input data, the convolution operation can extract the same features, making convolutional neural networks highly efficient and accurate when processing grid-structured data like images.

3 Algorithm Applications

CNNs have become a primary method in computer vision tasks due to their unique performance in object detection and image recognition. They are currently widely applied in the field of traditional Chinese medicine for recognizing tongue color and shape, as well as constructing syndrome prediction models. Scholars from the China Academy of Chinese Medical Sciences have established a syndrome prediction model for ulcerative colitis based on convolutional neural networks, providing a basis for the clinical diagnosis and treatment of this disease in traditional Chinese medicine. The specific construction method is as follows: (1) Collect electronic medical records of 9,186 patients diagnosed with “ulcerative colitis” at the Dongfang Hospital of Beijing University of Chinese Medicine; extract the database using Python 3.7, converting syndrome names into numbers for program processing; check the number of traditional Chinese medicine syndromes and the number of cases included in each syndrome, selecting data with a quantity >30 as training data, such as ‘Spleen and Kidney Yang Deficiency, Damp Heat Stagnation’:1, ‘Spleen Qi Deficiency, Damp Heat Obstruction’:2, ‘Spleen Yang Insufficiency, Damp Heat Obstruction’:3, ‘Phlegm Heat Stagnation, Qi Obstruction’:4, ‘Spleen Qi Deficiency with Phlegm Heat Stagnation’:5, ‘Spleen Qi Deficiency with Damp Heat Stagnation’:6, totaling 9,186 pieces of data, of which 7,348 pieces (80%) were randomly selected as training data (see Table 1). (2) Use convolutional neural networks to construct a syndrome classification model based on current symptoms. This study built a convolutional neural network algorithm model using the commonly used TensorFlow 1.0 platform and Python 3.7, calling the machine learning algorithm metrics from the Scikit-learn library to help measure and evaluate the accuracy of the algorithm. After completion, test code was run, and the results were converted into a txt document with three fields: original syndrome name, predicted syndrome name, and consultation status for easier identification and analysis. (3) Select the GRU algorithm from recurrent neural networks as a baseline control system to validate the effectiveness of the convolutional neural network. The variant of long short-term memory networks, GRU, combines the forget gate and input gate into a single update gate, making it simpler than the standard LSTM model, minimizing memory consumption and saving training time while maintaining the same operating environment as CNN. (4) Use this model to predict diagnostic classifications of the six syndrome types in the test set. The experimental results are as follows: (1) Both CNN and GRU ran for 14 epochs to achieve optimal results, with the test accuracy of CNN being 88% and recall rate being 88% (see Figure 1), while the test accuracy of GRU was 86% and recall rate was 86% (see Figure 2). (2) In the analysis of mispredicted and correctly predicted data, GRU had 267 erroneous predictions, while CNN had 163 errors (see Table 2), with an overlap rate of 61.04%. CNN had 236 errors, of which 163 were also errors from GRU, resulting in an overlap rate of 69.06%. The number of medical cases predicted correctly by both algorithms reached 1,603, with an overlap rate of 87.21%.

Table 1 Data Statistics and Annotation

Figure 2 Experimental Results Based on CNN

Figure 2 Experimental Results Based on GRU

Table 2 Some Incorrect Prediction Data

Thus, it can be seen that both CNN and GRU neural network classification models, based on the same training corpus, perform well, exceeding 85%, and have certain clinical practicality, serving as references for grassroots or less experienced doctors in syndrome differentiation. In comparison, although GRU has certain advantages in learning the nonlinear features of sequences, it is slightly inferior to CNN in this study. This shows that CNN, as a classic image recognition algorithm, also performs excellently in text classification, and CNN’s running speed is several times faster than that of GRU, making it a highly practical deep learning algorithm.

4 Conclusion

Traditional machine learning algorithms are limited by the constraints of natural language understanding in intelligent decision-making, with little progress in improving accuracy, while convolutional neural networks, as commonly used neural network algorithms, show good performance in the classification and prediction of natural languages such as traditional Chinese medicine syndromes, making them more practical and effectively advancing the informatization process of traditional Chinese medicine.

In summary, convolutional neural networks are powerful deep learning models that demonstrate outstanding performance in various fields such as computer vision, natural language processing, and speech recognition. With the continuous development of technology, the application prospects of convolutional neural networks will be even broader.

References:

[1] Zhu Ling, Zheng Wanting, Zhang Zhulu, et al. Research on Syndrome Prediction Model of Ulcerative Colitis Based on Convolutional Neural Network [J]. China Digital Medicine, 2022, 17(04):49-55.

[2] Convolutional Neural Networks (CNN) – Detailed Interpretation of Principles (Easy to Understand) – CSDN Blog, accessed on August 7, 2024.

https://blog.csdn.net/m0_71212744/article/details/140370628.

[3] Convolutional Neural Networks (CNN) – CSDN Blog, accessed on August 7, 2024.

https://blog.csdn.net/fuhanghang/article/details/135544761.

Recommended Reading:

Transformer: Deep Learning Model Based on Self-Attention Mechanism

Understanding an Algorithm Every Week – SVM, a Generalized Linear Classifier for Binary Classification

Understanding an Algorithm Every Week – ID3, an Important Tool for Classifying and Predicting Large-scale Medical Data

Ancient and Modern Medical Case Cloud Platform

Providing over 500,000 ancient and modern medical case search services

Supports manual, voice, OCR, and batch structured entry of medical cases

Designed with nine analysis modules, close to clinical practical needs

Supports collaborative analysis of massive medical cases and personal medical cases

EDC Traditional Chinese Medicine Research Case Collection System

Supports multi-center, online random grouping, data entry

SDV, audit trail, SMS reminders, data statistics

Analysis and other functions

Supports customized form design

Users can log in at: https://www.yiankb.com/edc

Free trial!

Institute of Information Research, China Academy of Chinese Medical Sciences

Smart R&D Center for Traditional Chinese Medicine Health

Big Data R&D Department

Phone: 010-64089619

13522583261

QQ: 2778196938

https://www.yiankb.com