Deep Learning (DL) is a new research direction in the field of Machine Learning (ML), introduced to bring machine learning closer to its original goal—Artificial Intelligence (AI).
Deep learning learns the intrinsic patterns and hierarchical representations of sample data, which significantly aids in interpreting data such as text, images, and sounds. Its ultimate goal is to enable machines to analyze and learn like humans, recognizing text, images, and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition far exceeding previous related technologies.
Deep learning is a new area of research in machine learning, motivated by the establishment and simulation of neural networks that analyze and learn like the human brain. It mimics the mechanisms of the human brain to interpret data, such as images, sounds, and text. Deep learning is a type of unsupervised learning. The concept of deep learning originated from research on artificial neural networks. Multi-layer perceptrons with multiple hidden layers are a type of deep learning structure. Deep learning combines low-level features to form more abstract high-level representations of attribute categories or features, discovering distributed feature representations of data. The concept of deep learning was proposed by Hinton et al. in 2006, introducing an unsupervised layer-by-layer training algorithm based on Deep Belief Networks (DBN), bringing hope for solving optimization problems related to deep structures. Subsequently, multi-layer autoencoders were proposed. Additionally, the convolutional neural networks proposed by Lecun et al. were the first true multi-layer structure learning algorithms, utilizing spatial relationships to reduce the number of parameters and improve training performance.
In the context of artificial intelligence, deep learning refers to the process of analyzing the inherent logic of things in the form of databases, using computer languages for recognition, reading, manipulation, feedback, correction, etc. Xiao Kun understands deep learning in humans as being similar to deep sleep, where a person in deep sleep enjoys the process and is completely immersed, unable to be awakened for a while?
It can be discussed from two aspects:
-
For deep learning in humans, is it necessary to clearly understand the essence of things, i.e., the inherent logic? For example, the discovery of Newton’s law of universal gravitation came from the inspiration of an apple falling to the ground, leading to the exploration of this rule. This is related to interest and perseverance.
-
Human deep learning should not be constantly interrupted. This is similar to sleep; a person who is constantly awakened will definitely not have high-quality sleep. Therefore, to achieve deep learning, time and space need to be well arranged, along with tools and materials, etc.
The reason deep learning generalizes so well is key to its successful applications:
Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, NIPS 2012., Clement Farabet, Camille Couprie, Laurent Najman and Yann LeCun, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013., Koray Kavukcuoglu, Pierre Sermanet, Y-Lan Boureau, Karol Gregor, Michaël Mathieu and Yann LeCun, Advances in Neural Information Processing Systems (NIPS 2010), 23, 2010.
Microsoft researchers, in collaboration with Hinton, first introduced RBM and DBN into speech recognition acoustic model training, achieving great success in large vocabulary speech recognition systems, reducing the error rate of speech recognition by approximately 30%. However, DNN still lacks effective parallel fast algorithms, and many research institutions are using large-scale data corpora on GPU platforms to improve the training efficiency of DNN acoustic models.
Internationally, companies like IBM and Google have rapidly conducted research on DNN speech recognition, moving at a fast pace.
Domestically, companies or research institutions such as iFlytek, Baidu, and the Institute of Automation of the Chinese Academy of Sciences are also conducting research on deep learning in speech recognition.
3. Natural Language Processing and Other Fields
Many institutions are conducting research, but deep learning has not yet produced systematic breakthroughs in natural language processing.
How to implement deep learning? Xiao Kun starts from the core idea of deep learning, which is to build a network, which is what we refer to as the deep learning neural network model. Deep learning can generally be summarized into the three steps shown in the following diagram:
-
The first step is that the neural network model is a complex function composed of simple functions. Typically, we design a neural network model (structure) and then train it using a computer from the given training data to obtain some parameters. These parameters ensure that our model can achieve the expected effect in the test set and have generalization ability.
-
The second step is to define a cost function based on the training data. The cost function can evaluate which parameters are effective and which are not very effective, what kind of functions in the model are good and which are not, and how to define a cost function is based on your specific task and actual training data.
-
The third step is to find the optimal function based on the results of the previous two steps, for example, by using the gradient descent method to find this optimal function.
Research on deep learning is an ongoing, open research process that continuously summarizes, refines, enhances, and renames all previous excellent teaching practices. It requires the joint efforts and exploration of more scholars.
After reading so much, do you have a general understanding of deep learning? Does it feel like the gateway to a new world? If you have developed a strong interest in deep learning, feel free to consult our professional guidance~
(Swipe left and right to view the poster)
———— e n d ————
Come and consult WeChat Assistant (Xiao Kun)~
City + Enrollment Year + School + Name + Major + Phone
