Big Data Digest authorized reprint from AI Technology Review Authors: Li Mei, Huang Nan Editor: Chen Caixian In the past decade, deep learning has achieved remarkable victories, with methods using large parameters and data through stochastic gradient descent proven effective. The gradient descent typically uses the backpropagation algorithm, which has led to ongoing questions about … Read more
Produced by Big Data Digest Compiled by: Shijin Tian, Ni Ni, Hu Jia, Yun Zhou In many machine learning labs, machines have undergone thousands of hours of training. During this process, researchers often take many detours and fix many bugs, but it is certain that the experience and knowledge gained during the research process are … Read more
Reported by New Intelligence Editor: LRS [New Intelligence Overview] It has almost become a consensus that bigger neural networks are better, but this idea contradicts traditional function fitting theory. Recently, researchers from Microsoft published a paper at NeurIPS proving the necessity of large-scale neural networks mathematically, suggesting they should be even larger than expected. As … Read more
Produced by Big Data Digest Compiled by: Andy The backpropagation algorithm belongs to deep learning and plays an important role in solving model optimization problems. This algorithm was proposed by Geoffrey Hinton, known as the father of deep learning. In 1986, he published a paper titled “Learning representations by back-propagating errors” (Rumelhart, Hinton & Williams, … Read more
Produced by Big Data Digest Authors:Ning Jing, Wei Zimin Have you ever thought about moving neural networks from computers into a piece of glass? Using neural networks for image recognition and intelligent recommendations has become very common. In recent years, the increase in computing power and parallel processing has made it a very practical technology. … Read more
Produced by Big Data Digest Source: eisenjulian Compiled by: Zhou Jiale, Qian Tianpei Using deep learning libraries like TensorFlow and PyTorch to write a neural network is no longer a novelty. But do you know how to elegantly build a neural network using Python and NumPy? Nowadays, there are many deep learning frameworks available, equipped … Read more
Soviet mathematician Andrey N. Kolmogorov (1903-1987). Image source: https://wolffund.org.il/ Introduction: Large models pose new questions to computational theory, which can also assist large models in revealing first principles, thereby finding boundaries and directions. For example, the KA Superposition Theorem completed by Soviet mathematician Kolmogorov and his student Arnold in the 1950s. Nick | Author Xiaoxue … Read more
-Distilling the Knowledge in a Neural Network Geoffrey Hinton∗†Google Inc. Mountain View [email protected] Oriol Vinyals† Google Inc. Mountain View [email protected] Jeff Dean Google Inc. [email protected] Abstract A simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then average their predictions.[3] Unfortunately, … Read more
In the field of deep learning, there are many specialized terms that can be quite overwhelming at first glance. However, as we delve deeper, we gradually start to understand them, albeit feeling like something is still missing. Today, we will discuss a specialized term called Attention mechanism! 1. Intuitive Understanding of Attention Imagine a scenario … Read more
The attention mechanism is mentioned in both of the following articles: How to make chatbot conversations more informative and how to automatically generate text summaries. Today, let’s take a look at what attention is. This paper is considered the first work using the attention mechanism in NLP. They applied the attention mechanism to Neural Machine … Read more