[Introduction] This article provides the background and overview of Dropout, as well as a parameter analysis of its application in language modeling using LSTM/GRU recurrent neural networks. Author｜Adrian G Compiled by｜Zhuanzhi Organized by｜Yingying Dropout Inspired by the role of gender in evolution, Hinton et al. first proposed Dropout, which temporarily removes units from the neural … Read more
Big Data Digest Works Compiled by: Wan Jun, Da Jie Qiong, Qian Tian Pei Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks, which have been incredibly popular, it’s time to abandon them! LSTM and RNN were invented in the 1980s and 1990s, resurrected in 2014. In the following years, they became the go-to … Read more
Click the above “MLNLP” to select the “star” public account Essential content delivered promptly Author: Yulin Ling CS224N（1.29）Vanishing Gradients, Fancy RNNs Vanishing Gradient The figure below is a more vivid example. Suppose we need to predict the next word after the sentence “The writer of the books”. Due to the vanishing gradient, the influence of … Read more
Author: Su Jianlin Affiliation: Guangzhou Flame Information Technology Co., Ltd. Research Direction: NLP, Neural Networks Personal Homepage: kexue.fm I had originally decided to stop working with RNNs as they actually correspond to numerical methods for ODEs (Ordinary Differential Equations). This realization provided me with insights into something I have always wanted to do—using deep learning … Read more
↑↑↑ Follow “Star Mark” Datawhale Daily Insights & Monthly Study Groups, Don’t Miss Out Datawhale Insights Focus: Neural Networks, Source: Artificial Intelligence and Algorithm Learning Neural networks are the carriers of deep learning, and among neural network models, the most classic non-RNN model belongs here. Although it is not perfect, it possesses the ability to … Read more
0. Introduction After reading many blog posts and tutorials about RNN online, I felt they were all the same, providing a vague understanding but failing to explain it clearly. RNN is the foundation of many complex models, and even in transformers, you can see the influence of RNN, so it is essential to have a … Read more
Renormalization group is a fundamental concept in physics research. It is not only a powerful tool for studying phase transitions and critical phenomena, as well as strong coupling problems, but it also shapes physicists’ worldview: physics is an effective theory about the emergence of phenomena at different scales and energy levels. In the practical applications … Read more
Compiled by Yu Yang | QbitAI Official Account Want to adjust your neural network to complete specific tasks? It’s not as simple as it seems. Deep Neural Networks (DNNs) are great building blocks, but moving them can be very costly in terms of computational resources and time. Now, Google AI has released MorphNet. After testing … Read more
Canonical transformations are classical methods used by physicists, mechanical engineers, and astronomers to handle Hamiltonian systems. By finding suitable variable substitutions, canonical transformations can simplify, or even completely solve the dynamics of Hamiltonian systems. For instance, in the 19th century, French scientist Charles Delaunay published approximately 1800 pages of analytical derivations attempting to simplify the … Read more
5.4 The Intellectual The Intellectual Image Source: Freepik ● ● ● Written by｜Zhang Tianrong As physicist and Manhattan Project leader Oppenheimer said, “We are not just scientists; we are also human.” Where there are humans, there is a community, and the scientific world is no exception. People often say that “science knows no borders, but … Read more