Artificial Neural Networks

Introduction to Recurrent Neural Networks (RNN): Vector to Sequence, Sequence to Sequence, Bidirectional RNN, Markovization

by
Introduction to Recurrent Neural Networks (RNN): Vector to Sequence, Sequence to Sequence, Bidirectional RNN, Markovization

Author: David 9 Address: http://nooverfit.com/ RNN seems to be better at information preservation and updating, while CNN seems to excel at precise feature extraction; RNN has flexible input and output dimensions, while CNN dimensions are relatively rigid. 1Question When talking about Recurrent Neural Networks, our first reaction might be: time sequence. Indeed, RNNs are good … Read more

A Detailed Explanation of RNN Stock Prediction (Python Code)

by
A Detailed Explanation of RNN Stock Prediction (Python Code)

Recurrent Neural Networks (RNN) are designed based on the recursive nature of sequential data (such as language, speech, and time series) and are a type of feedback neural network that contains loops and self-repetitions, hence the name “recurrent”. They are specifically used to handle sequential data, such as generating text word by word or predicting … Read more

New RNN: Independent Neurons for Improved Long-Term Memory

by
New RNN: Independent Neurons for Improved Long-Term Memory

In an era flooded with fragmented reading, fewer people pay attention to the exploration and thinking behind each paper. In this column, you will quickly get the highlights and pain points of selected papers, keeping up with the forefront of AI achievements. Click the “Read Original” at the bottom of this article to join the … Read more

Overview of Dropout Application in RNNs

by
Overview of Dropout Application in RNNs

[Introduction] This article provides the background and overview of Dropout, as well as a parameter analysis of its application in language modeling using LSTM/GRU recurrent neural networks. Author｜Adrian G Compiled by｜Zhuanzhi Organized by｜Yingying Dropout Inspired by the role of gender in evolution, Hinton et al. first proposed Dropout, which temporarily removes units from the neural … Read more

Attention Models: The Future Beyond RNN and LSTM

by
Attention Models: The Future Beyond RNN and LSTM

Big Data Digest Works Compiled by: Wan Jun, Da Jie Qiong, Qian Tian Pei Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks, which have been incredibly popular, it’s time to abandon them! LSTM and RNN were invented in the 1980s and 1990s, resurrected in 2014. In the following years, they became the go-to … Read more

Solving the Vanishing Gradient Problem in RNNs

by
Solving the Vanishing Gradient Problem in RNNs

Click the above “MLNLP” to select the “star” public account Essential content delivered promptly Author: Yulin Ling CS224N（1.29）Vanishing Gradients, Fancy RNNs Vanishing Gradient The figure below is a more vivid example. Suppose we need to predict the next word after the sentence “The writer of the books”. Due to the vanishing gradient, the influence of … Read more

Introduction to RNN and ODE: Understanding RNNs

by
Introduction to RNN and ODE: Understanding RNNs

Author: Su Jianlin Affiliation: Guangzhou Flame Information Technology Co., Ltd. Research Direction: NLP, Neural Networks Personal Homepage: kexue.fm I had originally decided to stop working with RNNs as they actually correspond to numerical methods for ODEs (Ordinary Differential Equations). This realization provided me with insights into something I have always wanted to do—using deep learning … Read more

Understanding Recurrent Neural Networks (RNNs)

by
Understanding Recurrent Neural Networks (RNNs)

↑↑↑ Follow “Star Mark” Datawhale Daily Insights & Monthly Study Groups, Don’t Miss Out Datawhale Insights Focus: Neural Networks, Source: Artificial Intelligence and Algorithm Learning Neural networks are the carriers of deep learning, and among neural network models, the most classic non-RNN model belongs here. Although it is not perfect, it possesses the ability to … Read more

Understanding RNN (Recurrent Neural Networks)

by
Understanding RNN (Recurrent Neural Networks)

0. Introduction After reading many blog posts and tutorials about RNN online, I felt they were all the same, providing a vague understanding but failing to explain it clearly. RNN is the foundation of many complex models, and even in transformers, you can see the influence of RNN, so it is essential to have a … Read more

Progress in Neural Network Renormalization Group

by
Progress in Neural Network Renormalization Group

Renormalization group is a fundamental concept in physics research. It is not only a powerful tool for studying phase transitions and critical phenomena, as well as strong coupling problems, but it also shapes physicists’ worldview: physics is an effective theory about the emergence of phenomena at different scales and energy levels. In the practical applications … Read more