Bi-RNN

Comparative Study of Transformer and RNN in Speech Applications

by
Comparative Study of Transformer and RNN in Speech Applications

Original link: https://arxiv.org/pdf/1909.06317.pdf Abstract Sequence-to-sequence models are widely used in end-to-end speech processing, such as Automatic Speech Recognition (ASR), Speech Translation (ST), and Text-to-Speech (TTS). This paper focuses on a novel sequence-to-sequence model called the Transformer, which has achieved state-of-the-art performance in neural machine translation and other natural language processing applications. We conducted an in-depth … Read more

Who Will Replace Transformer?

by
Who Will Replace Transformer?

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering graduate students, faculty, and researchers in NLP. The community’s vision is to promote communication and progress between the academic and industrial sectors of natural language processing and machine learning, especially for beginners. Reprinted from | AI Technology Review … Read more

In-Depth Analysis of the Connections Between Transformer, RNN, and Mamba!

by
In-Depth Analysis of the Connections Between Transformer, RNN, and Mamba!

Source: Algorithm Advancement This article is about 4000 words long and is recommended for an 8-minute read. This article deeply explores the potential connections between Transformer, Recurrent Neural Networks (RNN), and State Space Models (SSM). By exploring the potential connections between seemingly unrelated Large Language Model (LLM) architectures, we may open up new avenues for … Read more

Deep Learning Hyperparameter Tuning Experience

by
Deep Learning Hyperparameter Tuning Experience

From | DataWhale Training techniques are very important for deep learning. As a highly experimental science, even the same network architecture trained with different methods can yield significantly different results. Here, I summarize my experiences from the past year and share them with everyone. I also welcome additions and corrections. Parameter Initialization Any of the … Read more

Deep Learning Hyperparameter Tuning Experience

by
Deep Learning Hyperparameter Tuning Experience

Click on the “Datawhalee” above to select the “Starred“ public account Get valuable content at the first time Training techniques are very important for deep learning. As a highly experimental science, using different training methods on the same network structure can yield significantly different results. Here, I summarize my experiences from the past year and … Read more

Illustration of 3 Common Deep Learning Network Structures: FC, CNN, RNN

by
Illustration of 3 Common Deep Learning Network Structures: FC, CNN, RNN

Introduction: Deep learning can be applied in various fields, and the shapes of deep neural networks vary according to different application scenarios. The common deep learning models mainly include Fully Connected (FC), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN). Each of these has its own characteristics and plays an important role in different … Read more

Stanford Deep Learning Course Part 7: RNN, GRU, and LSTM

by
Stanford Deep Learning Course Part 7: RNN, GRU, and LSTM

This article is a translated version of the notes from Stanford University’s CS224d course, authorized by Professor Richard Socher of Stanford University. Unauthorized reproduction is prohibited; for specific reproduction requirements, please see the end of the article. Translation: Hu Yang & Xu Ke Proofreading: Han Xiaoyang & Long Xincheng Editor’s Note: This article is the … Read more

Stanford Chinese Professor: Sound Waves, Light Waves, All Are RNNs!

by
Stanford Chinese Professor: Sound Waves, Light Waves, All Are RNNs!

New Intelligence Report Source: Reddit, Science Editors: Daming, Pengfei [New Intelligence Guide]Recently, Stanford University Chinese Professor Shanhui Fan’s team published an article in a sub-journal of Science, pointing out that whether it is sound waves, light waves, or other forms of waves, their descriptive equations can be equivalent to Recurrent Neural Networks (RNNs). This discovery … Read more

A Simple Guide to Recurrent Neural Networks (RNN)

by
A Simple Guide to Recurrent Neural Networks (RNN)

Source: Panchuang AI, Author: VK Panchuang AI Share Author | Renu Khandelwal Compiler | VK Source | Medium We start with the following questions: Recurrent Neural Networks can solve the problems present in Artificial Neural Networks and Convolutional Neural Networks. Where can RNNs be used? What is RNN and how does it work? Challenges of … Read more

It’s Time to Abandon RNN and LSTM for Sequence Modeling

by
It's Time to Abandon RNN and LSTM for Sequence Modeling

Selected from Medium Author: Eugenio Culurciello Translation by Machine Heart Contributors: Liu Xiaokun, Siyuan The author states: We have been trapped in the pit of RNNs, LSTMs, and their variants for many years; it is time to abandon them! In 2014, RNNs and LSTMs were revived. We all read Colah’s blog “Understanding LSTM Networks” and … Read more