Author: Chen Zhi Yan This article is approximately 3500 words long and is recommended for a 7-minute read. The Transformer is the first model that completely relies on the self-attention mechanism to compute its input and output representations. The mainstream sequence-to-sequence models are based on encoder-decoder recurrent or convolutional neural networks. The introduction of the … Read more
In the previous lecture, we discussed the seq2seq model. Although the seq2seq model is powerful, its effectiveness can be significantly reduced if used in isolation. This section introduces the attention model, which simulates the human attention intuition within the encoder-decoder framework. Principle of Attention Mechanism The attention mechanism in the human brain is essentially a … Read more
In the previous article, we learned about the basic seq2seq model, which processes the input sequence through an encoder, passes the calculated hidden state to a decoder, and then decodes it to obtain the output sequence. The block diagram is shown again below: The basic seq2seq model is quite effective for short and medium-length sentences … Read more
The author of this article – Teacher Tom ▷ Doctorate from a double first-class domestic university, national key laboratory ▷ Published 12 papers at top international conferences, obtained 2 national invention patents, served as a reviewer for multiple international journals ▷ Guided more than ten doctoral and master’s students Research Areas: General visual-language cross-modal model … Read more
Selected from TowardsDataScience Author: Raimi Karim Contributors: Gao Xuan, Lu This article visually explains the attention mechanism with several animated diagrams and shares four NMT architectures that have emerged in the past five years, along with intuitive explanations of some concepts mentioned in the text. For decades, statistical machine translation has dominated translation models [9], … Read more
Selected from arXiv Author: Graham Neubig Translation by Machine Heart Contributors: Li Zenan, Jiang Siyuan This article is a detailed tutorial on machine translation, suitable for readers with a background in computer science. According to Paper Weekly (ID: paperweekly), this paper comes from CMU LTI and covers various foundational knowledge of the Seq2Seq method, including … Read more
Selected from arXiv Authors: Guillaume Lample et al. Translation by Machine Heart Contributors: Zhang Qian, Lu Recently, researchers from FAIR proposed two variants of machine translation models, one being a neural model and the other based on phrases. The researchers combined two recently proposed unsupervised methods, simplifying the structure and loss functions, resulting in a … Read more
Produced by Big Data Digest Author: Liu Junhuan As a very important application in natural language processing, the modern concept of machine translation has evolved since it was proposed in the 1940s, undergoing several generations of innovation, and has now begun to be implemented in various scenarios. In recent years, with the improvement in machine … Read more
Machine Heart (Overseas) Original Author: Mos Zhang Participated by: Panda Machine Translation (MT) is the process of “automatically translating text from one natural language (source language) to another (target language)” using machines [1]. The idea of using machines for translation was first proposed by Warren Weaver in 1949. For a long time (from the 1950s … Read more
Selected from Google Research Authors: Quoc V. Le, Mike Schuster Translated by: Machine Heart Contributors: Wu Pan 2016 was a year of continuous breakthroughs in artificial intelligence. This year, we experienced breakthroughs in speech recognition, the flourishing of style transfer, advancements in neural machine translation, and more. Machine Heart closely followed each announcement. As the … Read more