Follow the public account “ML_NLP“ Set as “Starred“, heavy content delivered to you first! Source | Zhihu Address | https://zhuanlan.zhihu.com/p/69290203 Author | Ph0en1x Editor | WeChat public account on Machine Learning Algorithms and Natural Language Processing This article is for academic sharing only. If there is any infringement, please contact us to delete it. This … Read more
This article will cover theessence of Transformer, the principles of Transformer, and improvements in Transformer architecture in three aspects to help you understand Transformer. 1. Essence of Transformer Transformer Architecture: It mainly consists of four parts: input section (input-output embeddings and position encoding), multi-layer encoder, multi-layer decoder, and output section (output linear layer and Softmax). … Read more
Edited by Machine Heart Contributors: Siyuan, Wang Shuting This month, Google’s BERT has received a lot of attention, as the research has refreshed the state-of-the-art performance records in 11 NLP tasks with its pre-trained model. The authors of the paper stated that they would release the code and pre-trained model by the end of this … Read more
Follow the official account “ML_NLP“ Set as “Starred“, delivering heavy content promptly! Source | Zhihu Link | https://zhuanlan.zhihu.com/p/132361501 Author | Alan Lee Editor | Machine Learning Algorithms and Natural Language Processing Public Account This article is authorized and reposting is prohibited This article was first published on my personal blog on 2019/10/16 and cannot be … Read more
Author Jay Alammar, Translated by QbitAI | WeChat Official Account QbitAI BERT, as a key player in the field of natural language processing, is an unavoidable topic for NLPer. However, for those with little experience and a weak foundation, mastering BERT can be a bit challenging. Now, tech blogger Jay Alammar has created a “Visual … Read more
New Intelligence Report Source: Google AI Editor: LRS [New Intelligence Guide] Based on pre-training experience, more data leads to better performance! Google recently published a paper at NAACL 2021 that can automatically generate text data from knowledge graphs, so there’s no need to worry about insufficient corpora anymore! Large pre-trained natural language processing (NLP) models, … Read more
Recommended by New Intelligence Source: Zhuanzhi (ID: Quan_Zhuanzhi) Editor: Sanshi [New Intelligence Guide] In the field of NLP, various advanced tools have emerged recently. However, practice is the key, and how to apply them to your own models is crucial. This article introduces this issue. Recently in NLP, various pre-trained language models like ELMO, GPT, … Read more
©PaperWeekly Original · Author｜Su Jianlin School｜Zhuiyi Technology Research Direction｜NLP, Neural Networks Recently, I learned about a BERT model compression method called “BERT-of-Theseus”, derived from the paper BERT-of-Theseus: Compressing BERT by Progressive Module Replacing. This is a model compression scheme built on the concept of “replaceability”. Compared to conventional methods like pruning and distillation, it appears … Read more
Author: Gao Kaiyuan School: Shanghai Jiao Tong University Research Direction: Natural Language Processing Zhihu Column: BERT on the Shoulders of Giants Original Article Link: https://zhuanlan.zhihu.com/p/91052495 Introduction In the previous blog, I discussed some knowledge representation learning models. Today, let’s explore the current most popular BERT model and how it develops with the addition of external … Read more
In October 2018, the BERT model launched by Google made a stunning impact, sweeping various rankings and even surpassing human baseline scores, achieving a milestone breakthrough in the field of NLP. Today, for NLP algorithm engineers, BERT has become an essential tool. “What if there’s too little data?” — “Just fine-tune BERT!” “What if RNN … Read more