Edited by Machine Heart Contributors: Siyuan, Wang Shuting This month, Google’s BERT has received a lot of attention, as the research has refreshed the state-of-the-art performance records in 11 NLP tasks with its pre-trained model. The authors of the paper stated that they would release the code and pre-trained model by the end of this … Read more
Follow the official account “ML_NLP“ Set as “Starred“, delivering heavy content promptly! Source | Zhihu Link | https://zhuanlan.zhihu.com/p/132361501 Author | Alan Lee Editor | Machine Learning Algorithms and Natural Language Processing Public Account This article is authorized and reposting is prohibited This article was first published on my personal blog on 2019/10/16 and cannot be … Read more
Author Jay Alammar, Translated by QbitAI | WeChat Official Account QbitAI BERT, as a key player in the field of natural language processing, is an unavoidable topic for NLPer. However, for those with little experience and a weak foundation, mastering BERT can be a bit challenging. Now, tech blogger Jay Alammar has created a “Visual … Read more
New Intelligence Report Source: Google AI Editor: LRS [New Intelligence Guide] Based on pre-training experience, more data leads to better performance! Google recently published a paper at NAACL 2021 that can automatically generate text data from knowledge graphs, so there’s no need to worry about insufficient corpora anymore! Large pre-trained natural language processing (NLP) models, … Read more
Recommended by New Intelligence Source: Zhuanzhi (ID: Quan_Zhuanzhi) Editor: Sanshi [New Intelligence Guide] In the field of NLP, various advanced tools have emerged recently. However, practice is the key, and how to apply them to your own models is crucial. This article introduces this issue. Recently in NLP, various pre-trained language models like ELMO, GPT, … Read more
©PaperWeekly Original · Author｜Su Jianlin School｜Zhuiyi Technology Research Direction｜NLP, Neural Networks Recently, I learned about a BERT model compression method called “BERT-of-Theseus”, derived from the paper BERT-of-Theseus: Compressing BERT by Progressive Module Replacing. This is a model compression scheme built on the concept of “replaceability”. Compared to conventional methods like pruning and distillation, it appears … Read more
Author: Gao Kaiyuan School: Shanghai Jiao Tong University Research Direction: Natural Language Processing Zhihu Column: BERT on the Shoulders of Giants Original Article Link: https://zhuanlan.zhihu.com/p/91052495 Introduction In the previous blog, I discussed some knowledge representation learning models. Today, let’s explore the current most popular BERT model and how it develops with the addition of external … Read more
In October 2018, the BERT model launched by Google made a stunning impact, sweeping various rankings and even surpassing human baseline scores, achieving a milestone breakthrough in the field of NLP. Today, for NLP algorithm engineers, BERT has become an essential tool. “What if there’s too little data?” — “Just fine-tune BERT!” “What if RNN … Read more
© Author | Wang Kunze Affiliation | The University of Sydney Research Direction | NLP The comparison between Batch Norm and Layer Norm has become a cliché in the field of algorithms. The question of why BERT uses layer norm instead of batch norm has been asked to death, and a casual search on Zhihu … Read more
Machine Heart Release Author: Tony, Researcher at Zhuiyi Technology AI Lab Everyone knows that language models like BERT have been widely used in natural language processing. However, a question sometimes arises: do these language models truly understand language? Experts and scholars have different opinions on this. The author of this article elaborates on this topic … Read more