AllenNLP

How Many Grades Can BERT Reach? Seq2Seq Tackles Elementary Math Problems

by
How Many Grades Can BERT Reach? Seq2Seq Tackles Elementary Math Problems

Follow our WeChat public account “ML_NLP“ Set as “Starred“, heavy content delivered to you instantly! Reprinted from｜PaperWeekly ©PaperWeekly Original · Author｜ Su Jianlin Unit｜Zhuiyi Technology Research Direction｜NLP, Neural Networks ▲ The Years of “Chicken and Rabbit in the Same Cage” “Profit and Loss Problems”, “Age Problems”, “Tree Planting Problems”, “Cows Eating Grass Problems”, “Profit Problems”… … Read more

Stabilizing BERT Fine-tuning on Small Datasets

by
Stabilizing BERT Fine-tuning on Small Datasets

Follow our public account “ML_NLP“ Set as “Starred“, heavy content delivered first! Author:Qiu Zhenyu (Algorithm Engineer, Huatai Securities Co., Ltd.) Zhihu Column:My AI Journey Recently, I came across a paper titled “Revisiting Few-sample BERT Fine-tuning”. The paper has just been released on arXiv, and although it hasn’t attracted much attention yet, I found it very … Read more

ALBERT: A Lightweight BERT for Self-Supervised Learning in Language Representation

by
ALBERT: A Lightweight BERT for Self-Supervised Learning in Language Representation

Follow us on WeChat “ML_NLP” Set as “Starred”, heavy content delivered first hand! Written by / Radu Soricut and Zhenzhong Lan, Researchers, Google Research Since the advent of BERT a year ago, natural language research has adopted a new paradigm: leveraging a large amount of existing text to pre-train model parameters in a self-supervised manner … Read more

Have You Read the Bert Source Code?

by
Have You Read the Bert Source Code?

Click the “MLNLP” above and select “Star” to follow the public account Heavyweight content delivered to you first Author:Old Song’s Tea Book Club Zhihu Column:NLP and Deep Learning Research Direction:Natural Language Processing Introduction A few days ago, during an interview, an interviewer directly asked me to analyze the source code of BERT. Emm, that was … Read more

Understanding WordPiece in BERT

by
Understanding WordPiece in BERT

Follow the public account “ML_NLP“ Set as “Starred“, heavy content delivered to you first! From | cnblogs Address | https://www.cnblogs.com/huangyc/p/10223075.html Author | hyc339408769 Editor | Machine Learning Algorithms and Natural Language Processing Public Account This article is for academic sharing only. If there is an infringement, please contact us to delete the article. Complete machine … Read more

Roberta: Fine-Tuning Bert

by
Roberta: Fine-Tuning Bert

Follow the WeChat public account “ML_NLP“ Set as “Starred“, heavy content delivered first hand! Recently, I need to start using Transformers for some tasks, so I specifically recorded related knowledge points to build a relevant and complete knowledge structure system, The following are the articles to be written, this article is the fourth in this … Read more

A Detailed Explanation from Transformer to BERT Model

by
A Detailed Explanation from Transformer to BERT Model

Follow the WeChat public account “ML_NLP“ Set as “Starred“, heavy content delivered first-hand! Table of Contents: A Brief Review of ELMo and Transformer DAE and Masked Language Model Detailed Explanation of BERT Model Different Training Methods of BERT Model How to Apply BERT Model in Real Projects How to Slim Down BERT Problems with BERT … Read more

Master Bert Source Code in 10 Minutes (PyTorch Version)

by
Master Bert Source Code in 10 Minutes (PyTorch Version)

The application of Bert in production environments requires compression, which demands a deep understanding of the Bert structure. This repository will interpret the Bert source code (PyTorch version) step by step. The repository can be found at https://github.com/DA-southampton/NLP_ability Code and Data Introduction First, for the code, I referenced this repository. I directly cloned the code … Read more

Post-BERT: Pre-trained Language Models and Natural Language Generation

by
Post-BERT: Pre-trained Language Models and Natural Language Generation

Wishing You a Prosperous Year of the Rat HAPPY 2020’S NEW YEAR Author:Tea Book Club of Lao Song Zhihu Column:NLP and Deep Learning Research Direction:Natural Language Processing Source:AINLP Introduction BERT has achieved great success in the field of natural language understanding, but it performs poorly in natural language generation due to the language model used … Read more

From BERT to ChatGPT: A Comprehensive Review of Pretrained Foundation Models

by
From BERT to ChatGPT: A Comprehensive Review of Pretrained Foundation Models

MLNLP community is a well-known machine learning and natural language processing community at home and abroad, covering NLP master’s and doctoral students, university teachers, and corporate researchers. The vision of the community is to promote communication and progress between the academic and industrial communities of natural language processing and machine learning, especially for the progress … Read more