BERT has been very popular recently, so let’s gather some related resources, including papers, code, and article interpretations.
1. Official Google resources:
1) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Everything started with this paper released by Google in October, which instantly ignited the entire AI community, including social media: https://arxiv.org/abs/1810.04805
2) GitHub: https://github.com/google-research/bert
In November, Google released the code and pre-trained models, causing another wave of excitement.
3) Google AI Blog: Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing
https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html
2. Third-party interpretations:
1) Dr. Zhang Junlin’s interpretation, Zhihu column: From Word Embedding to BERT Model—The Development History of Pre-training Technology in Natural Language Processing
https://zhuanlan.zhihu.com/p/49271699
We have reprinted this article and Dr. Zhang Junlin’s shared PPT on our AINLP WeChat public account. Welcome to follow us:
-
From Word Embedding to BERT Model—The Development History of Pre-training Technology in Natural Language Processing
-
The Development of Pre-training in Natural Language Processing: From Word Embedding to BERT Model
2) Zhihu: How to Evaluate the BERT Model?
https://www.zhihu.com/question/298203515
3) [NLP] Detailed Explanation of Google BERT
https://zhuanlan.zhihu.com/p/46652512
4) [NLP Natural Language Processing] In-depth Analysis of Google’s BERT Model
https://blog.csdn.net/qq_39521554/article/details/83062188
5) BERT Explained: State of the art language model for NLP
https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270
3. Third-party code:
1) pytorch-pretrained-BERT:
https://github.com/huggingface/pytorch-pretrained-BERT Google officially recommends the PyTorch BERT version implementation, which can load Google’s pre-trained models: PyTorch version of Google AI’s BERT model with a script to load Google’s pre-trained models
2) BERT-pytorch:
https://github.com/codertimo/BERT-pytorch Another PyTorch version implementation: Google AI 2018 BERT PyTorch implementation
3) BERT-tensorflow:
https://github.com/guotong1988/BERT-tensorflow TensorFlow version: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
4) bert-chainer:
https://github.com/soskek/bert-chainer Chainer version: Chainer implementation of “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”
5) bert-as-service:
https://github.com/hanxiao/bert-as-service Encodes sentences of varying lengths using the BERT pre-trained model, mapping them to a fixed-length vector: Mapping a variable-length sentence to a fixed-length vector using a pretrained BERT model. This is very interesting; can we build a sentence similarity calculation service based on this? Can anyone try it?
6) bert_language_understanding:
https://github.com/brightmart/bert_language_understanding BERT practical application: Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN
7) sentiment_analysis_fine_grain:
https://github.com/brightmart/sentiment_analysis_fine_grain BERT practical application, multi-label text classification, an attempt in the AI Challenger 2018 fine-grained sentiment analysis task: Multi-label Classification with BERT; Fine Grained Sentiment Analysis from AI challenger
8) BERT-NER:
https://github.com/kyzhouhzau/BERT-NER BERT practical application, named entity recognition: Use Google BERT to do CoNLL-2003 NER!
Continuously updated, more related resources about BERT are welcome to be added. Welcome to follow our WeChat public account: AINLP
Click “Read the original text” to access the relevant links.