Reviewing Progress and Insights on BERT Models

Reviewing Progress and Insights on BERT Models

Authorized Reprint from Microsoft Research AI Headlines Since BERT was published on arXiv, it has gained significant success and attention, opening the Pandora’s box of 2-Stage in NLP. Subsequently, a large number of pre-trained models similar to “BERT” have emerged, including the generalized autoregressive model XLNet that introduces bidirectional context information from BERT, as well … Read more

Choosing Between BERT, RoBERTa, DistilBERT, and XLNet

Choosing Between BERT, RoBERTa, DistilBERT, and XLNet

Planning | Liu Yan Author | Suleiman Khan Translation | Nuclear Cola Editor | Linda AI Frontline Overview: Google BERT and other transformer-based models have recently swept the entire NLP field, significantly surpassing previous state-of-the-art solutions in various tasks. Recently, Google has made several improvements to BERT, leading to a series of impressive enhancements. In … Read more

Thoughts on Building Target Maps

Thoughts on Building Target Maps

On the Chaoxing and Zhihuishu course platforms, there are application sections based on the OBE concept for knowledge graphs. Chaoxing refers to it as the target map, while Zhihuishu calls it OBE teaching management. Both are forms of application that measure, evaluate, and provide feedback on the achievement of course objectives under the guidance of … Read more

BERT: Training Longer and with More Data to Return to SOTA

BERT: Training Longer and with More Data to Return to SOTA

Machine Heart Report Contributors: Si Yuan, Qian Zhang The championship throne of XLNet has not yet warmed up, and the plot has once again taken a turn. Last month, XLNet comprehensively surpassed BERT on 20 tasks, creating a new record for NLP pre-training models and enjoyed a moment of glory. However, now, just a month … Read more