Reviewing Progress and Insights on BERT Models

Reviewing Progress and Insights on BERT Models

Authorized Reprint from Microsoft Research AI Headlines Since BERT was published on arXiv, it has gained significant success and attention, opening the Pandora’s box of 2-Stage in NLP. Subsequently, a large number of pre-trained models similar to “BERT” have emerged, including the generalized autoregressive model XLNet that introduces bidirectional context information from BERT, as well … Read more

Choosing Between BERT, RoBERTa, DistilBERT, and XLNet

Choosing Between BERT, RoBERTa, DistilBERT, and XLNet

Planning | Liu Yan Author | Suleiman Khan Translation | Nuclear Cola Editor | Linda AI Frontline Overview: Google BERT and other transformer-based models have recently swept the entire NLP field, significantly surpassing previous state-of-the-art solutions in various tasks. Recently, Google has made several improvements to BERT, leading to a series of impressive enhancements. In … Read more

Mission to TensorFlow World: Space Flight Tasks

Mission to TensorFlow World: Space Flight Tasks

Whether you are on your way to TensorFlow World or unable to attend in person, read below for the latest information on the demonstrations and experience the new GitHub codebase.You can also follow #MissionToTensorFlowWorld on Twitter for real-time experiences! Half a century ago, the Apollo 11 mission realized humanity’s dream of landing on the moon. … Read more

BERT: Training Longer and with More Data to Return to SOTA

BERT: Training Longer and with More Data to Return to SOTA

Machine Heart Report Contributors: Si Yuan, Qian Zhang The championship throne of XLNet has not yet warmed up, and the plot has once again taken a turn. Last month, XLNet comprehensively surpassed BERT on 20 tasks, creating a new record for NLP pre-training models and enjoyed a moment of glory. However, now, just a month … Read more