ALBERT

Understanding the Working Principle of GPT’s Transformer Technology

by
Understanding the Working Principle of GPT's Transformer Technology

Introduction The Transformer was proposed in the paper“Attention is All You Need”, and is now the recommended reference model for Google Cloud TPU. By introducing self-attention mechanisms and positional encoding layers, it effectively captures long-distance dependencies in input sequences and performs excellently when handling long sequences. Additionally, the parallel computing capabilities of the Transformer model … Read more

What Is the Transformer Model?

by
What Is the Transformer Model?

Welcome to the special winter vacation column “High-Tech Lessons for Kids” brought to you by Science Popularization China! Artificial intelligence, as one of the most cutting-edge technologies today, is changing our lives at an astonishing speed. From smart voice assistants to self-driving cars, from AI painting to machine learning, it opens up a future full … Read more

A Study of Policy Will Recognition Model for Public Opinion Texts Based on Multi-level Feature Fusion with BERT

by
A Study of Policy Will Recognition Model for Public Opinion Texts Based on Multi-level Feature Fusion with BERT

WENG Ke-ruiZHOU Ya-jieYU Shi-wei Abstract: Traditional policy needs research has gradually shifted to the use of social media for policy needs intelligence discovery due to cost and time factors. Although social media provides rich public policy will, capturing policy views in it is challenged by semantic ambiguity and complex comment network relationships. To address the … Read more

Why Is Your Saved BERT Model So Large?

by
Why Is Your Saved BERT Model So Large?

Follow the public account “ML_NLP” Set as “Starred”, heavy content delivered first-hand! Produced by Machine Learning Algorithms and Natural Language Processing Original Column Author on Public Account Liu Cong School | NLP Algorithm Engineer A while ago, a friend asked me this question: the ckpt file size of the bert-base model provided by Google is … Read more

Understanding BERT: Interview Questions and Insights

by
Understanding BERT: Interview Questions and Insights

Follow the WeChat public account “ML_NLP“ Set as “starred” for heavy content delivery! Author | Adherer Organizer | NewBeeNLP Interview tips knowledge compilation series, continuously updated Full of valuable content, recommended to collect, or as usual, see you in the background (code: BT) 1. What Is the Basic Principle of BERT? BERT comes from Google’s … Read more

Summary of Contrastive Learning Papers from ACL 2021

by
Summary of Contrastive Learning Papers from ACL 2021

MLNLP(Machine Learning Algorithms and Natural Language Processing) community is one of the largest natural language processing communities both domestically and internationally, gathering over 500,000 subscribers, with an audience covering NLP master’s and PhD students, university teachers, and corporate researchers. The Vision of the Community is to promote communication and progress between academia, industry, and enthusiasts … Read more

Must-See! Princeton’s Chen Danqi Latest Course on Understanding Large Language Models 2022!

by
Must-See! Princeton's Chen Danqi Latest Course on Understanding Large Language Models 2022!

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP graduate students, teachers from universities, and researchers from enterprises. The vision of the community is to promote communication and progress between the academic and industrial circles of natural language processing and machine learning, especially for the progress … Read more

Intelligent Manufacturing Model and Technology Paper Recommendations

by
Intelligent Manufacturing Model and Technology Paper Recommendations

Paper Title Intelligent Manufacturing Maturity Assessment Method Based on BERT and TextCNN Authors Zhang Gan1, Yuan Tangxiao1,2, Wang Huifen1 (Corresponding Author), Liu Linyan1 Affiliations 1. School of Mechanical Engineering, Nanjing University of Science and Technology 2. Lorrain University LCOMS Funding Supported by the High-end Foreign Experts Introduction Program of the Ministry of Science and Technology … Read more

XLNet Pre-training Model: Everything You Need to Know

by
XLNet Pre-training Model: Everything You Need to Know

Author | mantch Reprinted from WeChat Official Account | AI Technology Review 1. What is XLNet XLNet is a model similar to BERT, rather than a completely different model. In short, XLNet is a general autoregressive pre-training method. It was released by the CMU and Google Brain teams in June 2019, and ultimately, XLNet outperformed … Read more

Automatic Scoring System for Subjective Questions Based on Siamese Network and BERT Model

by
Automatic Scoring System for Subjective Questions Based on Siamese Network and BERT Model

Article Title: Automatic Scoring System for Subjective Questions Based on Siamese Network and BERT Model All Authors: Qian Shenghua First Institution: Beijing Normal University, School of Artificial Intelligence Publication Date: 2022, 31(3): 143–149 Abstract Summary Due to the current lack of automatic scoring for subjective questions in multilingual education, this paper proposes an automatic scoring … Read more