ALBERT

NLP Development Trends from Classic Models Like ULMFiT, Transformer, and BERT

by
NLP Development Trends from Classic Models Like ULMFiT, Transformer, and BERT

Natural Language Processing (NLP) is a subfield of computer science, information engineering, and artificial intelligence, focusing on human-computer language interaction and exploring how to process and utilize natural language. The research on NLP can be traced back to the Turing test, evolving from rule-based research methods to the currently popular statistical models and methods, transitioning … Read more

Breaking the BERT Ceiling: 11 Techniques to Boost NLP Classification SOTA

by
Breaking the BERT Ceiling: 11 Techniques to Boost NLP Classification SOTA

Follow the public account “ML_NLP“ Set as “Starred“, heavy content delivered first-hand! Source | Xixiaoyao’s Cute Selling House At this point in 2020, our focus on NLP classification tasks is no longer about how to construct models or being fixated on what classification models look like. Just like the current focus in the CV field, … Read more

A Comprehensive Learning Path for NLP Algorithms

by
A Comprehensive Learning Path for NLP Algorithms

As 2021 is about to end, looking back at the recent autumn recruitment for algorithm positions, it can be described as going from ashes to hell on earth. The trend has shifted: those considering a career change are starting to change careers, and those switching majors are starting to switch majors. Many people want to … Read more

Understanding BERT Model and Advanced Techniques in NLP

by
Understanding BERT Model and Advanced Techniques in NLP

The 2023 Zhejiang Programmer Festival is in full swing, and as part of the series of events, the knowledge sharing activity will successively launch the 【Artificial Intelligence Special】 knowledge sharing, including the development of AI large models, cutting-edge technologies, learning resources, etc. Stay tuned!The content of this issue is:BERT Model | Understanding Advanced Techniques and … Read more

Essential Technologies Behind Large Models

by
Essential Technologies Behind Large Models

Approximately 3500 words, recommended reading time 10 minutes. Today, we will explore the core technologies behind large models! 1. Transformer The Transformer model is undoubtedly the solid foundation of large language models, ushering in a new era in deep learning. In the early stages, Recurrent Neural Networks (RNNs) were the core means of handling sequential … Read more

In-Depth Analysis of Self-Attention from Source Code

by
In-Depth Analysis of Self-Attention from Source Code

Follow the WeChat public account “ML_NLP” Set as “Starred” to receive heavy content promptly! Reprinted from | PaperWeekly ©PaperWeekly Original · Author｜Hai Chenwei School｜Master’s student at Tongji University Research Direction｜Natural Language Processing In the current NLP field, Transformer/BERT has become a fundamental application, and Self-Attention is the core part of both. Below, we attempt to … Read more

Detailed Explanation of Masks in Attention Mechanisms

by
Detailed Explanation of Masks in Attention Mechanisms

来源：DeepHub IMBA This article is approximately 1800 words long and is recommended to be read in 5 minutes. This article will provide a detailed introduction to the principles and mechanisms of the masks in attention mechanisms. The attention mechanism mask allows us to send batches of data of varying lengths into the transformer at once. … Read more

A Simple Explanation of Transformer to BERT Models

by
A Simple Explanation of Transformer to BERT Models

In the past two years, the BERT model has become very popular. Most people know about BERT but do not understand what it specifically is. In short, the emergence of BERT has completely changed the relationship between pre-training to generate word vectors and downstream specific NLP tasks, proposing the concept of training word vectors at … Read more

Understanding Attention: Principles, Advantages, and Types

by
Understanding Attention: Principles, Advantages, and Types

Follow the public account “ML_NLP“ Set as “Starred“, heavy content delivered first time! From | Zhihu Address | https://zhuanlan.zhihu.com/p/91839581 Author | Zhao Qiang Editor | Machine Learning Algorithms and Natural Language Processing Public Account This article is for academic sharing only. If there is any infringement, please contact the backend for deletion. Attention is being … Read more

Understanding Attention Mechanisms in AI

by
Understanding Attention Mechanisms in AI

Follow the public account “ML_NLP” Set as “starred” to receive heavy content promptly! Author丨Electric Light Phantom Alchemy @ Zhihu Source丨https://zhuanlan.zhihu.com/p/362366192 Editor丨Machine Learning Algorithms and Natural Language Processing Attention has become a hot topic in the entire AI field, whether in machine vision or natural language processing, it is inseparable from Attention, transformer, or BERT. Below, … Read more