Language Modeling

The Arrival of ImageNet Era in NLP: Word Embeddings Are Dead

by
The Arrival of ImageNet Era in NLP: Word Embeddings Are Dead

Selected fromthe Gradient Author：Sebastian Ruder Translated by Machine Heart In the field of computer vision, models pre-trained on ImageNet are commonly used for various CV tasks such as object detection and semantic segmentation. In contrast, in the field of natural language processing (NLP), we typically only use pre-trained word embedding vectors to encode the relationships … Read more

The Origins of Natural Language Processing: Markov and Shannon’s Language Modeling Experiments

by
The Origins of Natural Language Processing: Markov and Shannon's Language Modeling Experiments

Excerpt from towardsdatascience Author: Raimi Karim Translated by: Machine Heart Contributors: Wang Zijia, Geek AI Language modeling and text generation are currently two hot research topics in the field of natural language processing. Yet, over a century ago, scientific giants Markov and Shannon began their preliminary explorations in this area… In 1913, Russian mathematician Andrey … Read more