New Intelligence Report Author: Gaode Intelligent Technology Center [New Intelligence Guide]The research and development team of Gaode Intelligent Technology Center designed a contrastive learning framework for knowledge distillation in their work, and proposed COS-NCE LOSS based on this framework. This paper has been accepted by AAAI 2021. NLP (Natural Language Processing) plays an important role … Read more
Big Data Digest authorized reprint from Data Pie Compiled by:Sun Siqi, Cheng Yu, Gan Zhe, Liu Jingjing In the past year, there have been many breakthrough advancements in the research of language models, such as GPT, which generates sentences that are convincingly realistic [1]; BERT, XLNet, RoBERTa [2,3,4], etc., have swept various NLP rankings as … Read more
Large models based on the Transformer architecture are playing an increasingly important role in artificial intelligence, especially in the fields of natural language processing (NLP) and computer vision (CV). Model compression methods reduce their memory and computational costs, which is a necessary step for implementing Transformer models on practical devices. Given the unique architecture of … Read more