Pre-training

Understanding Alibaba’s Qwen Model and Local Deployment

by
Understanding Alibaba's Qwen Model and Local Deployment

Introduction Overview Pre-training Data Sources Pre-processing Tokenization Model Design Extrapolation Capability Model Training Experimental Results Deployment Testing Alignment Supervised Fine-tuning (SFT) RM Model Reinforcement Learning Alignment Results (Automatic and Human Evaluation) Automatic Evaluation Human Evaluation Deployment Testing Conclusion Introduction This article mainly introduces the Chinese large model Alibaba Qwen, specifically including model details interpretation and … Read more

Understanding Model Pre-training in Neural Networks

by
Understanding Model Pre-training in Neural Networks

This article will explain the essence of pre-training principles, and applications in three aspects, helping you understand model pre-training Pre-training. Pre-training 1.Essence of Pre-training AI = Data + Algorithms + Computing Power Three Elements of AI Dataset:Data is one of the three pillars of AI and is very important in AI technology. Datasets are generally … Read more

From Word2Vec to BERT: The Evolution of NLP Pre-trained Models

by
From Word2Vec to BERT: The Evolution of NLP Pre-trained Models

Natural Language Processing Author: Zhang Junlin Source: Deep Learning Frontier Notes Zhihu Column Original Link: https://zhuanlan.zhihu.com/p/49271699 The theme of this article is the pre-training process in natural language processing (NLP). It will roughly explain how pre-training techniques in NLP have gradually developed into the BERT model, naturally illustrating how the ideas behind BERT were formed, … Read more

In-Depth Analysis of LLAMA3 Paper

by
In-Depth Analysis of LLAMA3 Paper

Introduction Recently, while reviewing the papers I had previously studied in depth, I found that some notes were still very valuable. I made some minor adjustments and am publishing them for everyone to see. LLama3 is a paper from a few months ago, but each reading still brings new insights. This article discusses key points, … Read more