Opportunities and Challenges of MoE Large Model Training and Inference
With the development of large model technology and the proposal of the Scaling Law in 2020, it has become a consensus in the industry to improve model performance by expanding data scale and increasing model parameters. However, current large models face many engineering challenges in training, inference, and application stages. Simply increasing the model size … Read more