Load Balancing

Key Details of Qwen MoE: Enhancing Model Performance Through Global Load Balancing

by
Key Details of Qwen MoE: Enhancing Model Performance Through Global Load Balancing

Today, we share with you the latest paper from Alibaba Cloud Tongyi Qianwen team – Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models (Original paper link: https://arxiv.org/abs/2501.11873) This paper focuses on improving the training method of Mixture-of-Experts (MoEs) by relaxing local balance to global balance through lightweight communication, significantly … Read more

Qwen’s Year-End Gift: Enhancing MoE Training Efficiency

by
Qwen's Year-End Gift: Enhancing MoE Training Efficiency

Click the top to follow me Before reading this article, we sincerely invite you to click the “Follow” button, so that we can conveniently push similar articles to you in the future, and also facilitate your discussions and sharing. Your support is our motivation to keep creating～ Today, we will learn about a powerful technology … Read more

In-Depth Learning of OpenShift Series 1/7: Router and Route

by
In-Depth Learning of OpenShift Series 1/7: Router and Route

1. Why Does OpenShift Need Router and Route? As the name suggests, the Router is a routing device, and Route refers to the paths configured within the router. These two concepts in OpenShift are designed to address the need to access services from outside the cluster (that is, from places other than the cluster nodes). … Read more

Differences Between Kubernetes Ingress and OpenShift Router

by
Differences Between Kubernetes Ingress and OpenShift Router

Objective: Discuss the differences between Kubernetes Ingress and OpenShift Router Prerequisite: Understanding of Kubernetes and OpenShift Background: Kubernetes Ingress and OpenShift Route can expose services (Service) through routing, facilitating external access to internal cluster resources while also providing load balancing. Kubernetes Ingress Overview: Kubernetes Ingress is a Kubernetes resource used to manage and configure how … Read more

Understanding Key Technology DeepSeekMoE in DeepSeek-V3

by
Understanding Key Technology DeepSeekMoE in DeepSeek-V3

1. What is Mixture of Experts (MoE)? In the field of deep learning, the improvement of model performance often relies on scaling up, but the demand for computational resources increases sharply. Maximizing model performance within a limited computational budget has become an important research direction. The Mixture of Experts (MoE) introduces sparse computation and dynamic … Read more