DeepSeek-V2: A Powerful MoE Language Model

DeepSeek-V2: A Powerful MoE Language Model

Abstract We propose DeepSeek-V2, a powerful Mixture of Experts (MoE) language model characterized by economical training and efficient inference. It has a total of 236 billion parameters, with 21 billion parameters activated per token, and supports 128K tokens of context length. DeepSeek-V2 adopts innovative architectures such as Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA ensures … Read more

Open Source AGI Agents: New Approaches to AGI Alignment

Open Source AGI Agents: New Approaches to AGI Alignment

New Intelligence Report Editor: Run [New Intelligence Guide] A netizen publicly shared an autonomous learning agent he created. According to his vision, such an agent will rapidly evolve into an omnipotent AGI with the support of LLM, and if humans control its growth process, specific alignment will not be necessary. A netizen created an open-source … Read more