The author is from Meta, an internet practitioner, focusing on LLM4Code and LLMinfra. The original text is from Zhihu, link: https://zhuanlan.zhihu.com/p/16270225772 This article is for academic/technical sharing only. If there is any infringement, please contact for removal. RLHF is an important part of LLM training. With the development of open-source models, we observe that some … Read more
01 Introduction What is RAG LLMs can produce misleading “hallucinations”, depend on information that may be outdated, and are inefficient when handling specific knowledge, lacking deep insights in specialized fields, while also having some deficiencies in reasoning capabilities. It is against this backdrop that Retrieval-Augmented Generation (RAG) technology has emerged, becoming a significant trend in … Read more
Introduction Official Documentation: Qwen1.5-MoE: Achieving the Performance of 7B Models with 1/3 Activation Parameters | Qwen On March 28, Alibaba announced the open-source MoE technology large model Qwen1.5-MoE-A2.7B for the first time. This model is based on the existing Qwen-1.8B model. The activation parameters of Qwen1.5-MoE-A2.7B are 270 million, but it can achieve the performance … Read more
01 Introduction On August 3, Alibaba Cloud released its first open-source large model: Tongyi Qwen-7B, which is open-source and commercially usable. Although everyone has been raised expectations with various hundred-billion parameter models, the fact that it is produced by Alibaba has attracted widespread attention and discussion among peers, and it has performed excellently on various … Read more