Finally, Kimi has been updated! I’ve been looking forward to this. It is said to be in grayscale: but my interface still looks like this. Let’s wait a bit and try later~ Let’s read the paper together and see what technical details have changed. Address: https://github.com/MoonshotAI/Kimi-k1.5/blob/main/Kimi_k1.5.pdf The pre-training methods of large language models (LLMs) have … Read more
1. Title: KIMI K1.5: SCALING REINFORCEMENT LEARNING WITH LLMS Link: https://github.com/MoonshotAI/kimi-k1.5 2. Authors and Key Points: 1- Authors The paper was published by: Kimi Team of the Dark Side of the Moon 2- Key Points 1. Core Content • Background and Motivation: • Traditional language model pre-training methods (based on next-word prediction) perform well in … Read more
Multimodal learning and attention mechanisms are currently hot topics in deep learning research, and cross-attention fusion serves as a convergence point for these two fields, offering significant development space and innovation opportunities. As a crucial component of multimodal fusion, cross-attention fusion establishes connections between different modules through attention mechanisms, facilitating the exchange and integration of … Read more
Large Language Models (LLMs) have made significant progress in the field of generative artificial intelligence, but they face the “hallucination” problem, which is the tendency to generate inaccurate or irrelevant information. This issue is particularly severe in high-risk applications such as medical assessments and insurance claims processing. To address this challenge, researchers from the University … Read more
Recently, it feels like the New Year has come early. Just last night, DeepSeek and Kimi both released their version 1.0, and Kimi was the first to publish its technical report, which is quite interesting… When it comes to Kimi, everyone has the impression that it has a technological first-mover advantage, being the first to … Read more
Following the release of large models for language, code, mathematics, etc., DeepSeek has brought another early achievement on the journey towards AGI… DeepSeekVL, jointly expanding training data, model architecture, and training strategies, attempts to build the strongest open-source 7B and 1.3B multimodal models. Highlights Data: Multi-source multimodal data enhances the model’s general cross-modal capabilities, mixing … Read more