Follow the public account to discover the beauty of CV technology This article is reprinted from Machine Heart, edited by Panda W. Transformers are powerful but not perfect, especially when dealing with long sequences. State Space Models (SSMs) perform quite well on long sequences. Researchers proposed last year that SSMs could replace Transformers, as seen … Read more
Source: Algorithm Advancement This article is about 4000 words long and is recommended for an 8-minute read. This article deeply explores the potential connections between Transformer, Recurrent Neural Networks (RNN), and State Space Models (SSM). By exploring the potential connections between seemingly unrelated Large Language Model (LLM) architectures, we may open up new avenues for … Read more
The Transformer architecture has become a major component of the success of large language models (LLMs). To further improve LLMs, new architectures that may outperform the Transformer architecture are being developed. One such approach is Mamba, a state space model. The paper “Mamba: Linear-Time Sequence Modeling with Selective State Spaces” introduces Mamba, which we have … Read more
Big Data Digest Authorized Reprint from Leading Technology Author丨Congerry Transformer Challenged! In June 2017, eight Google researchers published a groundbreaking paper titled “Attention is All You Need”. It is called groundbreaking because this paper proposed a new neural network architecture – the Transformer, which opened a new era of generative artificial intelligence and large models. … Read more