Jishi Guide
This article discusses the Attention mechanism in deep learning. It is not intended to review the various frameworks and applications of the Attention mechanism, but rather to introduce four representative and interesting works related to Attention and provide further thoughts on the rationality of the Attention mechanism. >> Tomorrow’s live broadcast! Tian Zhi: Innovative breakthrough in instance segmentation BoxInst, achieving COCO 33.2AP with only Box annotations!
Introduction: This article is about the Attention mechanism in deep learning. Unlike previous series of articles, this one does not focus on reviewing various frameworks and applications of the Attention mechanism, and therefore will not spend too much time introducing the historical development or mathematical computation methods of Attention; instead, it simply introduces several representative and interesting works related to Attention and reflects on the rationality of the Attention mechanism, proposing some further thoughts.
The main framework of this article is as follows:
-
Source of the Problem & Background -
The