Innovative Attention Mechanism Proposed by UESTC Improves MobileViT’s Attention QKV Operations
In this study, the authors propose an improved variant of MobileViT that performs attention-based QKV operations in the early stages of downsampling. Performing QKV operations directly on high-resolution feature maps is computationally intensive due to their large size and numerous tokens. To address this issue, the authors introduce a filtering attention mechanism that uses convolutional … Read more