Spatial Attention

Attention mechanisms have become the foundational architecture for model design; nowadays, it’s almost embarrassing to release a model without any Attention. Since the release of the attention mechanism, the academic community has been continuously modifying Attention in various innovative ways. The modified Attention can enhance the model’s expressive capability, improve cross-modal abilities and interpretability, as … Read more