Efficient Transformers

Transformer Advances Towards Dynamic Routing: TRAR for VQA and REC SOTA

by
Transformer Advances Towards Dynamic Routing: TRAR for VQA and REC SOTA

Follow our public account to discover the beauty of CV technology 1 Introduction Due to its superior capability for modeling global dependencies, the Transformer and its variants have become the primary architecture for many visual and language tasks. However, tasks like Visual Question Answering (VQA) and Referencing Expression Comprehension (REC) often require multi-modal predictions that … Read more

Overview of 17 Efficient Variants of Transformer Models

by
Overview of 17 Efficient Variants of Transformer Models

Source: Huang Yu Zhihu This article is about 3600 words long, and it is recommended to read it in 10 minutes. This article introduces the review paper "Efficient Transformers: A Survey" published by Google in September last year, which states that in the field of NLP, transformers have successfully replaced RNNs (LSTM/GRU), and applications have … Read more