Transformers, as an attention-based encoder-decoder architecture, have not only revolutionized the field of Natural Language Processing (NLP) but have also made groundbreaking contributions to the field of Computer Vision (CV). Compared to Convolutional Neural Networks (CNNs), Vision Transformers (ViT) rely on excellent modeling capabilities, achieving outstanding performance on several benchmarks including ImageNet, COCO, and ADE20k.
As Atlas Wang, a computer scientist at the University of Texas at Austin, said:We have every reason to try using Transformers across the entire range of AI tasks.
Therefore, whether researchers in academia or professionals in the industry, it is essential to have a deep understanding of Transformer technology and to keep up with the cutting-edge research on Transformers to solidify their technical foundation.
AI is a field that is easy to get into, but hard to master, which is also why there has always been a significant shortage of high-end AI talent.
In your work:
Are you able to flexibly propose new models according to actual scenarios?
Or propose modifications to existing models?
In fact, these are core competencies and also thresholds that one must pass to become high-end talent. Although it is challenging, once you pass this threshold, you will find yourself in the TOP 5% of the market.
Therefore, we have designed this course with one goal: to give you the opportunity to become part of the TOP 5% in the market. In this course, we will explain the principles, implementation methods, and application techniques of Transformers in the CV field in a step-by-step manner. During the learning process, you can expand your thinking through practical projects, integrating knowledge to genuinely enhance your problem-solving abilities.
-
Comprehensive content explanation: covering the hottest Transformers in today’s applications and research fields, including 10+ Transformer models and application cases.
-
In-depth technical analysis: detailed analysis of Transformer and framework technical details and cutting-edge model principles covered in each module.
-
Corporate practical projects: including image recognition and object detection, enhancing students’ theoretical and practical skills in applications.
-
Expert-level instructor team: each module is taught by scientists or researchers with years of frontline experience in their respective fields, accompanied by well-qualified and experienced teaching assistants, dedicated to providing the best learning experience.
▶ A comprehensive grasp of Transformer knowledge, flexibly applied in your work
▶ An understanding of how Transformer model frameworks are implemented, and proficiency in their key technologies and methods
▶ A deep understanding of cutting-edge Transformer technologies, broadening your technical vision in work and research
▶ A comprehensive and systematic understanding of a field in a short period, greatly saving learning time
▶ Connections with a group of like-minded individuals for mutual exchange and learning
Helping you become an industryTOP 10% engineer
Students interested in the course
Scan the QR code for consultation
Below is a detailed introduction to the CV section; interested friends can consult for more.
-
Comprehensive technical knowledge explanation
The course content covers explanations of over 10 models including Bert, ViT, SegFormer, DETR, UP-DETR, TimeSformer, DeiT, Mobile-Transformer, Efficient Transformer, SwinTransformer, Point Transformer, MTTR, MMT, Uniformer, etc.
Project practice to apply learning
Students use Transformer models to practice the most widely used tasks in the CV field, such as image recognition and object detection.
-
Course content rigorously refined by a professional team, cutting-edge and in-depth
The course content has undergone hundreds of hours of design refinement to ensure the content and project node settings are reasonable, truly achieving effective learning.
-
Employment-oriented, clear goals
Upon successful completion of the course, outstanding students can receive referral interview opportunities with major internet companies such as ByteDance, Alibaba, Tencent, Meituan, as well as AI unicorns like SenseTime and Megvii.
Theme: Review and Explanation of Transformer/Bert Knowledge in NLP
This lesson will guide everyone in reviewing the Transformer/Bert technologies in the field of NLP. This will deepen the understanding of the technical details and algorithm advantages of Transformer/Bert, facilitating further learning of the application of Transformer technology in other fields.
-
Self-Attention mechanism and parallelization principles in Transformer in NLP.
-
Advanced principles of Bert in Transformer.
Theme: Application of Transformers in Image Classification and Semantic Segmentation: Exploring ViT and SegFormer Technologies
Based on the content of the first lesson, further study how to transfer Transformer ideas to applications in two classification problems in computer vision: image classification and semantic segmentation. Using two classic structures, ViT and SegFormer, to help students experience how to apply Transformers to the visual domain.
-
How to apply the design ideas of Transformers to image classification and semantic segmentation problems.
Theme: Application of Transformers in Object Detection: Exploring DETR and UP-DETR Technologies
This lesson will further study how to apply Transformer technology to object detection tasks. Particularly, how to design Transformer network structures that allow neural networks to learn both object category information and location information simultaneously.
-
In-depth understanding of the design ideas for applying Transformers to object detection.
-
-
Theme: Application of Transformers in Video Understanding: Exploring TimeSformer Technologies
This lesson will further study how to apply Transformer technology to video understanding applications, enabling Transformers to learn correlations in both temporal and spatial dimensions. Using TimeSformer as an example, students can deeply appreciate the design ideas involved.
-
Considerations for extending Transformer design ideas to modeling temporal-spatial correlations.
Theme: Discussion on Efficient Transformer Design: Exploring DeiT and Mobile-Transformer Technologies
Efficient Transformers have always been a goal that researchers strive for. This course will discuss how to design efficient Transformer network structures. This lesson will use DeiT and Mobile-Transformer as examples to learn about considerations in the efficient design process.
-
Considerations in the design of Efficient Transformers, and discussions on optimizing Transformer perspectives.
Theme: Learning Classic Transformer Network Structures: Learning the SwinTransformer Model Family
This course will use the SwinTransformer model as an example to systematically learn about SwinTransformer and its variant models. The goal is to help students further understand the considerations needed for applying Transformers to visual tasks, including clever ideas and how to achieve parallel computation through reasonable design.
-
SwinTransformer model family
-
SwinTransformer design ideas. Considerations for designing Transformers to solve new problems.
Theme: Transformers in Point Cloud
This lesson will share the application of Transformers in 3D Point Clouds. Based on the characteristics of 3D Point Cloud data, we will explore how to design suitable Transformer networks to handle massive, unstructured point cloud data. Additionally, we will discuss how to further modify the Transformer structure for tasks such as segmentation and clustering.
-
Considerations when designing Transformers to handle point cloud data.
-
Theme: Transformer Design in Multi-Modal Applications
This lesson will cover the design issues of Transformers in multi-modal contexts. Transformers have been well applied in various fields. Recent works have explored how to design suitable Transformer structures for processing multi-modal data. We will provide explanations using MTTR, MMT, Uniformer, and other related Transformers as examples.
-
Design considerations for Transformers handling multi-modal data.
-
How to design suitable Transformers for multi-modal problems: MTTR, MMT, Uniformer.
Project 1: Image Recognition System Based on ViT Model
Project Description: As a classic application case of Transformers in the visual domain, the ViT model was the first to apply the Transformer ideas from the NLP field to the image field, providing excellent inspiration for a series of subsequent Transformer in Vision design works. We will use the ViT model for image classification tasks as an example to embark on a journey of applying Transformer ideas to the visual field.
Algorithms Used in the Project:
Multi-label/multi-class classification
Tools Used in the Project:
Expected Results of the Project:
-
Students will first implement the ViT model themselves and test the results on the dataset. They will then compare with the official implementation, and if there are significant differences, they will need to investigate the reasons.
-
Students will master how to apply the token and self-attention concepts from Transformers to the image domain. By understanding the principles, students should be able to apply Transformer ideas to other related problems.
-
Students will learn the training methods for ViT, running through the entire pipeline from data preparation, model training, parameter tuning, to model testing and metric calculation.
Corresponding Weeks of the Project: Weeks 1-3.
Project 2: Image Classification and Object Detection Tasks Based on SwinTransformer Model
Project Description: In the previous project, we learned about the ViT model, a successful visual Transformer model that applies Transformers to visual classification problems. However, the design of the ViT model is somewhat singular and has some shortcomings, especially regarding issues present in images, such as scale transformation problems that are not well addressed, and efficiency concerns that are not considered. In this project, we will study another advanced visual Transformer model: the SwinTransformer model.
Algorithms Used in the Project:
Forward-Backward Propagation
Tools Used in the Project:
Expected Results of the Project:
-
Students will implement the SwinTransformer code themselves (or refer to the official implementation) and optimize their implementation based on the official one. If there are significant differences in experimental results, students will need to investigate the reasons.
-
Students will appreciate the idea of using SwinTransformer for object detection.
-
Students will master how to optimize the implementation of the self-attention mechanism of SwinTransformer from local to global perspectives.
-
Students will learn how to apply Transformer ideas to their actual work or study-related problems.
Corresponding Weeks of the Project: Weeks 6-7.
Helping you become an industryTOP 10% engineer
Students interested in the course
Scan the QR code for consultation
-
Good foundation in programming and deep learning, aiming to enter the AI industry.
-
Strong interest in Transformers or federated learning, wishing to practice.
-
Need to apply machine learning, deep learning, and other technologies in their work.
-
Aiming to become AI algorithm engineers in the AI algorithm industry.
-
Wishing to broaden future career paths by mastering advanced AI knowledge.
PhD in Computer Science from the University of Oxford
Former algorithm scientist at multiple companies including BAT
Engaged in research related to computer vision, deep learning, and speech signal processing
Published several papers in top international conferences and journals such as CVPR, ICML, AAAI, ICRA
Course Development Consultant
Head of Recommendation Systems at Microsoft (Headquarters)
Senior Engineer at Amazon (Headquarters)
PhD from New Jersey Institute of Technology
14 years of research and project experience in artificial intelligence, digital image processing, and recommendation systems
Published over 20 papers in AI-related international conferences
PhD from the University of Southern California
Former Chief Data Scientist at unicorn JinKe Group, Senior Engineer at Amazon and Goldman Sachs
First to pioneer knowledge graphs for big data anti-fraud in the financial industry
Published over 15 papers in international conferences such as AAAI, KDD, AISTATS, CHI
Basic knowledge explanation
Interpretation of cutting-edge papers
Practical applications of this knowledge content
Project practice of this knowledge
Extension of knowledge in this direction and explanation of future trends
Helping you become an industryTOP 10% engineer
Students interested in the course
Scan the QR code for consultation
