Transformers, as an attention-based encoder-decoder architecture, have not only revolutionized the field of Natural Language Processing (NLP) but have also made groundbreaking contributions in the field of Computer Vision (CV). Compared to Convolutional Neural Networks (CNNs), Visual Transformers (ViT) rely on their excellent modeling capabilities, achieving outstanding performance across multiple benchmarks such as ImageNet, COCO, and ADE20k.
As Atlas Wang, a computer scientist at the University of Texas at Austin, said: We have every reason to try using Transformers across the entire range of AI tasks.
Therefore, whether in academia or industry, it is essential for researchers and practitioners to gain a deep understanding of Transformer technology and keep up with cutting-edge research in Transformers to solidify their technical foundation.
AI is an easy subject to start with, but it is very difficult to delve into, which is also a significant reason why high-end AI talent is always in short supply.
In the workplace:
Are you able to flexibly propose new models according to practical scenarios?
Or suggest modifications to existing models?
In fact, these are core competencies and are thresholds that one must pass to become high-end talent. Although it is challenging, once you pass this threshold, you will find yourself in the TOP 5% of the market.
Therefore, we have designed this course with one purpose: to give you the opportunity to become part of the TOP 5% in the market. In this course, we will explain the principles, implementation methods, and application techniques of Transformers in the CV field from basic to advanced levels. Throughout the learning process, you can expand your thinking through real-world projects and integrate your knowledge, thereby truly enhancing your problem-solving abilities.
-
Comprehensive content explanation: covering the hottest Transformers in current applications and research fields, including 10+ Transformer models and application cases.
-
In-depth technical analysis: deeply analyze the technical details of Transformers and framework technologies, as well as the cutting-edge model principles covered by each module.
-
Real-world projects: including image recognition and object detection, enhancing students’ theoretical and practical skills in applications.
-
Expert instructor team: each module is taught by scientists or researchers with years of frontline experience in their respective fields, supported by experienced teaching assistants, dedicated to providing the highest quality learning experience.
▶ Comprehensive mastery of Transformer knowledge, flexibly applied in your work
▶ Ability to understand the implementation of Transformer model frameworks and proficiently master its key technologies and methods
▶ In-depth understanding of cutting-edge Transformer technologies, broadening your technical vision in work and research
▶ A comprehensive and systematic understanding of a field in a short period, greatly saving learning time
▶ Meet a group of like-minded individuals, exchange ideas, and learn from each other
Helping you become an industryTOP 10% engineer
Students interested in the course
Scan the QR code for consultation
Below is a detailed introduction to the CV part of the content; interested friends can inquire for more.
-
Comprehensive technical knowledge explanation
The course content covers explanations of over 10 models, including Bert, ViT, SegFormer, DETR, UP-DETR, TimeSformer, DeiT, Mobile-Transformer, Efficient Transformer, SwinTransformer, Point Transformer, MTTR, MMT, Uniformer, etc.
-
Project practice, applying what you’ve learned
Students will use Transformer models to practice image recognition and object detection tasks, which are the most widely used in the CV field.
-
Professionally crafted course content that is cutting-edge and in-depth
The course content has undergone hundreds of hours of design refinement to ensure that the content and project milestones are reasonable, truly achieving meaningful learning outcomes.
-
Employment-oriented, clear objectives
Outstanding students who successfully complete the course will have opportunities for internal referrals and interviews at major Internet companies such as ByteDance, Alibaba, Tencent, Meituan, as well as AI unicorn companies like SenseTime and Megvii.
Theme: Overview of Transformer/Bert Knowledge in NLP
This lesson will guide everyone to review the Transformer/Bert technology in the NLP field, allowing for a deeper understanding of the technical details and algorithm advantages of Transformer/Bert, facilitating further learning of Transformer technology in other fields.
-
Self-Attention mechanism, parallelization principles, etc., in Transformers in NLP.
-
Advanced principles of Transformer and Bert.
Theme: Applications of Transformer in Image Classification and Semantic Segmentation: Exploring ViT and SegFormer Technologies
Based on the content of the first lesson, further research how to transfer the ideas of Transformers to applications in classification problems in computer vision: image classification and image semantic segmentation. Using two classic structures, ViT and SegFormer, to let students experience how to apply Transformers to the visual field.
-
How to apply the design ideas of Transformers to image classification and semantic segmentation problems.
Theme: Applications of Transformer in Object Detection: Exploring DETR and UP-DETR Technologies
This lesson will further study how to apply Transformer technology to object detection tasks, especially how to design Transformer network structures that allow neural networks to learn both category information and location information of objects simultaneously.
-
In-depth understanding of the design ideas of applying Transformers to object detection.
-
-
Theme: Applications of Transformer in Video Understanding: Exploring TimeSformer Technology
This lesson will further study how to apply Transformer technology to video understanding applications, allowing Transformers to learn spatial and temporal correlations simultaneously. Using TimeSformer as an example, students will deeply experience the design ideas involved.
-
Issues to consider when extending Transformer design ideas to modeling temporal-spatial correlations.
Theme: Discussion on Efficient Transformer Design: Exploring DeiT and Mobile-Transformer Technologies
Efficient Transformers have always been a goal that researchers strive for. This course will discuss how to design efficient Transformer network structures. This lesson will use DeiT and Mobile-Transformer as examples to delve into the considerations needed during the efficient design process.
-
Considerations in the design of Efficient Transformers and discussions on optimizing Transformer perspectives.
Theme: Learning Classic Transformer Network Structures: Learning the SwinTransformer Model Family
This course will systematically study the SwinTransformer model as an example, aiming to help students further understand the issues that need to be considered when applying Transformers to visual tasks, the ingenious ideas involved, and how to achieve parallel computation through reasonable design.
-
SwinTransformer model family
-
SwinTransformer design ideas. Considerations when designing Transformers to solve new problems.
Theme: Transformer in Point Cloud
This lesson will share the application of Transformers in 3D Point Clouds. Based on the characteristics of 3D Point Cloud data, we will explore how to design suitable Transformer networks to handle massive, unstructured point cloud data, as well as how to further modify the Transformer structure for tasks such as segmentation and clustering.
-
Considerations when designing Transformers to handle point cloud data.
-
Theme: Transformer Design in Multi-modal Applications
This lesson will explore the design issues of Transformers in multi-modality. Transformers have been well applied in various fields. Recent work has explored how to design suitable Transformer structures to handle multi-modal data. We will use MTTR, MMT, and Uniformer as examples for explanation.
-
Investigating considerations when designing Transformers to handle multi-modal data.
-
How to design suitable Transformers for multi-modal-related issues: MTTR, MMT, Uniformer.
Project 1: Image Recognition System Based on ViT Model
Project Description: As a classic application case of Transformers in the visual field, the ViT model was the first to apply the Transformer concept from the NLP field to the image domain, providing great inspiration for subsequent Transformer in Vision design work. Tracing back, we will take the ViT model for image classification tasks as an example to embark on a journey of applying Transformer ideas to the visual domain.
Algorithms used in the project:
Multi-label/multi-class classification
Tools used in the project:
Expected results of the project:
-
First, students will implement the ViT model themselves, testing results on the dataset. Then, they will compare with the official implementation; if there are significant differences, they need to investigate the reasons.
-
Master how to apply the concepts of tokens and self-attention from Transformers to the image domain. It is hoped that students can apply the Transformer ideas to other related problems based on a profound understanding.
-
Master the training methods of ViT, allowing students to run through this pipeline. From data preparation, model training, parameter tuning, to model testing and metric calculation.
Project corresponds to which week’s course: Weeks 1-3.
Project 2: Image Classification and Object Detection Tasks Based on SwinTransformer Model
Project Description: In the previous project, we learned about the ViT model, a successful visual transformer model that applies Transformers to visual classification problems. However, the design of the ViT model is still relatively singular and has some shortcomings, especially regarding issues present in images, such as scale transformation problems that were not well addressed, and efficiency issues were not considered. In this project, we will learn about another advanced visual transformer model: the SwinTransformer model.
Algorithms used in the project:
Forward-Backward Propagation
Tools used in the project:
Expected results of the project:
-
Students will implement the SwinTransformer code themselves (or refer to the official implementation) and optimize their implementation based on the official version. If there are significant differences in experimental results, students will need to investigate the reasons.
-
Experience the ideas of using SwinTransformer for object detection.
-
Master how to optimize the implementation of the self-attention mechanism of SwinTransformer from local to global from a coding perspective.
-
Students will master how to apply Transformer ideas to practical problems in their work or studies.
Project corresponds to which week’s course: Weeks 6-7.
Helping you become an industryTOP10% engineer
Students interested in the course
Scan the QR code for consultation
-
Have a good foundation in programming and deep learning, aiming to enter the AI industry for development.
-
Have a strong interest in Transformers or federated learning and wish to practice.
-
Need to apply machine learning, deep learning, and other technologies in their work.
-
Want to enter the AI algorithm industry to become an AI algorithm engineer.
-
Wish to broaden their future career paths by mastering advanced AI knowledge.
PhD in Computer Science from Oxford University
Former algorithm scientist at multiple companies including BAT
Engaged in research related to computer vision, deep learning, and speech signal processing
Has published multiple papers in top international conferences and journals such as CVPR, ICML, AAAI, ICRA
Course Development Consultant
Head of Recommendation Systems at Microsoft (Headquarters)
Senior Engineer at Amazon (Headquarters)
PhD from New Jersey Institute of Technology
14 years of research and project experience in artificial intelligence, digital image processing, and recommendation systems
Has published over 20 papers at international conferences related to AI
PhD from the University of Southern California
Former Chief Data Scientist at unicorn company JinKe Group, Senior Engineer at Amazon and Goldman Sachs
Pioneer in using knowledge graphs for big data anti-fraud in the financial industry
Has published over 15 papers at international conferences such as AAAI, KDD, AISTATS, CHI
Basic knowledge explanation
Interpretation of cutting-edge papers
Practical applications of the knowledge
Project practice of the knowledge
Extension of knowledge in this direction and explanation of future trends
Helping you become an industryTOP10% engineer
Students interested in the course
Scan the QR code for consultation
