Pytorch

Summary and Code Implementation of Attention Mechanisms in Deep Learning (2017-2021)

by
Summary and Code Implementation of Attention Mechanisms in Deep Learning (2017-2021)

Machine Learning Algorithms and Natural Language Processing(ML-NLP) is one of the largest natural language processing communities both domestically and internationally, gathering over 500,000 subscribers, covering NLP master’s and doctoral students, university teachers, and corporate researchers. Community Vision is to promote communication and progress between the academic and industrial circles of natural language processing and enthusiasts … Read more

Understanding Self-Attention Mechanism: 8 Steps with Code

by

Originally from New Machine Vision Source: towardsdatascience Author: Raimi Karim Edited by: Xiao Qin [Introduction]The recent rapid advancements in the field of NLP are closely related to architectures based on Transformers. This article guides readers to fully understand the self-attention mechanism and its underlying mathematical principles through diagrams and code, and extends to Transformers. BERT, … Read more

Understanding Q, K, V in Attention Mechanism

by
Understanding Q, K, V in Attention Mechanism

Source | Zhihu Q&A Address | https://www.zhihu.com/question/298810062 This article is for academic sharing only. Please contact us for removal if there are any copyright issues. 01 Answer 1: Author – Not Uncle Let’s directly use torch to implement a Self-Attention and discuss: 1. First, define three linear transformation matrices: query, key, and value: class BertSelfAttention(nn.Module): … Read more

Fine-Tuning TrOCR for Curved and Blurry Text Recognition

by
Fine-Tuning TrOCR for Curved and Blurry Text Recognition

Author: Sovit Rath Translated by: ronghuaiyang Introduction This article fine-tunes the TrOCR model on a dataset of curved and blurry text, analyzing the code and training results at each step. TrOCR (Transformer based Optical Character Recognition) model is one of the best OCR models. In previous articles, we analyzed how well this model performs on … Read more

Implementing OCR Character Recognition with Transformer

by
Implementing OCR Character Recognition with Transformer

Click on the above “Visual Learning for Beginners“, select to add “Starred” or “Top“ Heavyweight content delivered first-hand Authors: An Sheng, Yuan Mingkun, Datawhale Members In the field of CV, what else can transformers do besides classification? This article will use a word recognition task dataset to explain how to use transformers to implement a … Read more

Unlocking the World of OCR: Comprehensive Toolkit and Datasets

by
Unlocking the World of OCR: Comprehensive Toolkit and Datasets

In daily life, do you often use WeChat to recognize images and extract text information? Besides this, there are other applications like photo-based question searching, photo translation, document information extraction, and logistics information recognition, all thanks to the support of OCR technology. With the continuous development of deep learning technologies, intelligent OCR algorithms and applications … Read more

Essential Knowledge! 5 Major Deep Generative Models!

by
Essential Knowledge! 5 Major Deep Generative Models!

About 5200 words, recommended reading time 10 minutes. This article summarizes commonly used deep learning models, providing an in-depth introduction to their principles and applications. With the rise of models like Sora, diffusion, and GPT, deep generative models have once again become the focus of attention. Deep generative models are a class of powerful machine … Read more

Complete Guide to Pretraining LLAMA3 from Scratch: Exploring Scaling Law

by
Complete Guide to Pretraining LLAMA3 from Scratch: Exploring Scaling Law

MLNLP community is a well-known machine learning and natural language processing community at home and abroad, covering NLP master’s and doctoral students, university teachers, and corporate researchers. The vision of the community is to promote communication and progress between the academic and industrial circles of natural language processing and machine learning at home and abroad, … Read more

Llama 3.1 Training Issues: GPU Failures and Performance Impact

by
Llama 3.1 Training Issues: GPU Failures and Performance Impact

MLNLP community is a well-known machine learning and natural language processing community in China and abroad, covering a wide audience including NLP master’s and doctoral students, university teachers, and corporate researchers. The vision of the community is to promote communication and progress between the academic and industrial circles of natural language processing and machine learning, … Read more

Building an LLM from Scratch: A Step-by-Step Guide

by
Building an LLM from Scratch: A Step-by-Step Guide

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP master’s and PhD students, university teachers, and researchers from enterprises. The vision of the community is to promote communication and progress between the academic and industrial sectors of natural language processing and machine learning, especially for beginners. … Read more