Diffusion Transformer

Speech Recognition Method Based on Multi-Task Loss with Additional Language Model

by
Speech Recognition Method Based on Multi-Task Loss with Additional Language Model

Click the blue text to follow us DOI：10.3969/j.issn.1671-7775.2023.05.010 Open Science (Resource Service) Identifier Code (OSID): Citation Format: Liu Yongli, Zhang Shaoyang, Wang Yuheng, et al. Speech Recognition Method Based on Multi-Task Loss with Additional Language Model[J]. Journal of Jiangsu University (Natural Science Edition), 2023, 44(5):564-569. Fund Project: Shaanxi Provincial Key Industry Innovation Chain (Group) Project … Read more

Multi-Dialect Voice Recognition Method for the Railway Sector

by
Multi-Dialect Voice Recognition Method for the Railway Sector

0 Introduction The railway, as an important national infrastructure, integrates intelligent customer service systems with cloud computing, big data, and artificial intelligence technologies, enhancing service efficiency and passenger experience. Since 2018, the railway industry has been exploring the intelligentization of the 12306 customer service system, fully implementing the intelligent customer service system by the end … Read more

Fast and Effective Overview of Lightweight Transformers in Various Fields

by
Fast and Effective Overview of Lightweight Transformers in Various Fields

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP master’s and doctoral students, university teachers, and enterprise researchers. Community Vision is to promote communication and progress between the academic and industrial sectors of natural language processing and machine learning, especially for beginners. Reprinted from | RUC … Read more

Overview of 17 Efficient Variants of Transformer Models

by
Overview of 17 Efficient Variants of Transformer Models

Follow the public account “ML_NLP“ Set as “Starred” for heavy content delivered first-hand! Reprinted from | Xiaoyao’s Cute Selling House Written by | Huang Yu Source | Zhihu In the field of NLP, transformer has successfully replaced RNNs (LSTM/GRU), and has also found applications in CV, such as object detection and image annotation, as well … Read more

Understanding Transformer Architecture: A Complete PyTorch Implementation

by
Understanding Transformer Architecture: A Complete PyTorch Implementation

MLNLP ( Machine Learning Algorithms and Natural Language Processing ) community is a well-known natural language processing community both domestically and internationally, covering NLP master’s and doctoral students, university professors, and corporate researchers. The vision of the community is to promote communication between the academic and industrial circles of natural language processing and machine learning, … Read more

Understanding Mamba: The Strongest Competitor to Transformers

by
Understanding Mamba: The Strongest Competitor to Transformers

Source: Machine Heart This article is about 5400 words, and it is recommended to read for more than 10 minutes. Mamba is promising, but its development is still in the early stages. There are many deep learning architectures, but in recent years, none have been as successful as the Transformer, which has established its dominance … Read more

Introduction to Attention Mechanisms in Three Transformer Models and PyTorch Implementation

by
Introduction to Attention Mechanisms in Three Transformer Models and PyTorch Implementation

This article delves into three key attention mechanisms in Transformer models: self-attention, cross-attention, and causal self-attention. These mechanisms are core components of large language models (LLMs) like GPT-4 and Llama. By understanding these attention mechanisms, we can better grasp how these models work and their potential applications. We will discuss not only the theoretical concepts … Read more

What Is the Transformer Model?

by
What Is the Transformer Model?

Welcome to the special winter vacation column “High-Tech Lessons for Kids” presented by Science Popularization China! Artificial intelligence, as one of the most cutting-edge technologies today, is rapidly changing our lives at an astonishing pace. From smart voice assistants to self-driving cars, from AI painting to machine learning, it opens up a future full of … Read more

Understanding Transformer Architecture: A PyTorch Implementation

by
Understanding Transformer Architecture: A PyTorch Implementation

This article shares a detailed blog post about the Transformer from Harvard University, translated by our lab. The Transformer architecture proposed in the paper “Attention is All You Need” has recently attracted a lot of attention. The Transformer not only significantly improves translation quality but also provides a new structure for many NLP tasks. Although … Read more

2025 Large Models and Transformer Architecture: Technology Frontiers and Future Trends Report

by

“Omega Future Research Institute” focuses on the future development trends of technology, studying the major opportunities and challenges faced by humanity in the evolution process towards the Omega point. We will periodically recommend and publish important technological research progress and future trend studies from around the world. (Click here to view the Omega theory) In … Read more