Think Tank Highlights #Global Defense Dynamics #US Military Dynamics #Russian Military Dynamics #Taiwan Affairs #Micro Store Sales
#South Korea #Raytheon #Japan #Electronic Warfare #Northeast Asia Military Dynamics #Drones
Neural Machine Translation Technology
Author: Military Eagle Think Tank Source: Military Eagle Dynamics
Neural Machine Translation (Neural Machine Translation, hereinafter referred to as “NMT”) is a self-driven machine translation technology based on deep neural networks. Since its introduction in 2014, NMT has been applied to various machine translation systems and has consistently achieved good performance. Its advantages can be summarized as: ① flexible model architecture with good scalability; ② the ability to utilize information and prior knowledge beyond the training dataset. However, it also has the following shortcomings that hinder the further practical deployment and application of NMT: ① the computational cost of training and translation inference is very high; ② the vocabulary is fixed after model training, making it difficult to handle rare words. To overcome these issues, Google developed the Google Neural Machine Translation System (GNMT).
Figure 1 Google GNMT Architecture
GNMT achieves technological breakthroughs in several areas:
(1) Enhancing Parallelism. To enhance parallelism and reduce training time, the attention mechanism of GNMT connects the bottom layer of the decoder to the top layer of the encoder.
(2) Increasing Computational Speed. To accelerate the final translation speed, GNMT uses low-precision calculations during the inference computation process.
(3) Handling Rare Words. In both input and output, GNMT divides words into multiple finite common subword units, effectively balancing the flexibility of letter-limited models with the effectiveness of word-limited models.
(4) Introducing Incentive Strategies. The beam search technology of GNMT uses a length normalization process and a coverage penalty, which can incentivize the generation of output sentences that are most likely to cover all words in the source sentence.
With the increasing international communication, enhancing multilingual military intelligence processing capabilities has become a priority, particularly for multilingual autonomous machine translation (as shown in Figure 2). Neural Machine Translation relies on large-scale data resources, but for resource-scarce languages (also known as “minor languages“), there is a lack of sufficient data resources to support it. To address this issue, GNMT introduced transfer learning strategies into autonomous machine translation, achieving a universal representation for multiple languages with a single model, thereby achieving the goal of “zero-shot“ translation, which allows the system to translate between languages it has never encountered without prior data.
Figure 2 Multilingual Autonomous Machine Translation
GNMT adds artificial tags at the beginning of input sentences to clarify the target language for translation, while the other parts of the model (including the encoder, decoder, and attention model) remain unchanged and can be shared across all languages. Parameter sharing allows the system to transfer “translation knowledge“ from one language pair to other language pairs, successfully achieving a universal representation modeling of “intermediate languages“, thereby connecting language pairs that did not appear in the training data. This model represents the possibility of achieving a “universal language“ model.
In large-scale multilingual autonomous machine translation experiments, GNMT achieved results comparable to human translation: ① the average translation error was reduced by 60% compared to Google’s already deployed phrase-based autonomous machine translation system; ② translation errors were reduced by more than 55%-85% in translations of several major language pairs. The successful development of Google’s GNMT system marks a milestone in making neural machine translation a reality for large-scale deployment.
Some Views
1Analysis of the Development Trend of Neural Machine Translation Technology
With the abundance of data resources and the enhancement of hardware computing capabilities, neural machine translation technology has become the mainstream technology in today’s autonomous machine translation, achieving significant improvements in timeliness and accuracy. For a considerable time in the future, the field of autonomous machine translation will still be dominated by neural machine translation. Looking at the development path and research status of autonomous machine translation, neural machine translation systems will exhibit the following future development trends:
(1) Hardware Upgrades Accompanying Improvements in Neural Machine Translation Performance
The development history of autonomous machine translation technology is also a history of hardware innovation. The achievements of Google GNMT are not only a significant breakthrough in theoretical research but also a reflection of strong engineering capability, highlighting the importance of hardware processing power: ① Google’s second-generation AI toolkit TensorFlow supports distributed computing across a large number of heterogeneous devices; ② Google’s Tensor Processing Units (TPU) provide sufficient computing power for deploying powerful GNMT, while also greatly reducing translation latency and improving translation timeliness.
(2) Leveraging Human Knowledge to Enhance Autonomous Machine Translation Performance
For a considerable period, research and applications in autonomous machine translation have struggled to address how to equip computers with sufficient cognitive abilities to overcome the “complexity“ of human language. Although the current GNMT is ahead of its time, it still makes some translation errors that humans would never make. The first international conference on machine translation proposed the discussion that “autonomous machine translation requires systematic formal knowledge about the world“. With the continuous improvement of high-quality knowledge bases such as Freebase, Probase, and Wikipedia, leveraging human knowledge to overcome the “semantic barrier“ faced by autonomous machine translation has gradually become feasible.
(3) Dual Learning and Reinforcement Learning to Reduce Data Annotation Costs
The tremendous success of neural machine translation is largely due to the large-scale labeled data. However, this approach has two limitations: ① the cost of obtaining labels through manual annotation is very high; to train an autonomous machine translation model that supports hundreds of languages, the cost of manual annotation can reach thousands of billions; ② in many application scenarios, it is impossible to obtain large-scale labeled data, such as mutual translation of minor languages. In such cases, dual learning and reinforcement learning techniques are believed to effectively compensate for the above shortcomings, while achieving true “unsupervised“ learning in neural machine translation is still a long way to go.
2 Neural Machine Translation Technology Plays a Huge Role in Multilingual Intelligence Processing
Global intelligence collection, military operations in non-native language environments, and multinational joint military exercises all require high-tech assistance to meet language activity needs. Therefore, autonomous machine translation technology plays an important role in multilingual military intelligence processing, where the demands for speed and quality in practical applications are very high. In today’s information age, characterized by explosive growth in data and knowledge, traditional autonomous machine translation technologies can no longer cope with the increasingly prominent characteristics of massive, open, and dynamic network intelligence, and have seriously lagged behind the actual needs of military intelligence processing work.With the emergence of neural machine translation technology, the previously seemingly “unachievable” goals are now close to realization.
(1）Accuracy of Autonomous Machine Translation for Military Intelligence: Approaching “Human-like“ Intelligence Neural Network Models, Ensuring the Accuracy of Neural Machine Translation.
The design mechanism of deep neural network models simulates the structure of the human brain. With the in-depth research and application, the results based on deep neural networks will continuously approach “human-like“ intelligence; moreover, due to their highly scalable architecture, prior knowledge can be effectively integrated. The translation accuracy of neural machine translation systems in large languages and specific fields has reached a level nearly indistinguishable from human translation.
(2) Timeliness of Autonomous Machine Translation for Military Intelligence: Rapidly Developing Hardware Technology Ensures the Timeliness of Neural Machine Translation.
In the ever-changing battlefield environment, the speed of intelligence processing determines decision-making efficiency. Relying on high-performance hardware support, using neural machine translation technology to replace (or assist) manual processes will effectively address the challenges of large volumes and short cycles of multilingual military intelligence processing, reducing the time costs of manual post-editing, and greatly enhancing the timeliness of multilingual intelligence processing.
