Introducing Huawei’s Noah’s Ark Lab
Chief Scientist of Voice Semantics Liu Qun
Answers our questions one by one
Natural Language Processing: The Crown Jewel of AI
Natural language processing is both a science and an applied technology that attempts to enable machines to simulate human language capabilities.
The research subject of natural language processing is human language, such as words, phrases, sentences, and texts. By analyzing these linguistic units, the goal is not only to understand literal meanings but also to grasp the emotions expressed by the speaker and the intentions conveyed.
Without successful natural language processing, there can be no true cognitive intelligence. Therefore, natural language processing is regarded as one of the core issues in artificial intelligence and is referred to as the “crown jewel of AI.”
Two Major Challenges Await Breakthroughs
As humanity enters the intelligent age, the growth of intelligent devices and data is unprecedented. Despite significant progress in the field of natural language processing over the years, many challenges remain. The two main issues are:
First, semantic understanding, specifically the learning of knowledge and common sense.
While understanding common sense is not a problem for humans, it is challenging to teach machines. For example, we can tell a mobile assistant to “find nearby restaurants,” and it will show the locations on a map. However, if you say “I am hungry,” the mobile assistant might remain unresponsive because it lacks the common sense that “being hungry means needing to eat.”
Much of this common sense is hidden deep in our consciousness, making it nearly impossible for AI system designers to summarize and input all such knowledge into the system.
Second, the low-resource problem.
In the face of scarce labeled data resources, such as machine translation for low-resource languages, specific domain dialogue systems, customer service systems, and multi-turn question-answering systems, there is currently no universal and effective solution in natural language processing.
In engineering practice, in addition to attempting to introduce domain knowledge (dictionaries, rules) to enhance data capabilities, we can also increase more manually labeled data based on active learning methods, utilize unlabeled data through unsupervised and semi-supervised methods, or employ multi-task learning to leverage information from other tasks or even other languages, as well as use transfer learning to utilize other models.
How Will It Change Our Lives?
The natural language processing research at Huawei’s Noah’s Ark Lab mainly includes three major directions: speech technology, machine translation, and dialogue.
Huawei’s mobile voice assistant integrates Noah’s Ark’s speech recognition and dialogue technology. Noah’s Ark’s machine translation technology supports the translation of a massive amount of technical documentation within Huawei. The knowledge graph-based question-answering technology from Noah’s Ark provides the global technical support system (GTS) of Huawei with the ability to quickly and accurately answer complex technical questions.
In fields such as finance, law, and healthcare, natural language processing technologies are also being increasingly applied.
For instance, natural language processing can provide various analytical data for securities investment, conduct financial risk analysis, and fraud detection; assist legal professionals in case search, judgment prediction, automatic generation of legal documents, legal text translation, and intelligent Q&A and help doctors with assisted entry of medical records, retrieval and analysis of medical information, and auxiliary diagnosis, among others.
Click “Read the original text” to view the full article “Natural Language Processing: Making Machines Understand Us Better.”