Yann LeCun highly affirms GAN.
Introduction: Generative Adversarial Networks (GAN) is a generative model proposed by Ian Goodfellow et al. in 2014. Researcher Wang Feiyue and others elaborated on the research progress and development trends of GAN in the third issue of the Journal of Automation. They first summarized the background, theory, implementation models, and application fields of GAN, then analyzed the advantages and disadvantages of GAN and looked forward to its development trends, focusing on the relationship between GAN and parallel intelligence.
Under the combined influence of the artificial intelligence boom, the accumulation of generative models, the deepening of neural networks, and the success of adversarial ideas, GAN emerged and immediately attracted the attention of AI researchers.
Structurally, GAN is inspired by the two-player zero-sum game in game theory (where the sum of benefits for two players is zero; what one gains is what the other loses). The system consists of a generator G and a discriminator D. The generator aims to capture the potential distribution of real data samples and generate new data samples; the discriminator aims to correctly determine whether the input data comes from real data or the generator. Both the generator and discriminator can use deep neural networks. To win the game, these two participants need to continuously optimize and improve their generation and discrimination capabilities.
The optimization process of GAN is a minimax game problem, with the optimization goal of reaching Nash equilibrium, allowing the generator to estimate the distribution of data samples. The computational process and structure of GAN are shown in Figure 1. Since Goodfellow et al. proposed GAN, several derivative models based on GAN have emerged (e.g., W-GAN, LS-GAN, Semi-GAN, C-GAN, Bi-GAN, Info-GAN, AC-GAN, Seq-GAN, etc.), with innovations including model structure improvements, theoretical expansions, and applications.
Computational Process and Structure of GAN
Research and Applications of GAN
In the current boom of artificial intelligence, the introduction of GAN meets the research and application needs of many fields, injecting new development momentum into these areas. GAN has become a hot research direction in the AI academic community, with the famous scholar Yann LeCun even calling it “the most exciting idea in machine learning in the past decade.” Currently, the image and vision fields are the most widely researched and applied areas for GAN, capable of generating digital objects, faces, and various realistic indoor and outdoor scenes, restoring original images from segmented images, coloring black-and-white images, recovering object images from outlines, and generating high-resolution images from low-resolution ones. Figure 2 shows a super-resolved image generated by GAN. Figure 3 shows a driving scene image generated by GAN. Moreover, GAN has begun to be applied to research problems in speech and language processing, computer virus monitoring, and chess competition programs.
Super-resolved image generated by GAN
Driving scene image generated by GAN
(Odd columns are generated images, even columns are target images)
Advantages and Disadvantages of GAN and Development Trends
GAN employs adversarial learning criteria, and theoretically, it cannot yet determine the convergence and existence of equilibrium points of the model. The training process needs to ensure the balance and synchronization of the two adversarial networks; otherwise, it is difficult to achieve good training results. However, in practice, synchronizing the two adversarial networks is not easy to control, and the training process may become unstable. Additionally, as a generative model based on neural networks, GAN has the general shortcomings of neural network models, namely poor interpretability. Although the samples generated by GAN exhibit diversity, they also suffer from collapse mode phenomena, potentially generating diverse samples that are not significantly different to humans.
Despite these issues, the research progress achieved indicates that GAN has broad development prospects. Wasserstein GAN has completely addressed the training instability problem while essentially resolving the collapse mode phenomenon. How to completely solve the collapse mode and continue to optimize the training process is a research direction for GAN. Furthermore, theoretical inferences regarding the convergence and existence of equilibrium points of GAN are also important research topics for the future. From the perspective of developing applications for GAN, how to generate diverse data that can interact with humans based on simple random inputs is a recent application development direction. From the perspective of integrating GAN with other methods, exploring how to better integrate GAN with feature learning, imitation learning, reinforcement learning, and other technologies to develop new AI applications or promote the development of these methods is a significant and meaningful development direction.
In the long run, how to utilize GAN to promote the development and application of artificial intelligence, enhance AI’s ability to understand the world, and even stimulate AI’s creativity is a question worth researchers’ consideration.
Relationship Between GAN and Parallel Intelligence
Finally, the relationship between GAN and parallel intelligence is discussed, suggesting that GAN can deepen the concept of virtual-real interaction and integrated interaction in parallel systems, especially the idea of computational experiments, providing very concrete and rich algorithmic support for ACP (Artificial societies, computational experiments, and parallel execution) theory. Parallel systems emphasize virtual-real interaction, constructing artificial systems to describe and represent actual systems, using computational experiments to learn and evaluate various computational models, and enhancing the performance of actual systems through parallel execution, allowing artificial and actual systems to advance together. The ACP theory and parallel systems method have now developed into a broader theory of parallel intelligence.
During the GAN training process, real data samples and generated data samples interact through adversarial networks, and the trained generator can produce more virtual samples than real samples. In several parallel systems, such as parallel vision, parallel control, and parallel learning, GAN can deepen the concept of virtual-real interaction and integrated interaction by generating data samples that share the same distribution as real data, supporting theoretical and application research in parallel systems. Therefore, as an effective generative model, GAN can be integrated into the research framework of parallel intelligence.
Authors:
Wang Kunfeng, Associate Researcher at the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. His main research directions include intelligent transportation systems, intelligent visual computation, and machine learning.
E-mail: [email protected]
Gou Chao, PhD student at the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. His main research directions include intelligent transportation systems, image processing, and pattern recognition.
E-mail: [email protected]
Duan Yanjie, PhD student at the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. His main research directions include intelligent transportation systems, machine learning, and applications.
E-mail: [email protected]
Lin Yilun, PhD student at the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. His main research directions include social computing, intelligent transportation systems, deep learning, and reinforcement learning.
E-mail: [email protected]
Zheng Xinhua, Graduate student at the University of Minnesota, Department of Computer Science and Engineering. His main research directions include social computing, machine learning, and data analysis.
E-mail: [email protected]
Wang Feiyue, Researcher at the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. Director of the Military Computing Experiment and Parallel Systems Technology Research Center at National University of Defense Technology. His main research directions include modeling, analysis, and control of intelligent systems and complex systems. Corresponding author of this paper.
E-mail: [email protected]