New Method Proposed by Software Institute to Enhance GAN Model Performance

Recently, the research team of the National Key Laboratory of Space-Based Integrated Information System at the Software Institute had their paper Intriguing Property and Counterfactual Explanation of GAN for Remote Sensing Image Generation accepted by the top journal in the field of computer vision, the International Journal of Computer Vision (IJCV). The paper presents a new method that effectively enhances the performance of Generative Adversarial Networks (GAN) to improve the quality of remote sensing image generation. The first author of the paper is PhD student Su Xingzhe, and the corresponding author is Special Research Assistant Qiang Wenwen.

Since GAN models are generally designed based on natural images, they often overlook the characteristics of remote sensing images when applied in this field. The research team found that, compared to natural images, the quality of remote sensing images generated by GAN shows more significant changes when the number of categories in the training dataset or the number of samples in a single category changes.

To explore the reasons for this phenomenon, the research team first established a structural causal model for the data generation process, proving that the quality of GAN-generated remote sensing images is positively correlated with the amount of feature information. Meanwhile, the team conducted experiments on the features of the generated remote sensing images. The results showed that the generated features become sparser as the number of training categories decreases; when the number of samples in each category decreases, the distribution of generated samples in feature space becomes increasingly uneven. Inspired by information theory, which states that sparser feature information has lower entropy while more uniformly distributed information has higher entropy, the research team concluded that when the amount of training data or the number of samples in a single category decreases, the amount of feature information contained in the GAN model significantly decreases, leading to a decline in generation quality.

Based on the above experimental results, the research team proposed a plug-and-play method to enhance the performance of GAN models. Specifically, on one hand, it increases the amount of feature information learned by the GAN model at the distribution level by using a Gaussian potential kernel function to propose Uniformity Regularization (UR), constraining the sample feature distribution to be uniform in feature space; on the other hand, it enhances the amount of feature information at the sample level by proposing Entropy Regularization (ER), maximizing the feature information entropy of individual samples to improve the quality of remote sensing image generation.

Method Architecture Diagram

The research team validated their method using different network architectures, data augmentation, and regularization combinations on remote sensing images of varying resolutions and modalities. The results showed that after applying the method proposed by the research team, the quality of remote sensing image generation was significantly improved. Experimental results on natural images also demonstrated the effectiveness and universality of the method.

Experimental Results on Multiple Remote Sensing Image Datasets

Experimental Results on Natural Image Datasets

Paper Link:

https://link.springer.com/article/10.1007/s11263-024-02125-4

END