Catch Me If You Can: A Modified GAN Architecture Leveraging SIFT and Feature-matching for Highly Stable Training

Asraa Saeed; Ahmed A. Hashim

doi:10.32628/IJSRSET25121165

Authors

Asraa Saeed Iraqi Commission for Computers and Informatics, Information Institute for Postgraduate Studies, Baghdad, Iraq Author
Ahmed A. Hashim College of Business Informatics, University of Information Technology and Communications, Baghdad, Iraq Author

DOI:

https://doi.org/10.32628/IJSRSET25121165

Keywords:

GAN, SIFT, Keypoints, Feature-matching, Frank_GAN

Abstract

Generative Adversarial Networks (GANs) traditionally consist of two networks: a generator that creates new samples and a discriminator that evaluates the authenticity of these samples. Both networks are trained together competitively to generate samples indistinguishable from real data. This paper proposes a novel GAN architecture, Frank-GAN, which comprises two generators and one discriminator. The inspiration for this work comes from the movie "Catch Me If You Can." In the film, the main character, Frank, deceives the government by forging checks and certifications. After apprehending him, the government decides to utilize his expertise by appointing him to banking security to catch other forgers. Similarly, the second generator in Frank-GAN is specially trained using the feature-matching technique to match the activations of hidden discriminator-layer feature values between real and fake images, to enhance the GAN's effectiveness and stability. Unlike traditional GANs that rely on random noise as input, the input to the second generator consists of keypoints that are extracted from real images using a Scale-invariant feature (SIFT). Frank-GAN was evaluated on the CelebA and The Oxford 102 Flower datasets. The Fréchet inception distance metric (FID) improved from 48.5 to 7.04 on the CelebA dataset and from 49.84 to 6.04 on the Oxford 102 Flower dataset, while the inception score (IS) increased significantly from 2.70 to 18.68 on the CelebA dataset, when using Frank_GAN compared to traditional DCGANs. This demonstrates the good performance of Frank-GAN in generating high-quality and realistic images at 128×128 size.

Downloads

Download data is not yet available.

References

G. Ian, J. Pouget-Abadie, M. Mirza, B. Xu, and D. Warde-Farley, "Generative adversarial nets.” In Advances in neural information processing systems," 2014.

T. Chakraborty, U. R. KS, S. M. Naik, M. Panja, and B. Manvitha, "Ten years of generative adversarial nets (GANs): a survey of the state-of-the-art," Machine Learning: Science and Technology, vol. 5, p. 011001, 2024. DOI: https://doi.org/10.1088/2632-2153/ad1f77

Z. Liu, "Understanding Generative Adversarial Networks in the Context of Game Theory and Control Theory," University of Toronto (Canada), 2023.

G. Zhu, K. Zhou, L. Lu, Y. Fu, Z. Liu, and X. Yang, "Partial Discharge Data Augmentation Based on Improved Wasserstein Generative Adversarial Network With Gradient Penalty," IEEE Transactions on Industrial Informatics, 2022. DOI: https://doi.org/10.1109/TII.2022.3197839

A. J. Saeed and A. A. Hashim, "Precise GANs Classification Based on Multi-Scale Comprehensive Performance Analysis," Journal of Engineering Science & Technology Review, vol. 16, 2023. DOI: https://doi.org/10.25103/jestr.163.18

D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, pp. 91-110, 2004. DOI: https://doi.org/10.1023/B:VISI.0000029664.99615.94

A. Radford, L. Metz, and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv:1511.06434, 2015.

T. A. R. Shyaa and A. A. Hashim, "Enhancing real human detection and people counting using YOLOv8," in BIO Web of Conferences, 2024, p. 00061. DOI: https://doi.org/10.1051/bioconf/20249700061

T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, "Improved techniques for training gans," Advances in neural information processing systems, vol. 29, 2016.

I. O. Tolstikhin, S. Gelly, O. Bousquet, C.-J. Simon-Gabriel, and B. Schölkopf, "Adagan: Boosting generative models," Advances in neural information processing systems, vol. 30, 2017.

S. Cho and M. Choi, "MGDGAN: Multiple generator and discriminator generative adversarial networks for solving stochastic partial differential equations," IEEE Access, vol. 10, pp. 130908-130920, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3229696

S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang, "Generalization and equilibrium in generative adversarial nets (gans)," in International conference on machine learning, 2017, pp. 224-232.

A. Ghosh, V. Kulharia, V. P. Namboodiri, P. H. Torr, and P. K. Dokania, "Multi-agent diverse generative adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8513-8521. DOI: https://doi.org/10.1109/CVPR.2018.00888

Q. Hoang, T. D. Nguyen, T. Le, and D. Phung, "MGAN: Training generative adversarial nets with multiple generators," in International conference on learning representations, 2018.

H. Wu and J. Zhou, "Privacy leakage of sift features via deep generative model based image reconstruction," IEEE Transactions on Information Forensics and Security, vol. 16, pp. 2973-2985, 2021. DOI: https://doi.org/10.1109/TIFS.2021.3070427

J. Wei, M. Liu, J. Luo, A. Zhu, J. Davis, and Y. Liu, "DuelGAN: a duel between two discriminators stabilizes the GAN training," in European Conference on Computer Vision, 2022, pp. 290-317. DOI: https://doi.org/10.1007/978-3-031-20050-2_18

S. Pandey, "Neuroevolution of a Multi-Generator GAN (Student Abstract)," in Proceedings of the AAAI Conference on Artificial Intelligence, 2024, pp. 23608-23609. DOI: https://doi.org/10.1609/aaai.v38i21.30493

A. J. Saeed and A. A. Hashim, "Application of Smoothing Labels to Alleviate Overconfident of the GAN's Discriminator," Revue d'Intelligence Artificielle, vol. 38, pp. 407-415, 2024. DOI: https://doi.org/10.18280/ria.380204

K. Junji, T. Kosuke, K. Kyoto, and I. Taizo, "A Comparison of PSNR, WSNR and ESNR Evaluation Methods for The Two Value Modulated Images," Journal of Broadcast Engineering, vol. 10, pp. 149-155, 2005.

A. Hore and D. Ziou, "Image quality metrics: PSNR vs. SSIM," in 2010 20th international conference on pattern recognition, 2010, pp. 2366-2369. DOI: https://doi.org/10.1109/ICPR.2010.579

T. A. Shyaa and A. A. Hashim, "Superior Use of YOLOv8 to Enhance Car License Plates Detection Speed and Accuracy," Revue d'Intelligence Artificielle, vol. 38, 2024. DOI: https://doi.org/10.18280/ria.380114

B. Lingenfelter, S. R. Davis, and E. M. Hand, "A quantitative analysis of labeling issues in the celeba dataset," in International Symposium on Visual Computing, 2022, pp. 129-141. DOI: https://doi.org/10.1007/978-3-031-20713-6_10

M.-E. Nilsback and A. Zisserman, "Automated flower classification over a large number of classes," in 2008 Sixth Indian conference on computer vision, graphics & image processing, 2008, pp. 722-729. DOI: https://doi.org/10.1109/ICVGIP.2008.47

A. Brock, J. Donahue, and K. Simonyan, "Large scale GAN training for high fidelity natural image synthesis," arXiv preprint arXiv:1809.11096, 2018.

T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation," arXiv preprint arXiv:1710.10196, 2017.

T. Karras, S. Laine, and T. Aila, "A style-based generator architecture for generative adversarial networks," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4401-4410. DOI: https://doi.org/10.1109/CVPR.2019.00453