Deep Learning Based Text to Image Generation

Abstract

Text-to-image generation is a method used for generating images related to given textual descriptions. It has a significant influence on many research areas as well as a diverse set of applications (e.g., photo-searching, photo-editing, art generation, computer-aided design, image re-construction, captioning, and portrait drawing). The most challenging task is to consistently produce realistic images according to given conditions. Existing algorithms for text-to-image generation create pictures that do not properly match the text. We considered this issue in our study and built a deep learning-based architecture for semantically consistent image generation: recurrent convolutional generative adversarial network (RC-GAN). RC-GAN successfully bridges the advancements in text and picture modelling, converting visual notions from words to pixels. The proposed model was trained on the Oxford-102 flowers dataset, and its performance was evaluated using an inception score and PSNR. The experimental results demonstrate that our model is capable of generating more realistic photos of flowers from given captions, with an inception score of 4.15 and a PSNR value of 30.12 dB, respectively. Generating images from natural language is one of the primary applications of conditional generative models. This project uses Generative Adversarial Networks (GANs) to generate an image given a text description. GANs are Deep Neural Networks that are generative models of data. Given a group of coaching data, GANs can learn to estimate the underlying probability distribution of the info. In this project, the model is trained on the Caltech birds dataset. Recent progress has been made using GANs.