Publication
Conditional GANs for Image Captioning with Sentiments
Tushar Karayil; Federico Raue; Jörn Hees; Dengel
In: 2019 International Conference on Artificial Neural Networks. International Conference on Artificial Neural Networks (ICANN-2019), 28th International Conference on Artificial Neural Networks. September 17-19, Munich, Germany, Springer, 2019.
Abstract
The area of automatic image captioning has witnessed much progress recently. However, generating captions with sentiment, which is a common dimension in human generated captions, still remains a challenge. This work presents a generative approach that combines sentiment (positive/negative) and variation for caption generation. The presented approach consists of a Generative Adversarial Network which takes as input, an image and a binary vector indicating the sentiment of the caption to be generated. We evaluate our model quantitatively on the state-of-the-art image caption dataset and qualitatively using a crowd-sourcing platform. Our results, along with human evaluation prove that we competitively succeed in the task of creating variations and sentiment in image captions.