Publication
Saliency Based Adjective Noun Pair Detection System
Marco Stricker; Syed Saqib Bukhari; Damian Borth; Andreas Dengel
In: Proceedings of the 10th International Conference on Agents and Artificial Intelligence. International Conference on Agents and Artificial Intelligence (ICAART), January 16-18, Funchal, Madeira, Portugal, SCITEPRESS, 2018.
Abstract
This paper investigates if it is possible to increase the accuracy of Convolutional Neural Networks trained on Adjective Noun Concepts with the help of saliency models.
Although image classification reaches high accuracy rates, the same level of accuracy is not reached for Adjective Noun Pairs, due to multiple problems. Several benefits can be gained through understanding Adjective Noun Pairs, like automatically tagging large image databases and understanding the sentiment of these images. This knowledge can be used for e.g. a better advertisement system.
In order to improve such a sentiment classification system a previous work focused on searching saliency methods that can reproduce the human gaze on Adjective Noun Pairs and found out that ``Graph-Based Visual Saliency'' belonged to the best for this problem. Utilizing these results we used the ``Graph-Based Visual Saliency'' method on a big dataset of Adjective Noun Pairs and incorporated these saliency data in the training phase of the Convolutional Neural Network. We tried out three different approaches to incorporate this information in three different cases of Adjective Noun Pair combinations. These cases either share a common adjective or a common noun or are completely different. Our results showed only slight improvements which were not significantly better besides for one technique in one case.