Saliency Based Adjective Noun Pair Detection System

Publication

Marco Stricker; Syed Saqib Bukhari; Damian Borth; Andreas Dengel

In: Proceedings of the 10th International Conference on Agents and Artificial Intelligence. International Conference on Agents and Artificial Intelligence (ICAART), January 16-18, Funchal, Madeira, Portugal, SCITEPRESS, 2018.

Abstract

This paper investigates if it is possible to increase the accuracy of Convolutional Neural Networks trained on Adjective Noun Concepts with the help of saliency models. Although image classification reaches high accuracy rates, the same level of accuracy is not reached for Adjective Noun Pairs, due to multiple problems. Several benefits can be gained through understanding Adjective Noun Pairs, like automatically tagging large image databases and understanding the sentiment of these images. This knowledge can be used for e.g. a better advertisement system. In order to improve such a sentiment classification system a previous work focused on searching saliency methods that can reproduce the human gaze on Adjective Noun Pairs and found out that ``Graph-Based Visual Saliency'' belonged to the best for this problem. Utilizing these results we used the ``Graph-Based Visual Saliency'' method on a big dataset of Adjective Noun Pairs and incorporated these saliency data in the training phase of the Convolutional Neural Network. We tried out three different approaches to incorporate this information in three different cases of Adjective Noun Pair combinations. These cases either share a common adjective or a common noun or are completely different. Our results showed only slight improvements which were not significantly better besides for one technique in one case.

Projekte

SDSD - Smarte Daten, Smarte Dienste. Landwirtschaftliche Datendrehscheibe für effiziente ressourcenschonende Prozesse

Saliency_Based_Adjective_Noun_Pair_Detection_System.pdf (pdf, 8 MB )

Saliency Based Adjective Noun Pair Detection System

Abstract

Projekte

Follow us on:

Editor

Newsletter

Locations