Publikation
Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs
Damian Borth; Rongrong Ji; Tao Chen; Thomas Breuel; Shih-Fu Chang
In: ACM MM 2013. ACM International Conference on Multimedia (ACM MM-13), October 21-25, Barcelona, Spain, ACM, 10/2013.
Zusammenfassung
We address the challenge of sentiment analysis from visual content. In contrast to existing methods which infer sentiment or emotion directly from low-level features, we propose a novel approach based on understanding the semantics of images. Our key contribution is two-fold: first, we present a psychology theory based method to automatically construct a large-scale Visual Sentiment Ontology (VSO) consisting of more than 3,000 Adjective Noun Pairs (ANP). Second, we propose SentiBank, a novel mid-level representation framework built upon the VSO encoding the concept presence of 1,200 ANPs from visual content. The Visual Sentiment Ontology and SentiBank are distinct from existing works and will open a gate towards various high-level applications. Sentiment analysis experiments on real-world Twitter data covering 2,000 image tweets demonstrate that our joint visual-text approach improves prediction accuracy by 13% (absolute gain) over the state-of-the-art text only methods. The effort will also leads to a large publicly available resource consisting of a concept ontology, a detector library, and the training/testing benchmark for visual sen- timent analysis.