SmartWeb Handheld - Multimodal Interaction with Ontological Knowledge Bases and Semantic Web Services

Daniel Sonntag, Ralf Engel, Gerd Herzog, Alexander Pfalzgraf, Norbert Pfleger, Massimo Romanelli, Norbert Reithinger

In: International Workshop on AI for Human Computing (AI4HC) in conjunction with (IJCAI) 2007 International Workshop on AI for Human Computing (AI4HC) in conjunction with (IJCAI) 2007. International Workshop on AI for Human Computing (AI4HC) 1/2007.


SMARTWEB aims to provide intuitive multimodal access to a rich selection of Web-based information services. We report on the current prototype with a smartphone client interface to the Semantic Web. An advanced ontology-based representation of facts and media structures serves as central description for rich media content. Underlying content is accessed through conventional web service middleware to connect the ontological knowledge base and an intelligent web service composition module for external web services, which is able to translate between ordinary XML-based data structures and explicit semantic representations for user queries and system responses. The presentation module renders the media content and the results generated from the services and provides a detailed description of the content and its layout to the fusion module. The user is then able to employ multiple modalities, like speech and gestures, to interact with the presented multimedia material in a multimodal way.


ijcai07-sw.pdf (pdf, 804 KB ) 2007_SmartWeb_Handheld_-_Multimodal_Interaction_with_On.pdf (pdf, 4 MB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz