Publication
Declarative AR and Image Processing on the Web with Xflow
Felix Klein; Dmitri Rubinstein; Kristian Sons; Farshad Einabadi; Stephan Herhut; Philipp Slusallek
In: Proceedings of the 18th International Conference on Web 3D Technology. ACM International Conference on 3D Web Technology (Web3D-13), 18th, June 20-22, San Sebastian, Spain, ACM, 2013.
Abstract
Recently, modern Web browser became capable of supporting powerful, interactive 3D graphics both via the low-level, imperative API of WebGL as well as via a high-level, declarative approach like XML3D. The obvious next step (particularly with respect to mobile platforms) is to combine video from the real world with matched virtual content -- Augmented or Mixed Reality (AR/MR). However, AR requires extensive image or video processing, feature detection and tracking, and applying the results to 3D rendering -- all of which is hard to implement in a Web context.
In this paper we present a novel approach that encapsulates low-level image-processing and AR operations into re-usable high-level XML3D/Xflow components that are part of the HTML-5 DOM. A Web developer can then easily and flexibly arrange these components into (possibly complex) processing flow-graphs without having to worry about the internal computations and the structure of these modules. Our extended Xflow implementation automatically optimizes, schedules, and synchronizes the processing of the flow graph(s) in the context of real-time 3D rendering. Finally, we provide an integration model that greatly simplifies building AR applications for the browser.
We demonstrate this with several simple AR and image processing applications using a polyfill implementation working in all modern browsers and evaluate the performance. Finally, we show how the declarative framework can be optimized with respect to performance and usability using parallelization with Web Workers and RiverTrail.