1. Architectural Design
The high-level architecture shown in Figure 1 presents the basic functional blocks of the SPECTER system and
their fundamental interrelations. This conceptual base architecture and its modularization into main components
directly reflects the underlying organization into major research topics.
Figure 1: Conceptual Base Architecture
The necessary interconnection of the various components requires an ontological infrastructure for knowledge
exchange inside SPECTER. The two central aspects of our basic approach are the use of shared ontologies and to
utilize emerging standards for the notation of data and metadata, like XML, XML Schema, RDF/S, OWL, XQuery, etc.
Shared ontologies provide common vocabulary to enable declarative representations for data exchange between system
components. The knowledge engineering task can draw on general results in the following areas:
The architecture sketch in Figure 2 provides a more detailed view on the different software components of the
overall SPECTER system. This diagram as well highlights the important role of user model and personal journal as
central and persistent knowledge sources. The modular structure shown here indicates that instrumentation within
a mobile context does not only relate to the local environment, but includes also mobile equipment carried around
by the user, like a handheld computer and specific sensors to monitor his or her affective state.
Figure 2: Refined Component Architecture
SPECTER needs to be able to interoperate with different kinds of instrumented environments. A dedicated interface
component encapsulates the technical details of gaining access to the specific functionality provided by the current
2. Instrumented Environment
A major challenge for linking into an instrumented environment is the heterogenity of available infrastructure
frameworks. Current approaches use different technologies like Jini, UPnP, Bluetooth, Web Services, or custum research
prototypes and none of them directly supports semantic interoperability. The approach followed within SPECTER can
be characterized as follows:
- The specific instrumentation is conceptualized as a set of services which can be regarded as virtual sensors
- A separate interface component provides a generic bridge to the local infrastructure and maintains a semantic
model of the environment.
- The representation of services relies on OWL-S and related specifications.
- Individual Jini services as well as standardized UPnP and Bluetooth profiles can be recast as Semantic Web
Services to provide the necessary grounding.
The initial test environment for SPECTER mimics an instrumented shop with RFID equipped products and readers
within the shelf and basket (see Figure 3). In a mobile context, such as a shopping scenario, all direct interactions
between user and SPECTER itself are performed via a PDA.The laptop on the right-hand side provides the software
infrastructure of the instrumented shop, enables communication between shop and mobile assistant, and provides
SPECTER with input. Using radio-frequency identification tags, the system can detect what product the user has
taken from the shelf or put into the basket.
Figure 3: Experimental Setup for an Instrumented Shop
The initial version of our experimental environment for the shopping scenario relies on the SmartShop software
provided by our cooperation partners at Universität des Saarlandes.
It provides a Jini-based software infrastructure for instrumented environments and includes basic service implementations
forRFID-sensors, object identification and product information. Information access is realized through virtual
shared memory using the tuple space paradigm. Future versions of the infrastructure software are planned to incorporate
a growing list of more advanced shopping services (e.g. for cross-selling). Figure 3 presents an overview of the
different runtime components on the infrastructure level, including standard Jini services as well as environment-specific
Figure 4: Runtime Components of the Infrastructure Software
3. Sample Interaction
This section presents a typical Interaction with the initial prototype of the fully integrated system. This
early demonstrator has also been documented in a short video, which is available for download
in AVI format (approx. 150 MBytes).
Figure 5: User picks item from shelf
Using RFID tags the system is able to detect what product the user has taken from the shelf. SPECTER stores
all observed interactions in an extended episodic memory, the personal journal.
Figure 6: SPECTER retrieves related offers and recommends alternative product
Depending on what the user chose, SPECTER can provide assistance. An acoustic signal is used to draw the user's
attention to new information on the handheld display. In the current context, SPECTER automatically queried the
shopping service for product information. In this case, SPECTER recommends an alternative product of the same category
that provides more features and can also be found in this store. All interactions between user and SPECTER itself
will also be added to the journal. For example, an immediate confirmation provides valuable information as SPECTER
is also able to learn from user feedback to further improve its decision making.
Figure 7: User puts item back and takes other product
The user follows the recommendation and puts the item back. This action can also be recognized.
Figure 8: User puts item into basket
The record of the user's actions and affective states in the personal journal can be mined to learn a model
of the user's preferences, habits, and typical affective reactions. SPECTER exploits the user model and the personal
journal to provide appropriate recommendations and assistance in the user's current situation. Even while walking
around, the user can have immediate access to his updated personal journal for further reflection and introspection.
Figure 9: Interactions can be Viewed within the Personal Journal
Also on the PC at home, the user can explore and modify the personal journal as well as the user model in order
to assist Specter in its learning process. Journal entries can be annotated to provide explicit ratings for the
corresponding low-level events or high-level interpretations. Such subjective evaluations offer an added value
for affect-aware personal assistance. All modifications provided by the user during the reflection and introspection
phase will be exploited for the automatic update of the persistent user model.