PNNL

Abstract

Invisible computing systems are highly context-dependent. Consequently, the influence that language has on contextual interpretation cannot be ignored by such systems. Rather, once language and other forms of human action are perceived by a system, its interpretative processes will of necessity be context-dependent. As an example, we illustrate how people simply and naturally create new contexts for interpretation by creating new names and referring expressions. We then describe Rasa, a mixed-reality system that invisibly observes and understands how users in a military command post create such contexts as part of the process of maintaining situational awareness. Rasa augments both the commander?s map and the Post-it notes pasted on it, which represent units in the field, with multimodal language, thereby allowing paper-based tools to interact with digital information. Finally, we argue that architectures for such context-aware systems must reduce the inherent ambiguity and uncertainty through fusion and other means.