PNNL

Abstract

The influence that language has on contextual interpretations cannot be ignored by computer systems that strive to be context-aware. Rather, once systems are designed to perceive language and other forms of human action, these interpretative processes will of necessity be context-dependent. As an example, we illustrate how people simply and naturally create new contexts by naming and referring. We then describe Rasa, a mixed-reality system that observes and understands how users in a military command post create such contexts as part of the process of maintaining situational awareness. In such environments, commander?s maps are covered with Post-it notes. These paper artifacts are contextually transformed to represent units in the field by the application of multimodal language. Rasa understands this language, thereby allowing paper-based tools to become the basis for digital interaction. Finally, we argue that architectures for such context-aware systems will need to be built to process the inherent ambiguity and uncertainty of human input in order to be effective.