Referencing Patterns in Collaborative Augmented Reality

Abstract

As application domains emerge, they often bring with them a variety of new interaction techniques. Augmented Reality (AR) is a technology that attempts to seamlessly integrate virtual artifacts into the physical world. Though many of the interaction challenges in AR parallel those found in purely virtual environments, the inclusion of the physical world forces us consider new scenarios. This chapter will discuss some of the user interface (UI) challenges in collaborative AR environments – or those with more than one participant. For these environments to be successful, it is critical that they support referential awareness. The reader may question why this is an important item to consider when designing such systems. After all, people collaborate fairly well in the physical world through gestures, such as pointing, and through deictic references (e.g. “this one” or “those”). One might therefore believe that re-integrating the physical world back into the digital one would make referencing easier. However, consider a multi-user environment that contains hundreds of artifacts that are similar in appearance – with participants geographically separated; one can see that things become complex quickly. Or, consider how one might reference virtual objects that are embedded within physical ones, such as during augmented surgery. References may be made at arbitrary depths, such as physically to the patient’s skin or virtually to the volumetric data (e.g. a CAT scan) embedded within.