Abstract
DyPERS, ‘Dynamic Personal Enhanced Reality System’, uses augmented reality and computer vision to autonomously retrieve ‘media memories’ based on associations with real objects the user encounters. These are evoked as audio and video clips relevant for the user and overlayed on top of real objects the user encounters. The system utilizes an adaptive, audio-visual learning system on a tetherless wearable computer. The user’s visual and auditory scene is stored in real-time by the system (upon request) and is then associated (by user input) with a snap shot of a visual object. The object acts as a key such that when the real-time vision system detects its presence in the scene again, DyPERS plays back the appropriate audio-visual sequence. The vision system is a probabilistic algorithm which is capable of discriminating between hundreds of everyday objects under varying viewing conditions (view changes, lighting, etc.). Once an audio-visual clip is stored, the vision system automatically recalls it and plays it back when it detects the object that the user wished to use to remind him of the sequence. The DyPERS interface augments the user without encumbering him and effectively mimics a form of audio-visual memory. First results on performance and usability are shown.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Y. Aloimonos. Purposive and qualitative active vision. In Image Understanding Workshop, pages 816–828, 1990.
R. Bajcsy. Active perception vs. pasive perception. In IEEE Workshop on Computer Vision, pages 55–59, 1985.
R. Bajcsy. Active perception. Proceedings of the IEEE, 296:996–1005, 1988.
D. Ballard. Animate vision. Aritifcal Intelligence, 48:57–86, 1991.
B.B. Bederson. Audio augmented reality: A prototype automated tour guide. In ACM SIGCHI, pages 210–211, 1995.
E.D. Dickmanns. Vehicles capaple of dynamic vision. In 15th International Joint Conference in Artificial Intelligence, 1997.
S. Feiner, B. MacIntyre, and D. Seligmann. Annotating the real world with knowledge-based graphics on see-through head-mounted display. In Proc. of Graphics Interface, pages 78–85, 1992.
[JEW+97]_T. Jebara, C. Eyster, J. Weaver, T. Starner, and A. Pentland. Stocha-sticks: Augmenting the billiards experience with probabilistic vision and wearable computers. In Intl. Symp. on Wearable Computers, 1997.
R.J.K. Jacob, J.J. Leggett, B.A. Myers, and R. Pausch. Interaction styles and input/output devices. Behaviour and Information Technology, 12(2):69–70, 1993.
S. Kakez, C. Vania, and P. Bisson. Virtually documented environment. In Intl. Symp. on Wearable Computers, 1997.
J. Levine. Real-time target and pose recognition for 3-d graphical overlay. Master’s thesis, EECS Dept., MIT, June 1997.
M. Lamming and Flynn. Forget-me-not: intimate computing in support of human memory. In Proceedings of FRIEND21 Intl. Symp. on Next Generation Human Interface, 1993.
S. Mann. Wearable, tetherless computer-mediated reality. Technical Report 361, M.I.T. Media Lab, 1996.
S. Mann. Wearable computing: A first step toward personal imaging. IEEE Computer; http://wearcam.org/ieeecomputer.htm, 30(2), February 1997.
D. Marr. Vision. W.H. Freeman and Company, 1982.
E.D. Mynatt, M. Back, R. Want, and R. Frederik. Audio aura: Light weight audio augmented reality. In UIST, pages 211–212, 1997.
K. Nagao and J. Rekimoto. Agent augmented reality: a software agent meets the real world. In Proc. of Intl. Conf. on Multiagent Sys., 1996.
J. Rekimoto and K. Nagao. The world through the computer: computer augmented interaction with real world environments. UIST, pages 29–36, 1995.
B. Rhodes and T. Starner. Remembrance agent: a continuously running automated information retrieval system. In Intl. Conf. on the Practical Application of Intelligent Agents and Multi Agent Technology, pages 487–495, 1996.
B. Schiele and J.L. Crowley. Probabilistic object recognition using multidimensional receptive field histograms. In 13th Intl. Conf. on Pattern Recognition, Volume B, pages 50–54, August1996.
B. Schiele. Object Recognition using Multidimensional Receptive Field Histograms. PhD thesis, I.N.P.Grenoble, July 1997. English translation.
[SMR+97]_T. Starner, S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R.W. Picard, and A.P. Pentland. Augmented reality through wearable computing. Presence, Special Issue on Augmented Reality, 1997.
M. Turk, editor. Perceptual User Interfaces Workshop Proceedings, 1997.
M. Weiser. The computer of the twenty-first century. Scientific American, 1991.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schiele, B., Oliver, N., Jebara, T., Pentland, A. (1999). An Interactive Computer Vision System DyPERS: Dynamic Personal Enhanced Reality System. In: Computer Vision Systems. ICVS 1999. Lecture Notes in Computer Science, vol 1542. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49256-9_4
Download citation
DOI: https://doi.org/10.1007/3-540-49256-9_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65459-9
Online ISBN: 978-3-540-49256-6
eBook Packages: Springer Book Archive