Augmented reality technologies, systems and applications
This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and problems encountered when building some Augmented Reality applications. It surveys mobile augmented reality systems challenges and requirements for successful mobile systems. This paper summarizes the current applications of Augmented Reality and speculates on future applications and where current research will lead Augmented Reality’s development. Challenges augmented reality is facing in each of these applications to go from the laboratories to the industry, as well as the future challenges we can forecast are also discussed in this paper. Section 1 gives an introduction to what Augmented Reality is and the motivations for developing this technology. Section 2 discusses Augmented Reality Technologies with computer vision methods, AR devices, interfaces and systems, and visualization tools. The mobile and wireless systems for Augmented Reality are discussed in Section 3. Four classes of current applications that have been explored are described in Section 4. These applications were chosen as they are the most famous type of applications encountered when researching AR apps. The future of augmented reality and the challenges they will be facing are discussed in Section 5.
KeywordsAugmented reality Augmented reality technologies Augmented reality systems Augmented reality applications Augmented reality iphone4 AR Augmented reality on mobile devices
This material is based upon work supported by the National Science Foundation under Grant No. OISE-0730065.
- 1.Ababsa F, Mallem M (2008) Robust camera pose estimation combining 2D/3D points and lines tracking. Industrial Electronics, 2008. ISIE 2008. IEEE International Symposium on, vol., no., pp.774–779, June 30 2008-July 2 doi: 10.1109/ISIE.2008.4676964 URL:http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4676964&isnumber=4676877
- 2.Akinbiyi T, Reiley CE, Saha S, Burschka D, Hasser CJ, Yuh DD, Okamura AM (2006) Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. Engineering in Medicine and Biology Society, 2006. EMBS ‘06. 28th Annual International Conference of the IEEE, vol., no., pp.567–570, Aug. 30 2006-Sept. 3Google Scholar
- 3.Arvanitis TN, Petrou A, Knight JF, Savas S, Sotiriou S, Gargalakos M, Gialouri E (2009) Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal and Ubiquitous Computing 13(3):243–250CrossRefGoogle Scholar
- 4.Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent Advances in Augmented Reality. IEEE, November/DecemberGoogle Scholar
- 5.Babak A. Parviz (2009) Augmented reality in a contact lens (http://spectrum.ieee.org/biomedical/bionics/augmented-reality-in-a-contact-lens/0) IEEE Spectrum, September
- 6.Barakonyi I, Fahmy T, Schmalstieg D (2004) Remote collaboration using Augmented Reality Videoconferencing. Proceedings of Graphics Interface. p.89–96, May 17–19, 2004, London, Ontario, CanadaGoogle Scholar
- 8.Beyond Reality, http://www.augmented-reality-games.com
- 9.Bichlmeier C, Wimmer F, Heining SM, Navab N (2007) Contextual anatomic mimesis: hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. IEEE Google Scholar
- 10.Bimber O, Raskar R, Inami M (2007) Spatial Augmented Reality. SIGGRAPH 2007 Course 17 NotesGoogle Scholar
- 11.Brown D, Julier S, Baillot Y, Livingston MA (2003) An event-based data distribution mechanism for collaborative mobile augmented reality and virtual environments. Virtual Reality, 2003. Proceedings IEEE, vol., no., pp. 23–29, 22–26 MarchGoogle Scholar
- 13.Caruso G, Re GM (2010) AR-Mote: A wireless device for Augmented Reality environment. 3D User Interfaces (3DUI), 2010 IEEE Symposium on. vol., no., pp.99–102, 20–21 MarchGoogle Scholar
- 14.Chen IYH, MacDonald B, Wünsche B (2008) Markerless augmented reality for robotic helicoptor applications. Lecture Notes In Computer Science, Proceedings of the 2nd international conference on Robot vision, pp 125–138Google Scholar
- 15.Compute Scotland, http://www.computescotland.com/optical-ingenuity-from-fraunhofer-ipms-2321.php, Optical ingenuity from Fraunhofer IPMS, May 8th, 2009.
- 16.Contactum, Augmented Reality Solutions (2006) http://www.augmented-reality.com/, Where reality meets the virtual world
- 17.Cooper N, Keatley A, Dahlquist M, Mann S, Slay H, Zucco J, Smith R, Thomas BH (2004) Augmented reality chinese checkers. The Australasian Computing Education Conference; Vol. 74, Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology, Singapore, pp 117–126Google Scholar
- 18.Costanza E, Inverso SA, Pavlov E, Allen R, Maes P (2006) eye-q: Eyeglass Peripheral Display for Subtle Intimate Notifications. Mobile HCI 2006, September 13–15, Google Scholar
- 19.Dahne P, Karigiannis JN (2002) Archeoguide: system architecture of a mobile outdoor augmented reality system. Mixed and Augmented Reality, 2002. ISMAR 2002. Proceedings. International Symposium on, vol., no., pp. 263–264, 2002Google Scholar
- 20.Merrill D, Maes P (2007) Augmenting looking, pointing and reaching gestures to enhance the searching and browsing of physical objects. Proceedings of the 5th International Conference on Pervasive Computing (Pervasive ‘07), Toronto, Canada, May 13–16Google Scholar
- 21.DiVerdi S, Hollerer T (2007) GroundCam: A Tracking Modality for Mobile Mixed Reality. Virtual Reality Conference, 2007. VR ‘07. IEEE, vol., no., pp.75–82, 10–14 MarchGoogle Scholar
- 22.Dr. Giuseppe De Pietro and Dr. Luigi Gallo of ICAR-CNR group, National Research Council, ItalyGoogle Scholar
- 23.Encyclopidia, http://encyclopedia.jrank.org/articles/pages/6843/Multimodal-Interfaces.html, Multimodal Interfaces.
- 24.Feldman A, Tapia EM, Sadi S, Maes P, Schmandt C (2005) ReachMedia: On-the-move interaction with everyday objects. iswc, pp.52–59, Ninth IEEE International Symposium on Wearable Computers (ISWC‘05), 2005Google Scholar
- 26.Geekology (2008) http://www.geekologie.com/2008/12/14-week, Cool: Augmented Reality Advertisements, Dec. 19
- 27.Github, Social Coding (2009) http://github.com/zac/iphonearkit/, iPhone ARKit
- 28.Google, http://www.google.com/mobile/goggles/#text, Google Goggles, 2010
- 29.Handheld Augmented Reality (2010) http://studierstube.icg.tu-graz.ac.at/handheld_ar/stbtracker.php, Studierstube Tracker
- 30.Horizon Report Johnson LF, Smith, RS (2005) Horizon Report. Austin, TX: The New Media Consortium, 2005.Google Scholar
- 31.https://www.icg.tugraz.at/~daniel/HistoryOfMobileAR/ (2009) History of mobile augmented reality
- 32.Huang Y, Liu Y, Wang Y (2009) AR-View: and Augmented Reality Device for Digital Reconstruction of Yuangmingyuan”, IEEE International Symposium on Mixed and Augmented Reality Google Scholar
- 33.iPhone OS Reference Library, http://developer.apple.com/iphone/library/documentation/MapKit/Reference/MapKit_Framework_Reference/index.html, Map Kit Framework Reference, 2010.
- 34.iPhoneNess, http://www.iphoneness.com/iphone-apps/best-augmented-reality-iphone-applications/, 26 Best Augmented Reality iPhone Applications.
- 35.Juan MC, Botella C, Alcaniz M, Banos R, Carrion C, Melero M, Lozano JA (2004) An augmented reality system for treating psychological disorders: application to phobia to cockroaches. Mixed and Augmented Reality, 2004. ISMAR 2004. Third IEEE and ACM International Symposium on, vol., no., pp. 256–257, 2–5 NovGoogle Scholar
- 36.Kato H, Billinghurst M, Poupyrev I, Imamoto K, Tachibana K (2000) Virtual object manipulation on a table-top AR environment. ISAR ’00, 111–119Google Scholar
- 37.Lee B, Chun J (2010) Interactive manipulation of augmented objects in marker-less AR using vision-based hand interaction. itng, pp.398–403, 2010 Seventh International Conference on Information TechnologyGoogle Scholar
- 38.Lee J-Y, Lee S-H, Park H-M, Lee S-K, Choi J-S, Kwon J-S (2010) Design and implementation of a wearable AR annotation system using gaze interaction. Consumer Electronics (ICCE), 2010 Digest of Technical Papers International Conference on. vol., no., pp.185–186, 9–13 JanGoogle Scholar
- 39.Lepetit V (2008) On computer vision for augmented reality. Ubiquitous Virtual Reality, 2008. ISUVR 2008. International Symposium on, vol., no., pp.13–16, 10–13 JulyGoogle Scholar
- 40.Li Y-bo, Kang S-p, Qiao Z-h, Zhu Q (2008) Development actuality and application of registration technology in augmented reality. Computational Intelligence and Design, 2008. ISCID ‘08. International Symposium on. Vol.2, No., pp.69–74, 17–18 Oct. 2008Google Scholar
- 41.Luo X, Kline T, Fischer HC, Stubblefield KA, Kenyon RV, Kamper DG (2005) Integration of augmented reality and assistive devices for post-stroke hand opening rehabilitation. Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the, vol., no., pp.6855–6858Google Scholar
- 42.Malaka R, Schneider K, Kretschmer U (2004) Stage-based augmented edutainment. Smart Graphics, pp.54–65Google Scholar
- 43.Marco Sacco, Stefano Mottura, Luca Greci, Giampaolo Viganņ, Insitute of Industrial Technologies and Automation, National Research Council, Italy.Google Scholar
- 44.Mashable, The Social Media Guide (2009) http://mashable.com/2009/12/05/augmented-reality-iphone/, 10 amazing Augmented Reality iPhone Apps
- 45.Sixth Sense—a wearable gestural interface (MIT Media Lab) (2010) http://www.pranavmistry.com/projects/sixthsense/, Sixth Sense—integrating information with the real world, Pranav Mistry, Fluid Interface Group, MIT Media Lab
- 46.Milgram P, Kishino AF (1994) Taxonomy of mixed reality visual displays (http://vered.rose.utoronto.ca/people/paul_dir/IEICE94/ieice.html). IEICE Transactions on Information Systems E77-D(12):1321–1329Google Scholar
- 47.MINI, www.mini.com
- 48.Mistry P, Kuroki T, Chuang C (2008) TaPuMa: Tangible Public Map for Information Acquirement through the Things We Carry. MIT Media Lab, Ambi-sys‘08, FebruaryGoogle Scholar
- 49.Mistry P, Maes P, Chang L (2009) WUW – Wear Ur World – A Wearable Gestural Interface”, ACM, CHI 2009, Boston, April 4-9Google Scholar
- 50.Miyashita T, Meier P, Tachikawa T, Orlic S, Eble T, Scholz V, Gapel A, Gerl O, Arnaudov S, Lieberknecht S (2008) An augmented reality museum guide. Mixed and Augmented Reality, 2008. ISMAR 2008. 7th IEEE/ACM International Symposium on, vol., no., pp.103–106, 15–18 SeptGoogle Scholar
- 51.Nilsson J, Odblom ACE, Fredriksson J, Zafar A, Ahmed F (2010) Performance evaluation method for mobile computer vision systems using augmented reality. IEEE Virtual RealityGoogle Scholar
- 53.Read Write Web, http://www.readwriteweb.com/archives/how_iphone_4_could_change_augmented_reality.php, How iPhone 4 Could Change Augmented Reality, Chris Cameron, June 10, 2010
- 54.Reitmayr G, Schmalstieg D (2003) Location based applications for mobile augmented reality. AUIC2003Google Scholar
- 55.Sandor C, Olwal A, Bell B, Feiner S (2005) Immersive mixed-reality configuration of hybrid user interfaces. In ISMAR ‘05, pp. 110–113Google Scholar
- 56.Schmalstieg D, Fuhrmann A, Hesina G (2000) bridging multiple user interface dimensions with augmented reality. IEEEGoogle Scholar
- 57.Schmalstieg D, Fuhrmann A, Hesina G, Zsalavari Z, Encarnacao LM, Gervautz M, Purgathofer W (2002) The studierstube augmented reality project. Massachusetts Institute of Technology 11(1):33–54Google Scholar
- 58.SlideShare, Present Yourself, http://www.slideshare.net/OmarCaf/augmented-reality-on-iphone-applications, Augmented Reality on iPhone Applications, Paolo Quadrani, Omar Caffini, 2010.
- 59.Stutzman B, Nilsen D, Broderick T, Neubert J (2009) MARTI: Mobile Augmented Reality Tool for Industry. Computer Science and Information Engineering, 2009 WRI World Congress on, vol.5, no., pp.425–429, March 31 2009-April 2Google Scholar
- 60.Takahashi H, Shimazaki S, Kawashima T (2007) Augmented reality system for development of handy information device with tangible interface”, In: Shumaker (ed.) Virtual Reality, HCII2007. LNCS, vol. 4563, pp. 564–573. Springer, BerlinGoogle Scholar
- 61.Wagner D, Schmalstieg D handheld augmented reality displays. Graz University of Technology, AustriaGoogle Scholar
- 62.Wang PP, Wang T, Ding D, Zhang Y, Bi W, Bao Y (2009) Mirror world navigation for mobile users based on augmented reality. International Multimedia Conference, Proceedings of the seventeen ACM international conference on Multimedia, pp 1025–1026Google Scholar
- 63.White S, Lister L, Feiner S (2007) Visual hints for tangible gestures in augmented reality. In ISMAR ‘07, 47–50Google Scholar
- 64.Wikipedia, the free encyclopedia (2009) http://en.wikipedia.org/wiki/Sensorama, Sensorama
- 65.Wikipedia, the free encyclopedia (2010) http://en.wikipedia.org/wiki/Augmented_reality, Augmented Reality,
- 66.Wikipedia, the free encyclopedia (2010) http://en.wikipedia.org/wiki/Sensoramahttp://en.wikipedia.org/wiki/ARToolKit, ARToolKit
- 67.Wikipedia, the free encyclopedia, http://en.wikipedia.org/wiki/Invasiveness_of_surgical_procedures, Invasiveness of surgical procedures, 2010
- 68.Wikipedia, the free encyclopedia, http://en.wikipedia.org/wiki/Robotic_surgery, Robotic surgery, 2010
- 69.Wikitude, http://www.wikitude.org/dewikitude-drive-eyes-road-againenwikitude-drive-eyes-road, WikitudeDrive: Never take your eyes off the road again, 2009.
- 70.Wired, http://www.wired.com/dangerroom/2008/03/darpa-wants-con/, Pentagon: ‘Augment’ Reality with ‘Videogame’ Contact Lenses (Updated), Noah Shachtman, March 20th, 2008.
- 71.Zhou F, Duh HBL, Billinghurst M (2008) Trends in augmented reality tracking, interaction and display: a review of ten years of ISMARGoogle Scholar