An Augmented Reality Framework for Supporting and Monitoring Operators during Maintenance Tasks

  • Guido Maria Re
  • Monica Bordegoni
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8526)


The paper proposes a framework for supporting maintenance services in industrial environments through the use of a mobile device and Augmented Reality (AR) technologies. 3D visual instructions about the task to carry out are represented in the real world by means of AR and they are visible through the mobile device. In addition to the solutions proposed so far, the framework introduces the possibility to monitor the operator’s work from a remote location. The mobile device stores information for each maintenance step that has been completed and it makes them available on a remote database. Supervisors can consequently check the maintenance activity from a remote PC at any time. The paper presents also a prototype system, developed according to the framework, and an initial case study in the field of food industry.


Augmented Reality Framework Maintenance tasks Remote Supervision 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. E77-D (1994)Google Scholar
  2. 2.
    Azuma, R.: A survey of augmented reality. Presence 6, 355–385 (1997)Google Scholar
  3. 3.
    Fite-Georgel, P.: Is there a reality in Industrial Augmented Reality? In: 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 201–210. IEEE (2011)Google Scholar
  4. 4.
    Davies, P., Sivich, L.: Augmented Reality and other Visualization Technologies for Manufacturing in Boeing. SAE Int. J. Aerosp. 4(2), 1133–1139 (2011)Google Scholar
  5. 5.
    Tang, A., Owen, C., Biocca, F., Mou, W.: Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI), pp. 73–80. ACM Press, New York (2003)CrossRefGoogle Scholar
  6. 6.
    Henderson, S.J., Feiner, S.: Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans. Vis. Comput. Graph. 17, 1355–1368 (2011)CrossRefGoogle Scholar
  7. 7.
    Feiner, S., Macintyre, B., Seligmann, D.: Knowledge-based augmented reality. Commun. ACM. 36, 53–62 (1993)CrossRefGoogle Scholar
  8. 8.
    Reiners, D., Stricker, D., Klinker, G., Stefan, M.: Augmented Reality for Construction Tasks: Doorlock Assembly. In: IEEE and ACM International Workshop on Augmented Reality (IWAR), pp. 31–46 (1998)Google Scholar
  9. 9.
    Henderson, S.J., Feiner, S.: Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In: 2009 8th IEEE International Symposium on Mixed and Augmented Reality, pp. 135–144. IEEE (2009)Google Scholar
  10. 10.
    Zhu, J., Ong, S.K., Nee, A.Y.C.: An authorable context-aware augmented reality system to assist the maintenance technicians. Int. J. Adv. Manuf. Technol., 1699–1714 (2012)Google Scholar
  11. 11.
    Ong, S.K., Yuan, M.L., Nee, A.Y.C.: Augmented reality applications in manufacturing: A survey. Int. J. Prod. Res. 46, 2707–2742 (2008)CrossRefzbMATHGoogle Scholar
  12. 12.
    Didier, J.-Y., Roussel, D., Mallem, M., Otmane, S., Naudet, S., Pham, Q.-C., Bourgeois, S., Mégard, C., Leroux, C., Hocquard, A.: AMRA: Augmented Reality assistance in train maintenance tasks. In: 4th ACM/IEEE Int. Symp. Mix. Augment. Real. - Work. Ind. Augment. Real., pp. 1–10 (2005)Google Scholar
  13. 13.
    Billinghurst, M., Hakkarainen, M., Woodward, C.: Augmented assembly using a mobile phone. In: Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia (MUM), pp. 84–87. IEEE (2008)Google Scholar
  14. 14.
    Klinker, G., Creighton, O., Dutoit, A.H., Kobylinski, R., Vilsmeier, C., Brugge, B.: Augmented maintenance of powerplants: a prototyping case study of a mobile AR system. In: Proceedings IEEE and ACM International Symposium on Augmented Reality, pp. 124–133. IEEE Comput. Soc. (2001)Google Scholar
  15. 15.
    Ishii, H., Shimoda, H., Nakai, T., Izumi, M., Bian, Z., Morishita, Y.: Proposal and Evaluation of a Supporting Method for NPP Decommissioning Work by Augmented Reality. In: 12th World Multi-Conference on Systemics, Cybernetics, pp. 157–162 (2008)Google Scholar
  16. 16.
    Goose, S., Güven, S., Zhang, X., Sudarsky, S., Navab, N.: Paris: Fusing vision-based location tracking with standards-based 3d visualization and speech interaction on a PDA. In: International Conference on Distributed Multimedia Systems, pp. 75–80 (2004)Google Scholar
  17. 17.
    Boulanger, P.: Application of augmented reality to industrial tele-training. In: Proceedings of First Canadian Conference on Computer and Robot Vision, pp. 320–328. IEEE (2004)Google Scholar
  18. 18.
    Reitmayr, G., Eade, E., Drummond, T.W.: Semi-automatic Annotations in Unknown Environments. In: 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–4 (2007)Google Scholar
  19. 19.
    Fite-Georgel, P., Schroeder, P., Navab, N.: Navigation Tools for Viewing Augmented CAD Models. IEEE Comput. Graph. Appl. 29, 65–73 (2009)CrossRefGoogle Scholar
  20. 20.
    Wagner, D., Schmalstieg, D.: Artoolkitplus for pose tracking on mobile devices. In: Proceedings of 12th Computer Vision Winter Workshop (CVWW), pp. 139–146 (2007)Google Scholar
  21. 21.
    Agrawala, M., Phan, D., Heiser, J., Haymaker, J., Klingner, J., Hanrahan, P., Tversky, B.: Designing effective step-by-step assembly instructions. In: ACM SIGGRAPH 2003 Papers on - SIGGRAPH 2003, p. 828. ACM Press, New York (2003)Google Scholar
  22. 22.
    Platonov, J., Heibel, H., Meier, P., Grollmann, B.: A mobile markerless AR system for maintenance and repair. In: Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), pp. 105–108 (2006)Google Scholar
  23. 23.
    De Crescenzio, F., Fantini, M., Persiani, F., Di Stefano, L., Azzari, P., Salti, S.: Augmented Reality for Aircraft Maintenance Training and Operations Support. IEEE Comput. Graph. Appl. 31, 96–101 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Guido Maria Re
    • 1
  • Monica Bordegoni
    • 1
  1. 1.Dipartimento di MeccanicaPolitecnico di MilanoMilanoItaly

Personalised recommendations