Information-level AR instruction: a novel assembly guidance information representation assisting user cognition

  • Zhuo Wang
  • Xiaoliang BaiEmail author
  • Shusheng ZhangEmail author
  • Weiping He
  • Xiangyu Zhang
  • Li Zhang
  • Peng Wang
  • Dechuan Han
  • Yuxiang Yan


Recently, there are some problems in AR instruction research in assembly field, such as irregular instruction design form and redundant display of instruction content. The reason is that there is no AR instruction design rule for AR assembly. The goal of this rule should be to maximize users’ cognitive efficiency of guided content. In order to solve this problem, this paper carries out relevant research work. Firstly, the definition of AR instruction at geometric level and information level is given, and the corresponding design rules of AR instruction, namely, geometric level visualization (GLV) and information-level visualization (ILV), are proposed under these two definitions. Then, a data processing model oriented to the above rules is established, and the relationship and difference between them are elaborated. And then, according to GLV and ILV, four visual interfaces are designed to guide AR assembly. A case study was designed to test the performance of the four interfaces under the two specifications in terms of assembly time, operation experience (including enjoyment, concentration, self-confidence, natural intuition, feasibility, effectiveness, availability, and comprehensibility). Finally, through the test results of each interface, the influencing factors of ILV on user’s assembly efficiency, cognitive efficiency, and understanding are determined, and three implications of ILV are summarized. The results show that ILV with MBD design elements can improve user’s operating experience better than GLV.


User cognition Augmented reality Assembly Precision Visualization 


Supplementary material

170_2019_4538_MOESM1_ESM.mp4 (95.6 mb)
ESM 1 (MP4 97849 kb)


  1. 1.
    Tao F, Cheng Y, Zhang L, Nee AY (2017) Advanced manufacturing systems: socialization characteristics and trends. J Intell Manuf 28:1079–1094CrossRefGoogle Scholar
  2. 2.
    Lee J, Bagheri B, Kao H-A (2015) A cyber-physical systems architecture for industry 4.0-based manufacturing systems. Manuf Lett 3:18–23CrossRefGoogle Scholar
  3. 3.
    Mourtzis D, Zogopoulos V (2019) Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling. Int J Adv Manuf Technol:1–12Google Scholar
  4. 4.
    Leitao P, Karnouskos S, Ribeiro L, Lee J, Strasser T, Colombo AW (2016) Smart agents in industrial cyber–physical systems. Proc IEEE 104:1086–1101CrossRefGoogle Scholar
  5. 5.
    Tan Q, Tong Y, Wu S, Li D (2019) Modeling, planning, and scheduling of shop-floor assembly process with dynamic cyber-physical interactions: a case study for CPS-based smart industrial robot production. Int J Adv Manuf Technol:1–11Google Scholar
  6. 6.
    Alemanni M, Destefanis F, Vezzetti E (2011) Model-based definition design in the product lifecycle management scenario. Int J Adv Manuf Technol 52:1–14CrossRefGoogle Scholar
  7. 7.
    Andre P, Sorito R (2002) Product Manufacturing Information (PMI) in 3D models: a basis for collaborative engineering in Product Creation Process (PCP), in Simulation in industry, 14 th European Simulation Symposim, pp 348–352Google Scholar
  8. 8.
    Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in augmented reality. Naval Research Lab, Washington DCCrossRefGoogle Scholar
  9. 9.
    Nee AY, Ong S, Chryssolouris G, Mourtzis D (2012) Augmented reality applications in design and manufacturing. CIRP Ann 61:657–679CrossRefGoogle Scholar
  10. 10.
    Jiang S, Ong S, Nee A (2014) An AR-based hybrid approach for facility layout planning and evaluation for existing shop floors. Int J Adv Manuf Technol 72:457–473CrossRefGoogle Scholar
  11. 11.
    Bruno F, Barbieri L, Marino E, Muzzupappa M, D’Oriano L, Colacino B (2019) An augmented reality tool to detect and annotate design variations in an Industry 4.0 approach. Int J Adv Manuf Technol:1–13Google Scholar
  12. 12.
    Qiu S, Jing X, Fan X, He Q (2011) Using AR technology for automotive visibility and accessibility assessment. In: 2011 International Conference on Virtual Reality and Visualization, pp 217–224Google Scholar
  13. 13.
    Wang Y, Zhang S, Wan B, He W, Bai X (2018) Point cloud and visual feature-based tracking method for an augmented reality-aided mechanical assembly system. Int J Adv Manuf Technol 99:2341–2352CrossRefGoogle Scholar
  14. 14.
    Wang X, Ong SK, Nee AY (2016) A comprehensive survey of augmented reality assembly research. Adv Manuf 4:1–22CrossRefGoogle Scholar
  15. 15.
    Caudell TP, Mizell DW (1992) Augmented reality: an application of heads-up display technology to manual manufacturing processes. In: System Sciences, 1992. Proceedings of the Twenty-Fifth Hawaii International Conference on, pp 659–669Google Scholar
  16. 16.
    Neumann U, Majoros A (1998) Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In: Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998, pp 4–11Google Scholar
  17. 17.
    Wiedenmaier S, Oehme O, Schmidt L, Luczak H (2003) Augmented reality (AR) for assembly processes design and experimental evaluation. Int J Hum Comput Interact 16:497–514CrossRefGoogle Scholar
  18. 18.
    Schwald B, De Laval B (2003) An augmented reality system for training and assistance to maintenance in the industrial contextGoogle Scholar
  19. 19.
    Yuan M, Ong S, Nee A (2008) Augmented reality for assembly guidance using a virtual interactive tool. Int J Prod Res 46:1745–1767CrossRefGoogle Scholar
  20. 20.
    Zhang J, Ong S, Nee A (2011) RFID-assisted assembly guidance system in an augmented reality environment. Int J Prod Res 49:3919–3938CrossRefGoogle Scholar
  21. 21.
    Hou L, Wang X, Bernold L, Love PE (2013) Using animated augmented reality to cognitively guide assembly. J Comput Civ Eng 27:439–451CrossRefGoogle Scholar
  22. 22.
    Funk M, Bächler A, Bächler L, Korn O, Krieger C, Heidenreich T, et al (2015) Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Acm International Conference on Pervasive Technologies Related to Assistive EnvironmentsGoogle Scholar
  23. 23.
    Oda O, Elvezio C, Sukan M, Feiner S, Tversky B (2015) Virtual replicas for remote assistance in virtual and augmented reality. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, pp 405–415Google Scholar
  24. 24.
    Wang Z, Ong S, Nee A (2013) Augmented reality aided interactive manual assembly design. Int J Adv Manuf Technol 69:1311–1321CrossRefGoogle Scholar
  25. 25.
    Fiorentino M, Monno G, Uva A (2009) Tangible digital master for product lifecycle management in augmented reality. Int J Interact Des Manuf 3:121–129CrossRefGoogle Scholar
  26. 26.
    Mizell D (2001) Boeing’s wire bundle assembly project, Fundamentals of wearable computers and augmented reality, vol 5Google Scholar
  27. 27.
    Uva AE, Cristiano S, Fiorentino M, Monno G (2010) Distributed design review using tangible augmented technical drawings. Comput Aided Des 42:364–372CrossRefGoogle Scholar
  28. 28.
    Huang J, Ong S-K, Nee AY (2017) Visualization and interaction of finite element analysis in augmented reality. Comput Aided Des 84:1–14CrossRefGoogle Scholar
  29. 29.
    Henderson S, Feiner S (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Vis Comput Graph 17:1355–1368CrossRefGoogle Scholar
  30. 30.
    Feiner S, Macintyre B, Seligmann D (1993) Knowledge-based augmented reality. Commun ACM 36:52–63CrossRefGoogle Scholar
  31. 31.
    Schwald, Bernd, et al. "STARMATE: Using Augmented Reality technology for computer guided maintenance of complex mechanical elements." E-work and ECommerce 1 (2001): 196-202.Google Scholar
  32. 32.
    Stork S, Schubö A (2010) Human cognition in manual assembly: theories and applications. Adv Eng Inform 24:320–328CrossRefGoogle Scholar
  33. 33.
    Uva AE, Gattullo M, Manghisi VM, Spagnulo D, Cascella GL, Fiorentino M (2018) Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations. Int J Adv Manuf Technol 94:509–521CrossRefGoogle Scholar
  34. 34.
    Billinghurst M, Hakkarainen M, Woodward C (2008) Augmented assembly using a mobile phone. In: Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia, pp 84–87Google Scholar
  35. 35.
    Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M et al (2019) 2.5 DHANDS: a gesture-based MR remote collaborative platform. Int J Adv Manuf Technol 102:1339–1353CrossRefGoogle Scholar
  36. 36.
    Webel S, Bockholt U, Engelke T, Gavish N, Olbrich M, Preusche C (2013) An augmented reality training platform for assembly and maintenance skills. Robot Auton Syst 61:398–403CrossRefGoogle Scholar
  37. 37.
    Doshi A, Smith RT, Thomas BH, Bouras C (2017) Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int J Adv Manuf Technol 89:1279–1293CrossRefGoogle Scholar
  38. 38.
    Henderson SJ, Feiner SK (2011) Augmented reality in the psychomotor phase of a procedural task, in 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp 191–200Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Zhuo Wang
    • 1
    • 2
  • Xiaoliang Bai
    • 1
    • 2
    Email author
  • Shusheng Zhang
    • 1
    Email author
  • Weiping He
    • 1
    • 2
  • Xiangyu Zhang
    • 1
    • 2
  • Li Zhang
    • 1
    • 2
  • Peng Wang
    • 1
    • 2
  • Dechuan Han
    • 1
    • 2
  • Yuxiang Yan
    • 1
    • 2
  1. 1.Key Laboratory of Contemporary Designing and Integrated Manufacturing Technology, Chinese Ministry of EducationNorthwestern Polytechnical UniversityXi’anChina
  2. 2.Cyber-physical Interaction Laboratory, College of Mechanical and Electrical EngineeringNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations