Simulation data integration-based approach for motion synthesis in virtual maintenance

  • Jie Geng
  • Xu Peng
  • Biao Qiu
  • Quanlei Wu
  • Chuan LvEmail author
  • Zili Wang
  • Dong Zhou


Immersive and non-immersive simulations have been widely adopted to improve product maintainability design based on virtual maintenance technique. On the one hand, immersive simulation shows that our body actions, especially those of the hand, are suitable for detailed operation simulation. Meanwhile, the problem associated with immersive simulation is hardware capacity as constrained by the environment. On the other hand, non-immersive simulation shows superiority in large-scale movement; however, problems, such as cumbersome operation, time-consuming labor, and inadequate precision, arise in terms of complicated and repeated operations. Considering the preceding advantages and limitations mentioned, this study presents a virtual simulation method by purposefully integrating human motion data from immersive and non-immersive simulations. We first decompose maintenance process based on common operations to determine typical types of maintenance motions. We then analyze the classified motion characters to identify a suitable simulation approach for each type of motion. Subsequently, motion data from immersive and non-immersive simulation are processed by spatial and temporary alignment. These data are further integrated by synchronous or asynchronous cooperative control for their respective simulation requirement. Lastly, we adopt interpolation to further process position and posture angle data and achieve smooth transition between adjacent simulation slices. Three cases are introduced to separately verify the feasibility of synchronous and asynchronous control and smooth transition in the method.


Data integration Motion synthesis Smooth transition Virtual simulation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The authors express their sincerest gratitude to the designers for their support in motion data collection.


This research was financially supported by the National Natural Science Foundation of China (Grant No. 71701005) and the open funding project of State Key Laboratory of Virtual Reality Technology and Systems (Grant No. BUAAVR-17KF-10).

Supplementary material

170_2018_2560_MOESM1_ESM.fpp (25 kb)
ESM 1 (FPP 25 kb)
170_2018_2560_MOESM2_ESM.fpp (67 kb)
ESM 2 (FPP 67 kb)
170_2018_2560_MOESM3_ESM.fpp (3 kb)
ESM 3 (FPP 3 kb)
170_2018_2560_MOESM4_ESM.fpp (25 kb)
ESM 4 (FPP 25 kb)
170_2018_2560_MOESM5_ESM.fpp (9 kb)
ESM 5 (FPP 9 kb)
170_2018_2560_MOESM6_ESM.fpp (4 kb)
ESM 6 (FPP 4 kb)
170_2018_2560_MOESM7_ESM.fpp (24 kb)
ESM 7 (FPP 23 kb)
170_2018_2560_MOESM8_ESM.fpp (15 kb)
ESM 8 (FPP 14 kb)
170_2018_2560_MOESM9_ESM.fpp (29 kb)
ESM 9 (FPP 28 kb)
170_2018_2560_MOESM10_ESM.fpp (4 kb)
ESM 10 (FPP 3 kb)
170_2018_2560_MOESM11_ESM.fpp (16 kb)
ESM 11 (FPP 15 kb)
170_2018_2560_MOESM12_ESM.fpp (39 kb)
ESM 12 (FPP 39 kb)
170_2018_2560_MOESM13_ESM.fpp (6 kb)
ESM 13 (FPP 6 kb)
170_2018_2560_MOESM14_ESM.fpp (17 kb)
ESM 14 (FPP 16 kb)


  1. 1.
    Peng G, Hou X, Gao J, Cheng D (2012) A visualization system for integrating maintainability design and evaluation at product design stage. Int J Adv Manuf Technol 61:269–284CrossRefGoogle Scholar
  2. 2.
    Xia P, Lopes AM, Restivo MT, Yao Y (2012) A new type haptics-based virtual environment system for assembly training of complex products. Int J Adv Manuf Technol 58:379–396CrossRefGoogle Scholar
  3. 3.
    Walter T, Tullio T, Marcello U (2015) A virtual factory approach for in situ simulation to support production and maintenance planning. CIRP Ann Manuf Technol 64:451–454CrossRefGoogle Scholar
  4. 4.
    Pilia A, Brun C, Doceul L, Gargiulo L, Hatchressian JC, Keller D, Le R, Poli S, Zago B (2015) Application of virtual reality tools for assembly of WEST components: comparison between simulations and physical mockups. Fusion Eng Des 98–99:1589–1592CrossRefGoogle Scholar
  5. 5.
    Geng J, Zhou D, Lv C, Wang Z (2013) A modeling approach for maintenance safety evaluation in a virtual maintenance environment. Comput Aided Des 45:937–949CrossRefGoogle Scholar
  6. 6.
    Gallegos-Nieto E, Medellín-Castillo HI, González-Badillo G, Lim T, Ritchie J (2017) The analysis and evaluation of the influence of haptic-enabled virtual assembly training on real assembly performance. Int J Adv Manuf Technol 89:581–598CrossRefGoogle Scholar
  7. 7.
    Chen J, Mitrouchev P, Coquillart S, Quaine F (2017) Disassembly task evaluation by muscle fatigue estimation in a virtual reality environment. Int J Adv Manuf Technol 88:1523–1533CrossRefGoogle Scholar
  8. 8.
    Li X, Zhang Z, Gao Q, Liang F, Huang X (2015) Assembly oriented control algorithm of collaborative disassembly and assembly operation in collaborative virtual maintenance process. J Manuf Syst 36:95–108CrossRefGoogle Scholar
  9. 9.
    Geng J, Li Y, Wang R, Wang Z, Lv C, Zhou D (2017) A virtual maintenance-based approach for satellite assembling and troubleshooting assessment. Acta Astronaut 138:434–453CrossRefGoogle Scholar
  10. 10.
    Heemskerk CJM, Baar MRD, Boessenkool H, Graafland B, Haye MJ, Koning JF, Vahedi M, Visser M (2011) Extending virtual reality simulation of ITER maintenance operations with dynamic effects. Fusion Eng Des 86:2082–2086CrossRefGoogle Scholar
  11. 11.
    Louison C, Ferlay F, Keller D, Mestre DR (2017) Operators’ accessibility studies for assembly and maintenance scenarios using virtual reality. Fusion Eng Des 124:610–614CrossRefGoogle Scholar
  12. 12.
    Yang Z, He T, Shang L, Long P, Hu L (2015) Development of high-immersive simulation system for designing maintenance strategy and its application to CLEAR-I. Ann Nucl Energy 83:309–315CrossRefGoogle Scholar
  13. 13.
    Geng J, Lv C, Zhou DG, Li Y, Wang Z (2014) Compensation-based methodology for maintenance time prediction in a virtual environment. Simul Model Pract Theory 47:92–109CrossRefGoogle Scholar
  14. 14.
    Liu X, Cui X, Song G, Xu B (2014) Development of a virtual maintenance system with virtual hand. Int J Adv Manuf Technol 70:2241–2247CrossRefGoogle Scholar
  15. 15.
    Patrona F, Chatzitofis A, Zarpalas D, Daras P (2018) Motion analysis: action detection, recognition and evaluation based on motion capture data. Pattern Recogn 76:612–622CrossRefGoogle Scholar
  16. 16.
    Adwan S, Alsaleh I, Majed R (2016) A new approach for image stitching technique using dynamic time warping (DTW) algorithm towards scoliosis X-ray diagnosis. Measurement 84:32–46CrossRefGoogle Scholar
  17. 17.
    Angeles J (2015) The role of the rotation matrix in the teaching of planar kinematics. Mech Mach Theory 89:28–37CrossRefGoogle Scholar
  18. 18.
    Xu X, Chang CC, Faber GS, Kingma I, Dennerlein JT (2010) Interpolation of segment Euler angles can provide a robust estimation of segment angular trajectories during asymmetric lifting tasks. J Biomech 43:2043–2048CrossRefGoogle Scholar
  19. 19.
    Czaplewski B (2016) Joint fingerprinting and decryption method for color images based on quaternion rotation with cipher quaternion chaining. J Vis Commun Image Represent 40:1–13CrossRefGoogle Scholar
  20. 20.
    Petrellis N (2016) A low complexity digital unit for scalar linear interpolation and compression. Measurement 93:288–302CrossRefGoogle Scholar
  21. 21.
    Zupan E, Zupan D (2014) On higher order integration of angular velocities using quaternions. Mech Res Commun 55:77–85CrossRefGoogle Scholar
  22. 22.
    Aràndiga F (2016) A nonlinear algorithm for monotone piecewise bicubic interpolation. Appl Math Comput 272:100–113MathSciNetzbMATHGoogle Scholar
  23. 23.
    Azevedo JMC, Belinha J, Dinis LMJS, Jorge RMN (2015) Crack path prediction using the natural neighbour radial point interpolation method. Eng Anal Bound Elem 59:144–158MathSciNetCrossRefGoogle Scholar
  24. 24.
    Alaya I, Jribi M, Ghorbel F, Sappey-Marinier D, Kraiem T (2017) Fast and accurate estimation of the HARDI signal in diffusion MRI using a nearest-neighbor interpolation approach. IRBM 38:156–166CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Jie Geng
    • 1
    • 2
  • Xu Peng
    • 2
  • Biao Qiu
    • 2
  • Quanlei Wu
    • 2
  • Chuan Lv
    • 1
    • 2
    Email author
  • Zili Wang
    • 2
  • Dong Zhou
    • 1
    • 2
  1. 1.State Key Laboratory of Virtual Reality Technology and SystemBeijingPeople’s Republic of China
  2. 2.School of reliability and systems engineeringBeijing University of Aeronautics and AstronauticsBeijingPeople’s Republic of China

Personalised recommendations