Automatic Identification of Excavator Activities Using Joystick Signals

  • Jangho Bae
  • Kiyoung Kim
  • Daehie HongEmail author
Regular Paper


Monitoring and analyzing the operations of construction equipment is critical in the construction engineering and management domain. The ability to detect and classify major activities that construction equipment performs can support a project manager in making proper project-related decisions such as resource allocation and scheduling, resulting in improved productivity. Earth-moving activities as performed by an excavator, which is one of the most frequently used pieces of construction equipment, are normally repetitive by nature and possess unique features in terms of their patterns of operation. This study develops an activity identification algorithm capable of automatically classifying predefined earth-moving activities that are currently in progress. Given that the excavator is operated using joysticks, the joystick signals include unique patterns that exhibit the similar overall shapes but may not uniformly line up with time for a specific activity. The proposed study examines a dynamic time warping algorithm that determines similarities between a predefined activity and a measured signal distorted in time. The feasibility of the algorithm is verified through experiments involving activities such as digging, leveling, lifting, and trenching that were easily and accurately identified by the algorithm. The proposed task-identification algorithm could be used to develop an automated system of establishing machine parameters and to calculate the durations of operations and cycle times.


Earth-moving tasks Excavator Activity identification Dynamic time warping (DTW) Pattern recognition 



This research was supported by a grant (19AUDP-B121595-04) from Architecture & Urban Development Research Program funded by the Ministry of Land, Infrastructure and Transport of the Korean Government.


  1. 1.
    Gong, J., & Caldas, C. H. (2010). Computer vision-based video interpretation model for automated productivity analysis of construction operations. Journal of Computing in Civil Engineering,24(3), 252–263. Scholar
  2. 2.
    Golparvar-Fard, M., Heydarian, A., & Niebles, J. C. (2013). Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers. Advanced Engineering Informatics,27(4), 652–663. Scholar
  3. 3.
    Kim, J., Chi, S., & Seo, J. (2018). Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks. Automation in Construction,87, 297–308. Scholar
  4. 4.
    Yang, J., Edwards, D. J., & Love, P. E. D. (2003). A computational intelligent fuzzy model approach for excavator cycle time simulation. Automation in Construction,12(6), 725–735. Scholar
  5. 5.
    Oloufa, A. A., Ikeda, M., & Oda, H. (2003). Situational awareness of construction equipment using GPS, wireless and web technologies. Automation in Construction,12(6), 737–748. Scholar
  6. 6.
    Ergen, E., Akinci, B., & Sacks, R. (2007). Tracking and locating components in a precast storage yard utilizing radio frequency identification technology and GPS. Automation in Construction,16(3), 354–367. Scholar
  7. 7.
    Teizer, J., Lao, D., & Sofer, M. (2007). Rapid automated monitoring of construction site activities using ultra-wide band. In The 24th international symposium on automation and robotics in construction (ISARC 2007) (pp. 23–28).Google Scholar
  8. 8.
    Zhang, C., Hammad, A., & Rodriguez, S. (2012). Crane pose estimation using UWB real-time location system. Journal of Computing in Civil Engineering,26(5), 625–637. Scholar
  9. 9.
    Rezazadeh Azar, E., & McCabe, B. (2012). Part based model and spatial–temporal reasoning to recognize hydraulic excavators in construction images and videos. Automation in Construction,24, 194–202. Scholar
  10. 10.
    Zou, J., & Kim, H. (2007). Using hue, saturation, and value color space for hydraulic excavator idle time analysis. Journal of Computing in Civil Engineering,21(4), 238–246. Scholar
  11. 11.
    Yuan, C., Li, S., & Cai, H. (2017). Vision-based excavator detection and tracking using hybrid kinematic shapes and key nodes. Journal of Computing in Civil Engineering,31(1), 04016038. Scholar
  12. 12.
    Ahn, C. R., Lee, S., & Peña-Mora, F. (2013). Acceleromter-based measurement of construction equipment operating efficiency for monitoring environmental performance. ASCE International Workshop on Computing in Civil Engineering. Scholar
  13. 13.
    Ahn, C. R., Lee, S., & Peña-Mora, F. (2015). Application of low-cost accelerometers for measuring the operational efficiency of a construction equipment fleet. Journal of Computing in Civil Engineering,29(2), 04014042. Scholar
  14. 14.
    Mathur, N., Aria, S., Adams, T., Ahn, C., & Lee, S. (2015). Automated cycle time measurement and analysis of excavator’s loading operation using smart phone-embedded IMU sensors. International Workshop on Computing in Civil Engineering,2015, 215–222. Scholar
  15. 15.
    Akhavian, R., & Behzadan, A. H. (2015). Construction equipment activity recognition for simulation input modeling using mobile sensors and machine learning classifiers. Advanced Engineering Informatics,29(4), 867–877. Scholar
  16. 16.
    Kim, H., Ahn, C. R., Engelhaupt, D., & Lee, S. (2018). Application of dynamic time warping to the recognition of mixed equipment activities in cycle time measurement. Automation in Construction,87, 225–234. Scholar
  17. 17.
    Domingue, B. B. (2016). Coordinated rate control user interface and task identification of an excavator. MSc thesis, Georgia Institute of Technology. Retrieved from Accessed 4 Sep 2019.
  18. 18.
    Aach, J., & Church, G. M. (2001). Aligning gene expression time series with time warping algorithms. Bioinformatics,17(6), 495–508. Scholar
  19. 19.
    Rath, T. M., & Manmatha, R. (2003). Word image matching using dynamic time warping. In 2003 IEEE computer society conference on computer vision and pattern recognition (Vol. 2, pp. II/521–II/527).
  20. 20.
    Fischer, A., & Plamondon, R. (2017). Signature verification based on the kinematic theory of rapid human movements. IEEE Transactions on Human-Machine Systems,47(2), 169–180. Scholar
  21. 21.
    Keogh, E., & Ratanamahatana, C. A. (2005). Exact indexing of dynamic time warping. Knowledge and Information Systems,7(3), 358–386. Scholar
  22. 22.
    Salvadora, S., & Chan, P. (2007). Toward accurate dynamic time warping in linear time and space. Intelligent Data Analysis,11(5), 561–580. Scholar
  23. 23.
    Jeong, Y. S., Jeong, M. K., & Omitaomu, O. A. (2011). Weighted dynamic time warping for time series classification. Pattern Recognition,44(9), 2231–2240. Scholar
  24. 24.
    Sankoff, D., & Kruskal, J. B. (1983). Time warps, string edits, and macromolecules: The theory and practice of sequence comparison. Boston: Addison-Wesley.zbMATHGoogle Scholar

Copyright information

© Korean Society for Precision Engineering 2019

Authors and Affiliations

  1. 1.School of Mechanical EngineeringKorea UniversitySungbuk-Ku, SeoulRepublic of Korea

Personalised recommendations