Real-Time Video Surveillance Based Structural Health Monitoring of Civil Structures Using Artificial Neural Network

  • Moushumi MedhiEmail author
  • Aradhana Dandautiya
  • Jagdish Lal Raheja


Modern world’s incessantly increasing outdoor traffic load has eventually led to structural health concern and continuous health monitoring of large scale civil structures such as bridges, roads, highways, etc. In this paper, we propose a computer vision based non-destructive structural health monitoring (SHM) method using high speed camera system combined with the brilliance of artificial intelligence. A number of appreciable SHM techniques had been reported that utilizes wired or wireless smart sensors, but the use of nondestructive techniques, such as, digital high speed imaging were rarely employed for detection of dynamic vibrations of civil structures. In the current research, we have developed a high speed video imaging based structural health monitoring system that utilizes blob detection based motion tracking algorithm. It provides factual information regarding localization and displacement of the target object or an existing feature in the civil structure. The modal parameters were subsequently extracted to analyze the level of severity of structural damage within the civil structures. Also, an artificial neural network is trained to infer the qualitative characteristics of structural vibrations based on vibration intensity and the network inferences can be correlated with the conditions of the structure. The efficacy of our vision system in remote measurement of dynamic displacements was demonstrated through a shaking table and a slip desk experiment. The experimental results demonstrate real-time output with satisfactory performance.


Structural health monitoring Target motion tracking Multi-resolution analysis Discrete wavelet transform Artificial neural network 



The authors are grateful for the project being funded by the Council of Scientific and Industrial Research (CSIR), India.


  1. 1.
    Alahi, A., Ortiz, R., Vandergheynst, P.: FREAK: fast retina keypoint. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 510–517 (2012)Google Scholar
  2. 2.
    Bao, Y., Tang, Z., Li, H., Zhang, Y.: Computer vision and deep learning-based data anomaly detection method for structural health monitoring. Struct. Health Monit. 18, 401–421 (2018)CrossRefGoogle Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.J.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  4. 4.
    Busca, G., Cigada, A., Mazzoleni, P., Zappa, E.: Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system. Exp. Mech. 54(2), 255–271 (2014)CrossRefGoogle Scholar
  5. 5.
    Cao, H., Jebali, C., Kouki, A.B., Thakar, S., Nguyen, C.M., Rao, S., Chiao, J.C.: Development of a laser micro-machined interdigitated capacitive strain sensor for structural health monitoring applications. In: IEEE Sensors, pp. 1741–1744 (2014)Google Scholar
  6. 6.
    Catt, S., Fick, B., Hoskins, M., Praski, J., Baqersad, J.: Development of a semiautonomous drone for structural health monitoring of structures using digital image correlation (DIC). Struct. Health Monit. Photogramm. DIC 6, 49–57 (2019)Google Scholar
  7. 7.
    Cha, Y.J., Chen, J., Büyüköztürk, O.: Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters. Eng. Struct. 132, 300–313 (2017)CrossRefGoogle Scholar
  8. 8.
    Chaabane, M., Mansouri, M., Nounou, H., Nounou, M., Slima, M.B., Hamida, A.B.: Damage detection in structural health monitoring using kernel PLS based GLR. In: International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–5 (2017)Google Scholar
  9. 9.
    Chen, J.G., Adams, T.M., Sun, H., Bell, E.S., Büyüköztürk, O.: Camera-based vibration measurement of the world war I memorial bridge in portsmouth, New Hampshire. J. Struct. Eng. 144(11), 04018207 (2018)CrossRefGoogle Scholar
  10. 10.
    Chew, Z.J., Ruan, T., Zhu, M., Bafleur, M., Dilhac, J.M.: A multifunctional device as both strain sensor and energy harvester for structural health monitoring. In: IEEE Sensors, pp. 1–3 (2016)Google Scholar
  11. 11.
    Chong, A.K., Al-Baghdadi, J.A., Alshadli, D.: High definition video cameras for measuring movement of vibrating bridge structure. In: International Conference on Vibration and Vibroacoustics, pp. 1–10 (2014)Google Scholar
  12. 12.
    Contreras, W.C., Ziavras, S.G.: Efficient structural health monitoring with wireless sensor networks using a vibration-based frequency domain pattern matching technique. In: IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference, pp. 356–362 (2017)Google Scholar
  13. 13.
    Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex Fourier series. Math. Comput. 19(90), 297–301 (1965)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Daubechies, I.: Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. 41(7), 909–996 (1988)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Ding, H., He, Y.T., Jiao, S.B.: Rosette eddy current sensor for structural health monitoring. Appl. Mech. Mater. 330, 430–436 (2013)CrossRefGoogle Scholar
  16. 16.
    Dworakowski, Z., Kohut, P., Gallina, A., Holak, K., Uhl, T.: Vision-based algorithms for damage detection and localization in structural health monitoring. Struct. Control Health Monit. 23(1), 35–50 (2016)CrossRefGoogle Scholar
  17. 17.
    Feng, D., Feng, M.Q.: Vision-based multipoint displacement measurement for structural health monitoring. Struct. Control Health Monit. 23(5), 876–890 (2016)CrossRefGoogle Scholar
  18. 18.
    Feng, D., Feng, M.Q., Ozer, E., Fukuda, Y.: A vision-based sensor for noncontact structural displacement measurement. Sensors 15(7), 16557–16575 (2015)CrossRefGoogle Scholar
  19. 19.
    Giannelli, P., Bulletti, A., Capineri, L.: Multifunctional piezopolymer film transducer for structural health monitoring applications. IEEE Sens. J. 17(14), 4583–4586 (2017)CrossRefGoogle Scholar
  20. 20.
    Gul, M., Catbas, F.N.: Structural health monitoring and damage assessment using a novel time series analysis methodology with sensor clustering. J. Sound Vib. 330(6), 1196–1210 (2011)CrossRefGoogle Scholar
  21. 21.
    Guo, J., Zhu, C.: Dynamic displacement measurement of large-scale structures based on the lucaskanade template tracking algorithm. Mech. Syst. Signal Process. 66–67, 425–436 (2016)CrossRefGoogle Scholar
  22. 22.
    Harmanci, Y.E., Gülan, U., Holzner, M., Chatzi, E.: A novel approach for 3D-structural identification through video recording: magnified tracking. Sensors 19(5), 1229 (2019)CrossRefGoogle Scholar
  23. 23.
    Hoskere, V., Park, J.W., Yoon, H., Spencer Jr., B.F.: Vision-based modal survey of civil infrastructure using unmanned aerial vehicles. J. Struct. Eng. 145(7), 04019062 (2019)CrossRefGoogle Scholar
  24. 24.
    Hu, X., Wang, B., Ji, H.: A wireless sensor network-based structural health monitoring system for highway bridges. Comput. Aided Civil Infrastruct. Eng. 28(3), 193–209 (2013)CrossRefGoogle Scholar
  25. 25.
    ITU-R: Parameter values for the HDTV standards for production and international programme exchange. ITU-R, Geneva (2002)Google Scholar
  26. 26.
    Judd, J., Schmidt, R., Danforth, M., Branscomb, J., Jung, M.: Radio-frequency identification triggering system for structural health monitoring of highway bridges. In: Conte, J.P., Astroza, R., Benzoni, G., Feltrin, G., Loh, K.J., Moaveni, B. (eds.) Experimental Vibration Analysis for Civil Structures, pp. 118–125. Springer, New York (2018)CrossRefGoogle Scholar
  27. 27.
    Kaphle, M., Tan, A.C.C.: Differentiating signals from different sources of acoustic emission for structural health monitoring purposes. In: Shen, G., Wu, Z., Zhang, J. (eds.) Advances in Acoustic Emission Technology, pp. 513–521. Springer, New York (2015)CrossRefGoogle Scholar
  28. 28.
    Khuc, T., Catbas, F.N.: Completely contactless structural health monitoring of real-life structures using cameras and computer vision. Struct. Control Health Monit. 24(1), e1852 (2017)CrossRefGoogle Scholar
  29. 29.
    Kong, X., Li, J.: Non-contact fatigue crack detection in civil infrastructure through image overlapping and crack breathing sensing. Autom. Constr. 99, 125–139 (2019)CrossRefGoogle Scholar
  30. 30.
    Kranz, M.S., English, B.A., Whitley, M.R.: RFID-inspired wireless microsensors for structural health monitoring. In: IEEE Aerospace Conference, pp. 1–7 (2016)Google Scholar
  31. 31.
    Kypris, O., Markham, A.: 3-D displacement measurement for structural health monitoring using low-frequency magnetic fields. IEEE Sens. J. 17(4), 1165–1174 (2017)CrossRefGoogle Scholar
  32. 32.
    Liu, X., Cao, J., Guo, P.: SenetSHM: towards practical structural health monitoring using intelligent sensor networks. In: IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), pp. 416–423 (2016)Google Scholar
  33. 33.
    Lowe, P.S., Scholehwar, T., Yau, J., Kanfoud, J., Gan, T.H., Selcuk, C.: Flexible shear mode transducer for structural health monitoring using ultrasonic guided waves. IEEE Trans. Ind. Inf. 14(7), 2984–2993 (2018)CrossRefGoogle Scholar
  34. 34.
    Lydon, D., Lydon, M., Taylor, S., Del Rincon, J.M., Hester, D., Brownjohn, J.: Development and field testing of a vision-based displacement system using a low cost wireless action camera. Mech. Syst. Signal Process. 121, 343–358 (2019)CrossRefGoogle Scholar
  35. 35.
    Lynch, J.P., Loh, K.J.: A summary review of wireless sensors and sensor networks for structural health monitoring. Shock Vib. Dig. 38(2), 91–128 (2006)CrossRefGoogle Scholar
  36. 36.
    Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)CrossRefGoogle Scholar
  37. 37.
    Moll, J., Heftrich, C., Fritzen, C.P.: A rapid and velocity-independent damage localization approach for ultrasonic structural health monitoring. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 59(6), 1309–1315 (2012)CrossRefGoogle Scholar
  38. 38.
    Oppenheim, A.V.: Discrete-Time Signal Processing. Pearson Education India, Chennai (1999)Google Scholar
  39. 39.
    Oshima, Y., Yamamoto, K., Sugiura, K.: Damage assessment of a bridge based on mode shapes estimated by responses of passing vehicles. Smart Struct. Syst. 13(5), 731–753 (2014)CrossRefGoogle Scholar
  40. 40.
    Poudel, U.P., Fu, G., Ye, J.: Structural damage detection using digital video imaging technique and wavelet transformation. J. Sound Vib. 286, 869–895 (2005)CrossRefGoogle Scholar
  41. 41.
    Ribeiro, D., Calada, R., Ferreira, J., Martins, T.: Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system. Eng. Struct. 75, 164–180 (2014)CrossRefGoogle Scholar
  42. 42.
    Sabato, A., Feng, M.Q., Fukuda, Y., Carn, D.L., Fortino, G.: A novel wireless accelerometer board for measuring low-frequency and low-amplitude structural vibration. IEEE Sens. J. 16(9), 2942–2949 (2016)CrossRefGoogle Scholar
  43. 43.
    Shaladi, R., Alatshan, F., Yang, C.: An overview on the applications of structural health monitoring using wireless sensor networks in bridge engineering. In: International Conference on Advances in Science, Engineering, Technology & Natural Resources, pp. 4–11 (2015)Google Scholar
  44. 44.
    Smyth, A., Wu, M.: Multi-rate Kalman filtering for the data fusion of displacement and acceleration response measurements in dynamic system monitoring. Mech. Syst. Signal Process. 21(2), 706–723 (2007)CrossRefGoogle Scholar
  45. 45.
    Song, Y.Z., Bowen, C.R., Kim, A.H., Nassehi, A., Padget, J., Gathercole, N.: Virtual visual sensors and their application in structural health monitoring. Struct. Health Monit. 13(3), 251–264 (2014)CrossRefGoogle Scholar
  46. 46.
    Swit, G., Krampikowska, A., Chinh, L.M.: A prototype system for acoustic emission-based structural health monitoring of my thuan bridge. In: Prognostics and System Health Management Conference (PHM-Chengdu), pp. 1–6 (2016)Google Scholar
  47. 47.
    Talai, S.M., Desai, D.A., Heyns, P.S.: Infrared thermography applied to the prediction of structural vibration behaviour. IOSR J. Mech. Civil Eng. (IOSR-JMCE) 13(6), 10–17 (2016)Google Scholar
  48. 48.
    Teuscher, C.: FPGA implementations of neural networks. IEEE Trans. Neural Netw. 18(5), 1550 (2007)CrossRefGoogle Scholar
  49. 49.
    Tokognon, C.A., Gao, B., Tian, G.Y., Yan, Y.: Structural health monitoring framework based on internet of things: a survey. IEEE Internet of Things J. 4(3), 619–635 (2017)CrossRefGoogle Scholar
  50. 50.
    Torr, P.H.S., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78(1), 138–156 (2000)CrossRefGoogle Scholar
  51. 51.
    Wang, P., Yan, Y., Tian, G.Y., Bouzid, O., Ding, Z.: Investigation of wireless sensor networks for structural health monitoring. J. Sens. 2012, 1–7 (2012)CrossRefGoogle Scholar
  52. 52.
    Wikipedia contributors: Pinhole camera model|Wikipedia, the free encyclopedia. 871297383 (2018)
  53. 53.
    Wu, Q., Zhang, Y.D., Amin, M.G., Golato, A., Ahmad, F., Santhanam, S.: Structural health monitoring exploiting MIMO ultrasonic sensing and group sparse Bayesian learning. In: Asilomar Conference on Signals, Systems and Computers, pp. 1162–1166 (2014)Google Scholar
  54. 54.
    Yang, H., Xu, X., Neumann, I.: Laser scanning-based updating of a finite-element model for structural health monitoring. IEEE Sens. J. 16(7), 2100–2104 (2016)CrossRefGoogle Scholar
  55. 55.
    Ye, X.W., Dong, C.Z., Liu, T.: Force monitoring of steel cables using vision-based sensing technology: methodology and experimental verification. Smart Struct. Syst. 18(3), 585–599 (2016)CrossRefGoogle Scholar
  56. 56.
    Ye, X.W., Dong, C.Z., Liu, T.: Image-based structural dynamic displacement measurement using different multi-object tracking algorithms. Smart Struct. Syst. 17(6), 856–935 (2016)Google Scholar
  57. 57.
    Zaurin, R., Catbas, F.N.: Computer vision and sensor fusion for structural health monitoring framework with emphasis on unit in uence line analysis. In: Proceedings of the IMAC-XXVII, pp. 1–9 (2009)Google Scholar
  58. 58.
    Zhao, Y., Zhu, Y., Yuan, M., Wang, J., Zhu, S.: A laser-based fiber Bragg grating ultrasonic sensing system for structural health monitoring. IEEE Photonics Technol. Lett. 28(22), 2573–2576 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.CSIR-Central Electronics Engineering Research InstitutePilaniIndia

Personalised recommendations