Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Three-Dimensional Reconstruction and Monitoring of Large-Scale Structures via Real-Time Multi-vision System

  • Conference paper
  • First Online:
Artificial Intelligence Algorithms and Applications (ISICA 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1205))

Included in the following conference series:

Abstract

A four-ocular vision system is proposed for the three-dimensional (3D) reconstruction of large-scale concrete-filled steel tube (CFST) materials under complex testing conditions. These measurements are vitally important for evaluating the seismic performance and 3D deformation of large-scale specimens. A four-ocular vision system is constructed to sample the large-scale CFST, then point cloud acquisition, filtering, and stitching algorithms are applied to obtain 3D point cloud of the specimen surface. Novel point cloud correction algorithms based on geometric features and deep learning are proposed to correct the coordinates of the stitched point cloud. The proposed algorithms center on the stitching error of the multi-view point cloud and the geometric and spatial characteristics of the targets for error compensation, which makes them highly adaptive and efficient. A high-accuracy multi-view 3D model for the purposes of real-time complex surface monitoring can be obtained via this method. Performance indicators of the two algorithms were evaluated on actual tasks. The cross-section diameters at specific heights in the reconstructed models were calculated and compared against laser range finder data to test the performance of the proposed method. A visual tracking test on a CFST under cyclic loading shows that the reconstructed output well reflects the complex 3D surface after point cloud correction and meets the requirements for dynamic monitoring. The proposed method is applicable to complex environments featuring dynamic movement, mechanical vibration, and continuously changing features.

Y. Tang and M. Chen—Authors contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pan, B.: Thermal error analysis and compensation for digital image/volume correlation. Opt. Lasers Eng. 101, 1–15 (2018)

    Article  Google Scholar 

  2. Genovese, K., Chi, Y., Pan, B.: Stereo-camera calibration for large-scale DIC measurements with active phase targets and planar mirrors. Opt. Express 27, 9040–9053 (2019)

    Article  Google Scholar 

  3. Dong, Y., Pan, B.: In-situ 3D shape and recession measurements of ablative materials in an arc-heated wind tunnel by UV stereo-digital image correlation. Opt. Lasers Eng. 116, 75–81 (2019)

    Article  Google Scholar 

  4. Fathi, H., Dai, F., Lourakis, M.: Automated as-built 3D reconstruction of civil infrastructure using computer vision: Achievements, opportunities, and challenges. Adv. Eng. Inform. 29, 149–161 (2015)

    Article  Google Scholar 

  5. Kim, H., Leutenegger, S., Davison, Andrew J.: Real-Time 3D reconstruction and 6-DoF tracking with an event camera. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 349–364. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_21

    Chapter  Google Scholar 

  6. Munda, G., Reinbacher, C., Pock, T.: Real-time intensity-image reconstruction for event cameras using manifold regularisation. Int. J. Comput. Vision 126, 1381–1393 (2018)

    Article  Google Scholar 

  7. Feng, D.-M., Feng, M.Q.: Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection–a review. Eng. Struct. 156, 105–117 (2018)

    Article  Google Scholar 

  8. Feng, D.-M., Feng, M.Q.: Vision-based multipoint displacement measurement for structural health monitoring. Struct. Control Health Monit. 23, 876–890 (2016)

    Article  Google Scholar 

  9. Cai, Z., Liu, X., Li, A., Tang, Q., Peng, X., Gao, B.Z.: Phase-3D mapping method developed from back-projection stereovision model for fringe projection profilometry. Opt. Express 25, 1262–1277 (2017)

    Article  Google Scholar 

  10. Hyun, J.S., Chiu, G.T., Zhang, S.: High-speed and high-accuracy 3D surface measurement using a mechanical projector. Opt. Express 26, 1474 (2018)

    Article  Google Scholar 

  11. Zhen, L., Li, X., Li, F., Zhang, G.: Flexible dynamic measurement method of three-dimensional surface profilometry based on multiple vision sensors. Opt. Express 23, 384–400 (2015)

    Article  Google Scholar 

  12. Wu, Q., Zhang, B., Huang, J., Wu, Z., Zeng, Z.: Flexible 3D reconstruction method based on phase-matching in multi-sensor system. Opt. Express 24, 7299–7318 (2016)

    Article  Google Scholar 

  13. Yang, X., Li, H., Yu, Y., Luo, X., Huang, T., Yang, X.: Automatic pixel-level crack detection and measurement using fully convolutional network. Comput. Aided Civ. Infrastruct. Eng. 33, 1090–1109 (2018)

    Article  Google Scholar 

  14. Huňady, R., Hagara, M.: A new procedure of modal parameter estimation for high-speed digital image correlation. Mech. Syst. Signal Process. 93, 66–79 (2017)

    Article  Google Scholar 

  15. Huňady, R., Pavelka, P., Lengvarský, P.: Vibration and modal analysis of a rotating disc using high-speed 3D digital image correlation. Mech. Syst. Signal Process. 121, 201–214 (2019)

    Article  Google Scholar 

  16. Tang, Y., et al.: Real-time detection of surface deformation and strain in recycled aggregate concrete-filled steel tubular columns via four-ocular vision. Robot. Comput.-Integr. Manuf. 59, 36–46 (2019)

    Article  Google Scholar 

  17. Ma, Z.-L., Liu, S.-L.: A review of 3D reconstruction techniques in civil engineering and their applications. Adv. Eng. Inform. 37, 163–174 (2018)

    Article  Google Scholar 

  18. Kim, H., Kim, H.: 3D reconstruction of a concrete mixer truck for training object detectors. Autom. Constr. 88, 23–30 (2018)

    Article  Google Scholar 

  19. Sun, L., Abolhasannejad, V., Gao, L., Li, Y.-W.: Non-contact optical sensing of asphalt mixture deformation using 3D stereo vision. Measurement 85, 100–117 (2016)

    Article  Google Scholar 

  20. Liu, Y., Yang, J.-C., Meng, Q.-G., Lv, Z.-H., Song, Z.-J., Gao, Z.-Q.: Stereoscopic image quality assessment method based on binocular combination saliency model. Sig. Process. 125, 237–248 (2016)

    Article  Google Scholar 

  21. Liu, Z., et al.: 3D real human reconstruction via multiple low-cost depth cameras. Sig. Process. 112, 162–179 (2015)

    Article  Google Scholar 

  22. Candau, N., Pradille, C., Bouvard, J.-L., Billon, N.: On the use of a four-cameras stereovision system to characterize large 3D deformation in elastomers. Polym. Testing 56, 314–320 (2016)

    Article  Google Scholar 

  23. Zhou, P., et al.: Experimental study of temporal-spatial binary pattern projection for 3D shape acquisition. Appl. Opt. 56, 2995–3003 (2017)

    Article  Google Scholar 

  24. Shen, X., Markman, A., Javidi, B.: Three-dimensional profilometric reconstruction using flexible sensing integral imaging and occlusion removal. Appl. Opt. 56, D151–D157 (2017)

    Article  Google Scholar 

  25. Malesa, M., et al.: Non-destructive testing of industrial structures with the use of multi-camera Digital Image Correlation method. Eng. Fail. Anal. 69, 122–134 (2016)

    Article  Google Scholar 

  26. Sinha, A., Bai, J., Ramani, K.: Deep learning 3D shape surfaces using geometry images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 223–240. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_14

    Chapter  Google Scholar 

  27. Li, F., et al.: Depth acquisition with the combination of structured light and deep learning stereo matching. Signal Process. Image Commun. 75, 111–117 (2019)

    Article  Google Scholar 

  28. Zhang, J., Hu, S., Shi, H.: Deep learning based object distance measurement method for binocular stereo vision blind area. Methods 9 (2018)

    Google Scholar 

  29. Sun, S., Liu, R., Pan, Y., Du, Q., Sun, S., Su, H.: Pose determination from multi-view image using deep learning. In: 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), pp. 1494–1498. IEEE, June 2019

    Google Scholar 

  30. Yang, Y., Qiu, F., Li, H., Zhang, L., Wang, M.-L., Fu, M.-Y.: Large-scale 3D semantic mapping using stereo vision. Int. J. Autom. Comput. 15(2), 194–206 (2018). https://doi.org/10.1007/s11633-018-1118-y

    Article  Google Scholar 

  31. Zou, X., Zou, H., Lu, J.: Virtual manipulator-based binocular stereo vision positioning system and errors modelling. Mach. Vis. Appl. 23, 43–63 (2012). https://doi.org/10.1007/s00138-010-0291-y

    Article  Google Scholar 

  32. Lin, G., Tang, Y., Zou, X., Xiong, J., Fang, Y.: Color-, depth-, and shape-based 3D fruit detection. Precision Agric. 21(1), 1–17 (2019). https://doi.org/10.1007/s11119-019-09654-w

    Article  Google Scholar 

  33. Lin, G., Tang, Y., Zou, X., Cheng, J., Xiong, J.: Fruit detection in natural environment using partial shape matching and probabilistic Hough transform. Precision Agric. 21(1), 160–177 (2019). https://doi.org/10.1007/s11119-019-09662-w

    Article  Google Scholar 

  34. Lin, G., Tang, Y., Zou, X., Xiong, J., Li, J.: Guava detection and pose estimation using a low-cost RGB-D sensor in the field. Sensors 19, 428 (2019)

    Article  Google Scholar 

  35. Luo, L., Tang, Y., Zou, X., Wang, C., Zhang, P., Feng, W.: Robust grape cluster detection in a vineyard by combining the AdaBoost framework and multiple color components. Sensors 16, 2098 (2016)

    Article  Google Scholar 

  36. Luo, L., Tang, Y., Zou, X., Ye, M., Feng, W., Li, G.: Vision-based extraction of spatial information in grape clusters for harvesting robots. Biosys. Eng. 151, 90–104 (2016)

    Article  Google Scholar 

  37. Tang, Y., Li, L., Feng, W., Liu, F., Zou, X., Chen, M.: Binocular vision measurement and its application in full-field convex deformation of concrete-filled steel tubular columns. Measurement 130, 372–383 (2018)

    Article  Google Scholar 

  38. Wang, C., Tang, Y., Zou, X., Luo, L., Chen, X.: Recognition and matching of clustered mature litchi fruits using binocular charge-coupled device (CCD) color cameras. Sensors 17, 2564 (2017)

    Article  Google Scholar 

  39. Wang, C., Tang, Y., Zou, X., SiTu, W., Feng, W.: A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot. Optik-Int. J. Light Electron Opt. 131, 626–631 (2017)

    Article  Google Scholar 

  40. Wang, C., Zou, X., Tang, Y., Luo, L., Feng, W.: Localisation of litchi in an unstructured environment using binocular stereo vision. Biosys. Eng. 145, 39–51 (2016)

    Article  Google Scholar 

  41. Song, S., Duan, J., Yang, Z., Zou, X., Fu, L., Ou, Z.: A three-dimensional reconstruction algorithm for extracting parameters of the banana pseudo-stem. Optik 185, 486–496 (2019)

    Article  Google Scholar 

  42. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000)

    Article  Google Scholar 

  43. Sereewattana, M., Ruchanurucks, M., Siddhichai, S.: Depth estimation of markers for UAV automatic landing control using stereo vision with a single camera. In: International Conference on Information and Communication Technology for Embedded System (2014)

    Google Scholar 

  44. Hirschmuller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE 2005), vol. 2, pp. 807–814 (2005)

    Google Scholar 

  45. Zeineldin, R.A., El-Fishawy, N.A.: Fast and accurate ground plane detection for the visually impaired from 3D organized point clouds. In: 2016 SAI Computing Conference (SAI), (IEEE 2016), pp. 373–379 (2016)

    Google Scholar 

  46. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV, p. 2 (1998)

    Google Scholar 

  47. Skinner, B., Vidal-Calleja, T., Miro, J.V., De Bruijn, F., Falque, R.: 3D point cloud upsampling for accurate reconstruction of dense 2.5 D thickness maps. In: Australasian Conference on Robotics and Automation, ACRA (2014)

    Google Scholar 

  48. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  Google Scholar 

  49. Qi, C.R., Yi, L., Su, H., et al.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)

    Google Scholar 

  50. Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Sensor Fusion IV Control Paradigms Data Structure, vol. 1611, pp. 586–607. International Society for Optics and Photonics (1992)

    Google Scholar 

  51. Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing (2006)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (51578162), the Key-area Research and Development Program of Guangdong Province (2019B020223003), and the Scientific and Technological Research Project of Guangdong Province (2016B090912005).

Author information

Authors and Affiliations

  1. College of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou, 510225, China

    Yunchao Tang

  2. Key Laboratory of Key Technology on Agricultural Machine and Equipment, College of Engineering, South China Agricultural University, Guangzhou, 510642, China

    Mingyou Chen, Kuangyu Huang, Fengyun Wu, Xiangjun Zou & Yuxin He

  3. Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education and School of Civil Engineering, Harbin Institute of Technology, Harbin, 150001, China

    Xiangguo Wu

  4. Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology, Harbin Institute of Technology, Harbin, 150090, China

    Yunchao Tang, Mingyou Chen, Xiangguo Wu, Kuangyu Huang, Fengyun Wu, Xiangjun Zou & Yuxin He

Authors
  1. Yunchao Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingyou Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangguo Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kuangyu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fengyun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiangjun Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yuxin He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunchao Tang .

Editor information

Editors and Affiliations

  1. South China Agricultural University, Guangzhou, China

    Kangshun Li

  2. Jiangxi University of Science and Technology, Ganzhou, China

    Wei Li

  3. South China Agricultural University, Guangzhou, China

    Hui Wang

  4. The University of Aizu, Aizu-Wakamatsu, Fukushima, Japan

    Yong Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tang, Y. et al. (2020). Three-Dimensional Reconstruction and Monitoring of Large-Scale Structures via Real-Time Multi-vision System. In: Li, K., Li, W., Wang, H., Liu, Y. (eds) Artificial Intelligence Algorithms and Applications. ISICA 2019. Communications in Computer and Information Science, vol 1205. Springer, Singapore. https://doi.org/10.1007/978-981-15-5577-0_35

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-5577-0_35

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-5576-3

  • Online ISBN: 978-981-15-5577-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation