Automated vision system for magnetic particle inspection of crankshafts using convolutional neural networks

The International Journal of Advanced Manufacturing Technology volume 112, pages3307–3326(2021)Cite this article

Abstract

This paper proposes a fully automated vision system to inspect the whole surface of crankshafts, based on the magnetic particle testing technique. A stepper motor combined with multiple cameras is needed to ensure the inspection of the whole surface of the crankshaft in real-time. Due to the very textured surface of crankshafts and the variability in defect shapes and types, defect detection methods based on deep learning algorithms, more precisely convolutional neural networks (CNNs), become a more efficient solution than traditional methods. This paper discusses the various approaches of defect detection with CNNs, mainly classification, object detection, and semantic segmentation. The advantages and weaknesses of each approach for real-time defect detection are presented. It is important to note that the proposed visual inspection system only replaces the manual inspection of crankshafts conducted by operators at the end of the magnetic particle testing procedure, allowing for an easy integration in any crankshaft factory.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. 1.

    Fonte M, Duarte P, Reis L, Freitas M, Infante V (2015) Failure mode analysis of two crankshafts of a single cylinder diesel engine. Eng Failure Anal 56:185–193. The sixth international conference on engineering failure analysis

    Article  Google Scholar 

  2. 2.

    Villanueva JB, Espadafor FJ, Cruz-Peragon F, García MT (2011) A methodology for cracks identification in large crankshafts. Mechan Syst Signal Process 25(8):3168–3185

    Article  Google Scholar 

  3. 3.

    McEvily A (2004) Failures in inspection procedures: Case studies. Eng Failure Anal 11(2):167–176

    Article  Google Scholar 

  4. 4.

    Espadafor FJ, Villanueva JB, García MT (2009) Analysis of a diesel generator crankshaft failure. Eng Fail Anal 16(7):2333–2341

    Article  Google Scholar 

  5. 5.

    Vetterlein T (2008) Application of magnetic particle inspection in the field of the automotive industry. In: Abstracts of 17th world conference on non-destructive testing

  6. 6.

    Guerra AS, Pillet M, Maire JL (2008) Control of variability for man measurement. In: 12th IMEKO TC1-TC7 joint symposium on man, science and measurement, p. nc. annecy, France

  7. 7.

    Maire JL, Pillet M, Baudet N (2013) Gage r2&e2: An effective tool to improve the visual control of products. Int J Qual Reliab Manag 30(2):161–176

    Article  Google Scholar 

  8. 8.

    Kopardekar P, Mital A, Anand S (1993) Manual, hybrid and automated inspection literature and current research. Integr Manuf Syst 4(1):18–29

    Article  Google Scholar 

  9. 9.

    Neogi N, Mohanta DK, Dutta PK (2014) Review of vision-based steel surface inspection systems. EURASIP J Image Video Process 2014(1):50

    Article  Google Scholar 

  10. 10.

    Koch C, Georgieva K, Kasireddy V, Akinci B, Fieguth P (2015) A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv Eng Inform 29(2):196–210

    Article  Google Scholar 

  11. 11.

    Cao W, Liu Q, He Z (2020) Review of pavement defect detection methods. IEEE Access 8:14531–14544

    Article  Google Scholar 

  12. 12.

    Hanbay K, Talu MF, Özgüven ÖF (2016) Fabric defect detection systems and methods–a systematic literature review. Optik 127(24):11960–11973

    Article  Google Scholar 

  13. 13.

    Ehret T, Davy A, Morel JM, Delbracio M (2019) Image anomalies: A review and synthesis of detection methods. J Math Imaging Vision 61(5):710–743

    MathSciNet  Article  Google Scholar 

  14. 14.

    Zhou F, Liu G, Xu F, Deng H (2019) A generic automated surface defect detection based on a bilinear model. Appl Sci 9(15):3159

    Article  Google Scholar 

  15. 15.

    Tout K, Cogranne R, Retraint F (2018) Statistical decision methods in the presence of linear nuisance parameters and despite imaging system heteroscedastic noise: Application to wheel surface inspection. Signal Process 144:430–443

    Article  Google Scholar 

  16. 16.

    Tao X, Zhang D, Ma W, Liu X, Xu D (2018) Automatic metallic surface defect detection and recognition with convolutional neural networks. Appl Sci 8(9):1575

    Article  Google Scholar 

  17. 17.

    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  18. 18.

    Ferguson MK, Ronay A, Lee YTT, Law KH (2018) Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning. Smart Sustain Manufact Syst 2

  19. 19.

    Hou W, Wei Y, Guo J, Jin Y, et al. (2018) Automatic detection of welding defects using deep neural network. In: Journal of physics: Conference series, vol 933. IOP Publishing, p 012006

  20. 20.

    Ye J, Ito S, Toyama N (2018) Computerized ultrasonic imaging inspection: from shallow to deep learning. Sensors 18(11):3820

    Article  Google Scholar 

  21. 21.

    Luo Q, Gao B, Woo WL, Yang Y (2019) Temporal and spatial deep learning network for infrared thermal defect detection. NDT E Int 108:102164

    Article  Google Scholar 

  22. 22.

    Tabernik D, Šela S, Skvarč J, Skočaj D (2020) Segmentation-based deep-learning approach for surface-defect detection. J Intell Manufact 31(3):759–776

    Article  Google Scholar 

  23. 23.

    Bastian BT, Jaspreeth N, Ranjith SK, Jiji C (2019) Visual inspection and characterization of external corrosion in pipelines using deep neural network. NDT E Int 102134:107

    Google Scholar 

  24. 24.

    Dung CV, Sekiya H, Hirano S, Okatani T, Miki C (2019) A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks. Autom Constr 102:217–229

    Article  Google Scholar 

  25. 25.

    Lv X, Duan F, Jiang JJ, Fu X, Gan L (2020) Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6):1562

    Article  Google Scholar 

  26. 26.

    Lin CY, Chen CH, Yang CY, Akhyar F, Hsu CY, Ng HF (2019) Cascading convolutional neural network for steel surface defect detection. In: International conference on applied human factors and ergonomics. Springer, New York, pp 202–212

  27. 27.

    Bamberger H, Hong E, Katz R, Agapiou JS, Smyth SM (2012) Non-contact, in-line inspection of surface finish of crankshaft journals. Int J Adv Manufact Technol 60(9-12):1039–1047

    Article  Google Scholar 

  28. 28.

    Iborra A, Alvarez B, Jimenez C, Fernandez-Merono J, Fernandez C, Suardiaz J (2000) Automated visual inspection system (avi) for crankshaft production processes. Europ J Mechan Environ Eng 45 (1):29–34

    Google Scholar 

  29. 29.

    Remeseiro B, Tarrío-saavedra J, Francisco-Fernández M, Penedo MG, Naya S, Cao R (2019) Automatic detection of defective crankshafts by image analysis and supervised classification. Int J Adv Manufact Technol 105(9):3761–3777

    Article  Google Scholar 

  30. 30.

    Myagkov L, Mahkamov K, Chainov N, Makhkamova I (2014) 11 - advanced and conventional internal combustion engine materials. In: Folkson R (ed) Alternative fuels and advanced vehicle technologies for improved environmental performance. Woodhead publishing, pp 370–408e

  31. 31.

    Chandna P, Chandra A (2009) Quality tools to reduce crankshaft forging defects: An industrial case study. J Indust Syst Eng (JISE)

  32. 32.

    Witek L, Sikora M, Stachowicz F, Trzepiecinski T (2017) Stress and failure analysis of the crankshaft of diesel engine. Eng Fail Anal 82:703–712

    Article  Google Scholar 

  33. 33.

    Metkar R, Sunnapwar V, Hiwase S (2011) A fatigue analysis and life estimation of crankshaft-a review. Int J Mechan Mater Eng 6(3):425–430

    Google Scholar 

  34. 34.

    Bhaumik S, Rangaraju R, Venkataswamy M, Bhaskaran T, Parameswara M (2002) Fatigue fracture of crankshaft of an aircraft engine. Eng Fail Anal 9(3):255–263

    Article  Google Scholar 

  35. 35.

    Pandey R (2003) Failure of diesel-engine crankshafts. Eng Failure Anal 10(2):165–175

    Article  Google Scholar 

  36. 36.

    Maass M, Deutsch WAK, Bartholomai F (2012) State of the art mt and ut test stations in the german automotive industry. In: Fall conference & quality testing show 2012, pp 2–6

  37. 37.

    Nishimine T, Tsuyama O, Tanaka T, Fujiwara H (1995) Automatic magnetic particle testing system for square billets. In: Industry applications conference, 1995. Thirtieth IAS annual meeting, IAS ’95., conference record of the 1995 IEEE, vol 2, pp 1585–1590 vol 2

  38. 38.

    Luo J, Tian Z, Yang J (2014) Fluorescent magnetic particle inspection device based on digital image processing. In: Proceeding of the 11th world congress on intelligent control and automation, pp 5677–5681

  39. 39.

    Ewert U, Jaenisch GR, Osterloh K, Zscherpel U, Bathias C, Hentschel M, Erhard A, Goebbels J, Hanselka H, Nuffer J, Daum W (2006) Performance control and condition monitoring. Springer, Berlin, pp 831–912

    Google Scholar 

  40. 40.

    Hellier C, Hellier C (2013) Handbook of nondestructive evaluation, 2nd edn. McGraw-Hill handbooks McGraw-Hill Education

  41. 41.

    Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge. http://www.deeplearningbook.org

    Google Scholar 

  42. 42.

    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  43. 43.

    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    MathSciNet  Article  Google Scholar 

  44. 44.

    Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  45. 45.

    Kotsiantis S, Kanellopoulos D, Pintelas P, et al. (2006) Handling imbalanced datasets: A review. GESTS Int Trans Comput Sci Eng 30(1):25–36

    Google Scholar 

  46. 46.

    Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22 (10):1345–1359

    Article  Google Scholar 

  47. 47.

    Zhao Z, Zheng P, Xu S, Wu X (2019) Object detection with deep learning: A review. IEEE Trans Neural Netw Learn Syst 1–21 https://doi.org/10.1109/TNNLS.2018.2876865

  48. 48.

    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision. Springer, New York, pp 21–37

  49. 49.

    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition

  50. 50.

    Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vision 88(2):303–338

    Article  Google Scholar 

  51. 51.

    Ferguson M, Ak R, Lee YTT, Law KH (2017) Automatic localization of casting defects with convolutional neural networks. In: 2017 IEEE international conference on big data (big data). IEEE, pp 1726–1735

  52. 52.

    Lateef F, Ruichek Y (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338:321–348. https://doi.org/10.1016/j.neucom.2019.02.003

    Article  Google Scholar 

  53. 53.

    Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, New York, pp 234–241

  54. 54.

    Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift

  55. 55.

    Milletari F, Navab N, Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth international conference on 3d vision (3DV). IEEE, pp 565–571

  56. 56.

    Yu T, Zhu H (2020) Hyper-parameter optimization: A review of algorithms and applications. arXiv:2003.05689

Download references

Funding

This work was supported by the TRAC project, grant FUI 24 by the Unique Inter-ministerial Fund.

Author information

Affiliations

  1. IRIMAS - EA 7499 - Université de Haute-Alsace, Mulhouse, France

    Karim Tout, Anis Meguenani, Jean-Philippe Urban & Christophe Cudel

Authors
  1. Karim Tout
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anis Meguenani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Philippe Urban
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christophe Cudel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karim Tout.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Consent to publish

Consent is given to publish.

Ethical approval

The manuscript contains original ideas which have never been published before in other journals.

Consent to participate

Consent is given to participate.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author’s contributions

All authors contributed in writing and revising the manuscripts.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tout, K., Meguenani, A., Urban, JP. et al. Automated vision system for magnetic particle inspection of crankshafts using convolutional neural networks. Int J Adv Manuf Technol 112, 3307–3326 (2021). https://doi.org/10.1007/s00170-020-06467-4

Download citation

Keywords

  • Magnetic particle inspection
  • Defect detection
  • Vision system
  • Quality control
  • Convolutional neural networks