Journal of Intelligent Manufacturing

, Volume 30, Issue 5, pp 2245–2256 | Cite as

A comparison of dimension reduction techniques for support vector machine modeling of multi-parameter manufacturing quality prediction

  • Yun Bai
  • Zhenzhong Sun
  • Bo Zeng
  • Jianyu Long
  • Lin Li
  • José Valente de Oliveira
  • Chuan LiEmail author


Manufacturing quality prediction model, as an effective measure to monitor the quality in advance, has been developed using various data-driven techniques. However, multi-parameter in multi-stage of the modern manufacturing industry brings about the curse of dimensionality, leading to the difficulties for feature extraction, learning and quality modeling. To address this issue, three dimension reduction techniques are investigated in this paper, i.e., principal component analysis (PCA), locally linear embedding (LLE), and isometric mapping (Isomap). Specifically, the PCA is a linear dimension reduction technique, the LLE is a nonlinear reduction technique with local perspective, and the Isomap is a nonlinear reduction technique from global perspective. After getting the low-dimensional information from the PCA, the LLE, and the Isomap methods respectively, a support vector machine (SVM) is utilized for modeling. To reveal the effectiveness of the dimension reduction techniques and compare the difference of the three dimension reduction techniques, two experimental manufacturing data are collected from a competition about manufacturing quality control in Tianchi Data Lab of China. The comparison experiments indicate that the dimension reduction techniques have capacity for improving the SVM modeling performance indeed, and the Isomap–SVM model with the nonlinear global dimension reduction outperforms all the candidate models in terms of qualitative and quantitative analysis.


PCA LLE Isomap SVM Manufacturing quality prediction 



This work is supported in part by the National Natural Science Foundation of China (51775112, 71771033), the Postdoctoral Science Foundation of China (2016M602459), and the Research Program of Higher Education of Guangdong (2016KZDXM054). We also extend special thanks to the editor/reviewers for their valuable comments in improving the quality of this paper.


  1. Abdi, H., & Williams, L. J. (2010). Principal component analysis. Wiley Interdisciplinary Reviews Computational Statistics, 2(4), 433–459.Google Scholar
  2. Akashi, K., & Kunitomo, N. (2015). The limited information maximum likelihood approach to dynamic panel structural equation models. Annals of the Institute of Statistical Mathematics, 67(1), 39–73.Google Scholar
  3. Bai, Y., & Li, C. (2016). Daily natural gas consumption forecasting based on a structure-calibrated support vector regression approach. Energy and Buildings, 127, 571–579.Google Scholar
  4. Bai, Y., Sun, Z. Z., Zeng, B., Deng, J., & Li, C. (2017). A multi-pattern deep fusion model for short-term bus passenger flow forecasting. Applied Soft Computing, 58, 669–680.Google Scholar
  5. Bustillo, A., & Correa, M. (2012). Using artificial intelligence to predict surface roughness in deep drilling of steel components. Journal of Intelligence Manufacturing, 23(5), 1893–1902.Google Scholar
  6. Chamkalani, A., Chamkalani, R., & Mohammadi, A. H. (2014). Hybrid of two heuristic optimizations with LSSVM to predict refractive index as asphaltene stability identifier. Journal of Dispersion Science and Technology, 35(8), 1041–1050.Google Scholar
  7. Du, S. C., Huang, D. L., & Wang, H. (2015a). An adaptive support vector machine-based workpiece surface classification system using high-definition metrology. IEEE Transactions on Instrumentation and Measurement, 64(10), 2590–2604.Google Scholar
  8. Du, S. C., Liu, C. P., & Xi, L. F. (2015b). A selective multiclass support vector machine ensemble classifier for engineering surface classification using high definition metrology. Journal of Manufacturing Science and Engineering, 137(1), 011003.Google Scholar
  9. Du, S. C., Xu, R., Huang, D. L., & Yao, X. F. (2015c). Markov modeling and analysis of multi-stage manufacturing systems with remote quality information feedback. Computers and Industrial Engineering, 88, 13–25.Google Scholar
  10. Duarte, E., & Wainer, J. (2017). Empirical comparison of cross-validation and internal metrics for tuning SVM hyperparameters. Pattern Recognition Letters, 88, 6–11.Google Scholar
  11. Fan, J., Jing, F., Fang, Z., & Tan, M. (2017). Automatic recognition system of welding seam type based on SVM method. International Journal of Advanced Manufacturing Technology, 92(1–4), 989–999.Google Scholar
  12. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861–874.Google Scholar
  13. Hao, L., Bian, L., Gebraeel, N., & Shi, J. (2017). Residual life prediction of multistage manufacturing processes with interaction between tool wear and product quality degradation. IEEE Transactions on Automation Science and Engineering, 14(2), 1211–1224.Google Scholar
  14. Hosein, K. M., Karim, A., & Saeed, K. S. M. (2013). Development of a new expert system for statistical process control in manufacturing industry. Iranian Electric Industry Journal of Quality and Productivity, 2(3), 29–40.Google Scholar
  15. Hougardy, S. (2010). The Floyd-Warshall algotithm on graphs with negative cycles. Information Processing Letters, 110, 279–281.Google Scholar
  16. Jin, X., Zhao, M., Chow, T. W. S., & Pecht, M. (2014). Motor bearing fault diagnosis using trace ratio linear discriminant analysis. IEEE Transactions on Industrial Electronics, 61(5), 2441–2451.Google Scholar
  17. Kao, L. J., Lee, T. S., & Lu, C. J. (2016). A multi-stage control chart pattern recognition scheme based on independent component analysis and support vector machine. Journal of Intelligence Manufacturing, 27(3), 653–664.Google Scholar
  18. Langone, R., Alzate, C., Ketelaere, B. D., Vlasselaer, J., Meert, W., & Suykens, J. (2015). LS-SVM based spectral clustering and regression for predicting maintenance of industrial machines. Engineering Applications of Artificial Intelligence, 37, 268–278.Google Scholar
  19. Li, C., Oliveira, J. V. D., Sanchez, R. V., Cerrada, M., Zurita, G., & Cabrera, D. (2016a). Fuzzy determination of informative frequency band for bearing fault detection. Journal of Intelligent and Fuzzy Systems, 30, 3513–3525.Google Scholar
  20. Li, C., Sanchez, R. V., Zurita, G., Cerrada, M., Cabrera, D., & Vásquez, R. (2016b). Gearbox fault diagnosis based on deep random forest fusion of acoustic and vibratory signals. Mechanical Systems and Signal Processing, 76–77, 283–293.Google Scholar
  21. Li, D. C., Chen, W. C., Liu, C. W., & Lin, Y. S. (2012). A non-linear quality improvement model using SVR for manufacturing TFT-LCDs. Journal of Intelligent Manufacturing, 23(3), 835–844.Google Scholar
  22. Li, F., Tang, B., & Yang, R. (2013). Rotating machine fault diagnosis using dimension reduction with linear local tangent space alignment. Measurement, 46(8), 2525–2539.Google Scholar
  23. Lieber, D., Stolpe, M., Konrad, B., Deuse, J., & Morik, K. (2013). Quality prediction in interlinked manufacturing processes based on supervised and unsupervised machine learning. Procedia Cirp, 7, 193–198.Google Scholar
  24. Lindau, B., Lindkvist, L., Andersson, A., & Söderberg, R. (2013). Statistical shape modeling in virtual assembly using PCA-technique. Journal of Manufacturing Systems, 32(3), 456–463.Google Scholar
  25. Liu, G., Gao, X., You, D., & Zhang, N. (2016). Prediction of high power laser welding status based on PCA and SVM classification of multiple sensors. Journal of Intelligent Manufacturing.
  26. Liu, S., Hu, Y., Li, C., Lu, H., & Zhang, H. (2017). Machinery condition prediction based on wavelet and support vector machine. Journal of Intelligent Manufacturing, 28(4), 1045–1055.Google Scholar
  27. Ni, J., Zhang, C., & Yang, S. X. (2011). An adaptive approach based on KPCA and SVM for real-time fault diagnosis of HVCBs. IEEE Transactions on Power Delivery, 26(3), 1960–1971.Google Scholar
  28. Paul, S. K. (2016). Prediction of complete forming limit diagram from tensile properties of various steel sheets by a nonlinear regression based approach. Journal of Manufacturing Processes, 23, 192–200.Google Scholar
  29. Rao, K. V., & Murthy, P. B. G. S. N. (2016). Modeling and optimization of tool vibration and surface roughness in boring of steel using RSM, ANN and SVM. Journal of Intelligent Manufacturing.
  30. Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.Google Scholar
  31. Rubio, G., Pomares, H., Rojas, I., & Herrera, L. J. (2011). A heuristic method for parameterselection in LS-SVM: Application to time series prediction. International Journal of Forecasting, 27(3), 725–739.Google Scholar
  32. Sun, H., Yang, J., & Wang, L. (2017). Resistance spot welding quality identification with particle swarm optimization and a kernel extreme learning machine model. International Journal of Advanced Manufacturing Technology, 91(5–8), 1879–1887.Google Scholar
  33. Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319–2323.Google Scholar
  34. Wu, C. W., & Liao, M. Y. (2012). Generalized inference for measuring process yield with the contamination of measurement errors-quality control for silicon wafer manufacturing processes in the semiconductor industry. IEEE Transactions on Semiconductor Manufacturing, 25(2), 272–283.Google Scholar
  35. Yang, L. (2008). Alignment of overlapping locally scaled patches for multidimensional scaling and dimensionality reduction. IEEE Transactions on Pattern Analysis & Machine Intelligence, 30(3), 438–450.Google Scholar
  36. Ye, Q., & Zhi, W. (2015). Discrete Hessian Eigenmaps method for dimensionality reduction. Journal of Computational and Applied Mathematics, 278, 197–212.Google Scholar
  37. Yousefian-Jazi, A., Ryu, J. H., Yoon, S., & Liu, J. J. (2014). Decision support in machine vision system for monitoring of TFT-LCD glass substrates manufacturing. Journal of Process Control, 24(6), 1015–1023.Google Scholar
  38. Zhang, C., & Zhang, H. (2016). Modelling and prediction of tool wear using LS-SVM in milling operation. International Journal of Computer Integrated Manufacturing, 29(1), 76–91.Google Scholar
  39. Zhang, Z., Chow, T. W. S., & Zhao, M. (2013). Trace ratio optimization-based semi-supervised nonlinear dimensionality reduction for marginal manifold visualization. IEEE Transactions on Knowledge and Data Engineering, 25(5), 1148–1161.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Yun Bai
    • 1
    • 2
  • Zhenzhong Sun
    • 1
  • Bo Zeng
    • 1
  • Jianyu Long
    • 1
  • Lin Li
    • 3
  • José Valente de Oliveira
    • 2
  • Chuan Li
    • 1
    Email author
  1. 1.School of Mechanical EngineeringDongguan University of TechnologyDongguanChina
  2. 2.CEOTUniversidade do AlgarveFaroPortugal
  3. 3.School of Computer Science and Network SecurityDongguan University of TechnologyDongguanChina

Personalised recommendations