Journal of Intelligent Manufacturing

, Volume 27, Issue 4, pp 845–874 | Cite as

Simultaneous monitoring of mean vector and covariance matrix shifts in bivariate manufacturing processes using hybrid ensemble learning-based model

  • Wen-An Yang


Many manufacturing processes are multivariate in nature because the quality of a given product is determined by several interrelated quality characteristics. Recently, various machine learning techniques (e.g., artificial neural network, support vector machine, support vector regression or decision tree) have been used as an effective tool to monitor process mean vector and covariance matrix shifts. However, most of these machine learning techniques-based approaches for process mean vector and covariance matrix have been developed separately in literature with the other parameter assumed to be under control. Little attention has been given to simultaneous monitoring of process mean vector and covariance matrix shifts. In addition, these approaches cannot provide more detailed shift information, for example the shift magnitude, which would be greatly useful for quality practitioners to search the assignable causes that give rise to the out-of-control situation. This study presents a hybrid ensemble learning-based model for simultaneous monitoring of process mean vector and covariance matrix shifts. The numerical results indicate that the proposed model can effectively detect and recognize not only mean vector or covariance matrix shifts but also mixed situations where mean vector and covariance matrix shifts exist concurrently. Meanwhile, the magnitude of the shift of each of the shifted quality characteristics can be accurately quantified. Empirical comparisons also show that the proposed model performs better than other existing approaches in detecting mean vector and covariance matrix shifts, while also providing the capability of recognition of shift types and quantification of shift magnitudes. A demonstrative example is provided.


Statistical process control Control charts Bivariate manufacturing processes Artificial neural network  Support vector machine Support vector regression 



The author would like to express their sincere thanks to the editor and the two anonymous reviewers for their detailed and helpful comments to improve the quality of this article.


  1. Barghash, M. A., & Santarisi, N. S. (2004). Pattern recognition of control charts using artificial neural networks: Analyzing the effect of the training parameters. Journal of Intelligent Manufacturing, 15(5), 635–644.CrossRefGoogle Scholar
  2. Bersimis, S., Psarakis, S., & Panaretos, J. (2007). Multivariate statistical process control charts: An overview. Quality and Reliability Engineering International, 23(5), 517–543.CrossRefGoogle Scholar
  3. Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.Google Scholar
  4. Chen, G., Cheng, S. W., & Xie, H. (2005). A new multivariate control chart for monitoring both location and dispersion. Communications in Statistics-Simulation and Computation, 34(1), 203–217.CrossRefGoogle Scholar
  5. Chou, C.-Y., Liu, H.-R., Huang, X. R., & Chen, C.-H. (2002). Economic-statistical design of multivariate control charts using quality loss function. International Journal of Advanced Manufacturing Technology, 20(12), 916–924.CrossRefGoogle Scholar
  6. Costa, A. F. B., & Machado, M. A. G. (2011). Monitoring the mean vector and the covariance matrix of multivariate processes with sample means and sample ranges. Produção, 21(2), 197–208.Google Scholar
  7. Chen, L.-H., & Wang, T.-Y. (2004). Artificial neural networks to classify mean shifts from multivariate \( \chi ^{2} \) chart signals. Computers and Industrial Engineering, 47(2–3), 195–205.CrossRefGoogle Scholar
  8. Cheng, C.-S. (1997). A neural network approach for the analysis of control chart patterns. International Journal of Production Research, 35(3), 667–697.CrossRefGoogle Scholar
  9. Cheng, C.-S., & Cheng, H.-P. (2008). Identifying the source of variance shifts in the multivariate process using neural networks and support vector machines. Expert Systems with Applications, 35(1–2), 198–206.CrossRefGoogle Scholar
  10. Cheng, C.-S., Chen, P.-W., & Huang, K.-K. (2011). Estimating the shift size in the process mean with support vector regression and neural networks. Expert Systems with Applications, 38(8), 10624–10630.CrossRefGoogle Scholar
  11. Cheng, C.-S., & Cheng, H.-P. (2011). Using neural networks to detect the bivariate process variance shifts pattern. Computers and Industrial Engineering, 60(2), 269–278.CrossRefGoogle Scholar
  12. Du, S. C., Lv, J., & Xi, L. F. (2012). On-line classifying process mean shifts in multivariate control charts based on multi-class support vector machines. International Journal of Production Research, 50(22), 6288–6310.CrossRefGoogle Scholar
  13. Dieterle, F., Müller-Hagedorn, S., Liebich, H. M., & Gauglitz, G. (2003). Urinary nucleosides as potential tumor markers evaluated by learning vector quantization. Artificial Intelligence in Medicine, 28(3), 265–279.CrossRefGoogle Scholar
  14. Gu, N., Cao, Z. Q., Xie, L. J., et al. (2013). Identification of concurrent control chart patterns with singular spectrum analysis and learning vector quantization. Journal of Intelligent Manufacturing, 24(6), 1241–1252.CrossRefGoogle Scholar
  15. Guh, R.-S. (2007). On-line identification and quantification of mean shifts in bivariate processes using a neural network-based approach. Quality and Reliability Engineering International, 23(3), 367–385.CrossRefGoogle Scholar
  16. Guh, R.-S., & Shiue, Y. R. (2008). An effective application of decision tree learning for online detection of mean shifts in multivariate control charts. Computers and Industrial Engineering, 55(2), 475–493.CrossRefGoogle Scholar
  17. Guh, R.-S., & Tannock, J. D. T. (1999). A neural network approach to characterize pattern parameters in process control charts. Journal of Intelligent Manufacturing, 10(5), 449–462.CrossRefGoogle Scholar
  18. Hansen, L. K., & Salamon, P. (1990). Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10), 993–1001.CrossRefGoogle Scholar
  19. He, S.-G., He, Z., & Wang, G. A. (2013). Online monitoring and fault identification of mean shifts in bivariate processes using decision tree learning techniques. Journal of Intelligent Manufacturing, 24(1), 25–34.CrossRefGoogle Scholar
  20. Ho, E. S., & Chang, S. I. (1999). An integrated neural network approach for simultaneous monitoring of process mean and variance shifts a comparative study. International Journal of Production Research, 37(8), 1881–1901.CrossRefGoogle Scholar
  21. Hwarng, H. B., & Hubele, N. F. (1993a). X-bar control chart pattern identification through efficient off-line neural network training. IIE Transactions, 25(3), 27–40.CrossRefGoogle Scholar
  22. Hwarng, H. B., & Hubele, N. F. (1993b). Back-propagation pattern recognizers for X-bar control charts: methodology and performance. Computers and Industrial Engineering, 24(2), 219–235.CrossRefGoogle Scholar
  23. Issam, B. K., & Mohamed, L. (2008). Support vector regression based residual MCUSUM control chart for autocorrelated process. Applied Mathematics and Computation, 201(1–2), 565–574.CrossRefGoogle Scholar
  24. Jiang, P. Y., Liu, D. Y., & Zeng, Z. J. (2009). Recognizing control chart patterns with neural network and numerical fitting. Journal of Intelligent Manufacturing, 20(6), 625–635.CrossRefGoogle Scholar
  25. Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimisation. Proceedings of the IEEE international conference on neural networks, 27 November-1 December, Perth, Australia, IV (pp. 1942–1948). Piscataway, NJ: IEEE Service Centre.Google Scholar
  26. Kennedy, J., & Eberhart, R. (1997). A discrete binary version of the particle swarm optimization. Proceedings of the IEEE international conference on computational cybernetics and simulation (pp. 4104–4108). Piscataway, NJ: IEEE Press.Google Scholar
  27. Khoo, M. B. C. (2005). A new bivariate control chart to monitor the multivariate process mean and variance simultaneously. Quality Engineering, 17(1), 109–118.CrossRefGoogle Scholar
  28. Krogh, A., & Vedelsby, J. (1995). Neural network ensembles cross validation, and active learning. Advances in neural information processing systems 7 (pp. 231–238). Denver, CO, Cambridge MA: MIT Press.Google Scholar
  29. Lapedes, A., & Farber, R. (1987). How neural nets work. In: Neural information processing systems (pp. 442–456). New York: American Institute of Physics.Google Scholar
  30. Low, C., Hsu, C. M., & Yu, F. J. (2003). Analysis of variations in a multi-variate process using neural networks. International Journal of Advanced Manufacturing Technology, 22(11–12), 911–921. Google Scholar
  31. Machado, M. A. G., Costa, A. F. B., & Marins, F. A. S. (2009). Control charts for monitoring the mean vector and the covariance matrix of bivariate processes. International Journal of Advanced Manufacturing Technology, 45(7–8), 772–785.CrossRefGoogle Scholar
  32. Quang, A. T., Zhang, Q. L., & Li, X. (2002). Evolving support vector machine parameters. In: Proceedings of the first international conference on machine learning and cybernetics (pp. 548–551). Beijing, China: IEEE.Google Scholar
  33. Ryan, T. P. (2011). Statistical methods for quality improvement (3rd ed.). New York: Wiley.CrossRefGoogle Scholar
  34. Salehi, M., Bahreininejad, A., & Nakhai, I. (2012). On line detection of mean and variance shift using neural networks and support vector machine in multivariate processes. Applied Soft Computing, 12(9), 2973–2984.CrossRefGoogle Scholar
  35. Schapire, R. E. (1990). The strength of weak learnability. Machine Learning, 5(2), 197–227.Google Scholar
  36. Smits, G. F., & Jordaan, E. M. (2002). Improved SVM regression using mixtures of kernels. Proceedings of the 2002 international joint conference on neural networks (pp. 2785–2790). Honolulu: Institute of Electrical and Electronics Engineers Inc.Google Scholar
  37. Takemoto, Y., & Arizono, I. (2005). A study of multivariate \((\overline{{X}}, S) \) control chart based on Kullback–Leibler information. International Journal of Advanced Manufacturing Technology, 25(11–12), 1205–1210.CrossRefGoogle Scholar
  38. Vapnik, V. N. (1999). The nature of statistical learning theory (2nd ed.). New York: Springer.Google Scholar
  39. Wang, T. Y., & Chen, L. H. (2002). Mean shifts detection and classification in multivariate process: A neural-fuzzy approach. Journal of Intelligent Manufacturing, 13(3), 211–221.CrossRefGoogle Scholar
  40. Yang, W.-A. (2013). Monitoring and diagnosing of mean shifts in multivariate manufacturing processes using two-level selective ensemble of learning vector quantization neural networks. Journal of Intelligent Manufacturing. doi: 10.1007/s10845-013-0833-z.
  41. Yang, W.-A., & Zhou, W. (2013). Autoregressive coefficient-invariant control chart pattern recognition in autocorrelated manufacturing processes using neural network ensemble. Journal of Intelligent Manufacturing,. doi: 10.1007/s10845-013-0847-6.Google Scholar
  42. Zhang, G. X., & Chang, S. I. (2008). Multivariate EWMA control charts using individual observations for process mean and variance monitoring and diagnosis. International Journal of Production Research, 46(24), 6855–6881.CrossRefGoogle Scholar
  43. Zhou, Z. H., Wu, J. X., & Tang, W. (2002). Ensembling neural networks: many could be better than all. Artificial Intelligence, 137(1–2), 239–263.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.School of Mechanical and Electrical EngineeringNanjing University of Aeronautics and AstronauticsNanjingPeople’s Republic of China

Personalised recommendations