Deep Learning for Soft Sensor Design

  • Salvatore GrazianiEmail author
  • Maria Gabriella Xibilia
Part of the Studies in Computational Intelligence book series (SCI, volume 867)


Soft Sensors are mathematical models used to predict the behavior of real systems. They are usefully applied to estimate hard-to-measure quantities in the process industry. Many Soft Sensors are designed by using data-driven approaches and exploiting historical databases. Machine learning is widely used for this aim. Here, the potentialities of deep learning in solving some challenges raising in industrial applications are introduced. More specifically, the paper focuses on three specific aspects: labelled data scarcity, computational complexity reduction, and unsupervised feature exploitation. The state of the art of Soft Sensors based on deep learning is described. Then, the focus is on Soft Sensors based on Deep Belief Networks, as a research field that the authors have been investigating since years. The improvements offered by Deep Belief Networks, over more conventional data-driven approaches, in designing Soft Sensors for real-world applications will be shown. Soft Sensors for specific cases study are described.


Soft sensors System identification Deep learning Deep belief networks Semi-supervised learning Industrial applications 


  1. 1.
    Kadlec, P., Gabrys, G., Strandt, S.: Data-driven Soft sensors in the process industry. Comp. Chem. Eng 33, 795–814 (2009)CrossRefGoogle Scholar
  2. 2.
    Fortuna, L., Graziani, S., Rizzo, A., Xibilia, M.G.: Soft sensors for monitoring and control of industrial processes. Advances in Industrial Control, Springer (2006)Google Scholar
  3. 3.
    Ge, Z., Huang, B., Song, Z.: Nonlinear semi-supervised principal component regression for soft sensor modeling and its mixture form. J. Chemom. 28, 793–804 (2014)CrossRefGoogle Scholar
  4. 4.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016)Google Scholar
  6. 6.
    Hinton, G.E.: Learning multiple layers of representation. Trends Cognit. Sci. 11, 428–434 (2007)CrossRefGoogle Scholar
  7. 7.
    Yu, J., Hong, C., Rui, Y., Tao, D.: Multi-task autoencoder model for recovering human poses. IEEE Trans. Ind. Electron. 65(6), 5060–5068 (2018)CrossRefGoogle Scholar
  8. 8.
    Lee, S., Chang, J.H.: Oscillometric blood pressure estimation based on deep learning. IEEE Trans. Ind. Inf. 13(2), 461–472 (2017)CrossRefGoogle Scholar
  9. 9.
    Samek, W., Binder, A., Montavon, G., Lapuschkin, S., Muller, K.R.: Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Netw. Learn. Syst. 28(11), 2660–2673 (2017)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Ren, S., Girshick, R., Girshick, R, Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)CrossRefGoogle Scholar
  11. 11.
    Xibilia, M.G., Gemelli, N., Consolo, G.: Input variables selection criteria for data-driven soft sensors design. In: Proceedings of 2017 IEEE 14th International Conference on Networking, Sensing and Control (ICNSC) 16–18 May 2017, Lamezia Terme, Italy (2017)Google Scholar
  12. 12.
    Yuan, X., Huang, B., Wang, Y., Yang, C., Gui, W.: Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE. IEEE Trans. Ind. Inf., in pressGoogle Scholar
  13. 13.
    Yuan, X., Ou, C., Wang, Y., Yang, C..: Nonlinear VW-SAE based deep learning for quality-related feature learning and soft sensor modeling. In: Proceedings of IECON 2018—44th Annual Conference of the IEEE Industrial Electronics Society, 21–23 Oct. (2018)Google Scholar
  14. 14.
    Wang, X., Liu, H.: A new input variable selection method for soft sensor based on stacked auto-encoders. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), December 12–15, 2017, Melbourne, Australia (2017)Google Scholar
  15. 15.
    Wang, J.G., Zhao, J.H., Shen, T., Ma, S.W., Yao, Y., Chen, T., Shen, B., Wu, Y.: Deep learning-based soft-sensing method for operation optimization of coke dry quenching process. In: Proceedings of the 35th Chinese Control Conference July 27–29, 2016, Chengdu, China (2016)Google Scholar
  16. 16.
    Yan, W., Tang, D., Lin, Y.: A data-driven soft sensor modeling method based on deep learning and its application. IEEE. Trans. Ind. Electron. 64(5), 4237–4245 (2017)CrossRefGoogle Scholar
  17. 17.
    Lin, Y., Yan, W.: Study of soft sensor modeling based on deep learning. In: Proceedings of the 2015 American Control Conference, July 1–3, 2015. Chicago, IL, USA (2015)Google Scholar
  18. 18.
    Shang, C., Yang, F., Huang, D., Lyu, W.: Data-driven soft sensor development based on deep learning technique. J. Process Control 24(3), 223–233 (2014)CrossRefGoogle Scholar
  19. 19.
    Liu, Y., Fan, Y., Chen, J.: Flame images for oxygen content prediction of combustion systems using DBN. Energy Fuels 31, 8776–8783 (2017)CrossRefGoogle Scholar
  20. 20.
    Andò, B., Graziani, S., Xibilia, M.G.: Low-order nonlinear finite impulse response soft sensors for ionic electroactive actuators based on deep learning. IEEE Trans. Instrum. Meas. 68(5), 1637–1646 (2019)CrossRefGoogle Scholar
  21. 21.
    Graziani, S., Xibilia, M.G.: A deep learning based soft sensor for a sour water stripping plant. In: Proceedings of IEEE I2MTC 2017, Torino, Italy, 22–25 May (2018)Google Scholar
  22. 22.
    Graziani, S., Xibilia, M.G.: Deep Structures for a reformer unit soft sensor. In: Proceedings of INDIN 2018, IEEE 16th International Conference on Industrial Informatics, Porto, Portugal, 18–20 July (2018)Google Scholar
  23. 23.
    Zhu, W., Ma, Y., Zhou, Y., Benton, M., Romagnoli, J.: Deep learning based soft sensor and its application on a pyrolysis reactor for compositions predictions of gas phase components. Comput. Aided Chem. Eng. 44, 2245–2250 (2018)CrossRefGoogle Scholar
  24. 24.
    Sun, S.B., He, Y.Y., Zhou, S.D., Yue, Z.J.: A data-driven response virtual sensor technique with partial vibration measurements using convolutional neural network. Sensors 17, 12 (2017)Google Scholar
  25. 25.
    De la Rosa, E., Yu, W.: Randomized algorithms for nonlinear system identification with deep learning modification. Inf. Sci. 364–365, 197–212 (2016)CrossRefGoogle Scholar
  26. 26.
    Yao, L., Ge, Z.: Deep learning of semi-supervised process data with hierarchical extreme learning machine and soft sensor application. IEEE. Trans. Ind. Electron. 65(2), 1490–1498 (2018)CrossRefGoogle Scholar
  27. 27.
    Liu, Y., Yang, C., Gao, Z., Yao, Y.: Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes. Chemom. Intell. Lab. Syst. 174, 15–21 (2018)CrossRefGoogle Scholar
  28. 28.
    Shao, W., Tian, X.: Semi-supervised selective ensemble learning based on distance to model for nonlinear soft sensor development. Neurocomputing 222, 15–21 (2017)CrossRefGoogle Scholar
  29. 29.
    Qiu, Y., Liu, Y., Huang, D.: Data-driven soft-sensor design for biological wastewater treatment using deep neural networks and genetic algorithms. J. Chem. Eng. Jpn. 49(10), 925–936 (2016)CrossRefGoogle Scholar
  30. 30.
    Sun, Q., Ge, Z.: Probabilistic sequential network for deep learning of complex process data and soft sensor application. IEEE Trans. Ind. Inf., in pressGoogle Scholar
  31. 31.
    Fischer, A., Igel, C.: Training restricted Boltzmann machines: an introduction. Pattern Recogn. 47(1), 25–39 (2014)CrossRefGoogle Scholar
  32. 32.
    Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends Signal Process. 7(3–4), 197–387 (2013)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Keyvanrad M.A., Homayounpour, M.M.: A brief survey on deep belief networks and introducing a new object-oriented toolbox (DeeBNet). arXiv:1408.3264 [cs] (2014)
  34. 34.
    Fortuna, L., Giannone, P., Graziani, S., Xibilia, M.G.: Virtual instruments based on stacked neural networks to improve product quality monitoring in a refinery. IEEE Trans. Instrum. Meas. 56, 95–101 (2007)CrossRefGoogle Scholar
  35. 35.
    De Luca, V., Digiamberardino, P., Di Pasquale, G., Graziani, S., Pollicino, A., Umana, E., Xibilia, M.G.: Ionic electroactive polymer metal composites: fabricating, modeling, and applications of postsilicon smart devices. J. Polym. Sci., Part B: Polym. Phys. 51, 699–734 (2013)CrossRefGoogle Scholar
  36. 36.
    Shahinpoor, M., Kim, K. J.: Ionic polymer–metal composites: IV. Industrial and medical applications. Smart Mater. Struct. 14, 197 (2005)CrossRefGoogle Scholar
  37. 37.
    Newbury, K.M., Leo, D.J.: Linear electromechanical model of ionic polymer transducers–Part II: experimental validation. J. Intel. Mat. Sys. Struct. 14, 343–358 (2003)CrossRefGoogle Scholar
  38. 38.
    Bonomo, C., Fortuna, L., Giannone, P., Graziani, S., Strazzeri, S.: A nonlinear model for ionic polymer metal composites as actuators. Smart Mat. Struct. 16, 1–12 (2007)CrossRefGoogle Scholar
  39. 39.
    Fortuna, L., Graziani, S., Xibilia, M.G.: Virtual instruments in refineries. IEEE Instrum. Meas. Mag. 8, 26–34 (2005)CrossRefGoogle Scholar
  40. 40.
    Fortuna, L., Graziani, S., Xibilia, M.G.: Soft sensors for product quality monitoring in debutanizer distillation columns. Control Eng. Pract. 13, 499–508 (2005)CrossRefGoogle Scholar
  41. 41.
    Graziani, S., Xibilia, M.G.: Design of a soft sensor for an industrial plant with unknown delay by using deep learning. In: Proceedings IEEE I2MTC 2019, Auckland, New Zealand, 20–23 May (2019)Google Scholar
  42. 42.
    Fortuna, L., Graziani, S., Xibilia, M.G., Barbalace, N.: Fuzzy activated neural models for product quality monitoring in refineries. IFAC Proc. Volumes 16, 159–164 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.DIEEIUniversity of CataniaCataniaItaly
  2. 2.Department of EngineeringUniversity of MessinaMessinaItaly

Personalised recommendations