Abstract
We present a new self-organizing algorithm for classification of a data that combines and extends the strengths of several common machine learning algorithms, such as algorithms in self-organizing neural networks, ensemble methods and deep neural networks. The increased expression power is combined with the explanation power of self-organizing networks. Our algorithm outperforms both deep neural networks and ensembles of deep neural networks. For our evaluation case, we use production monitoring data from a complex steel manufacturing process, where data is both high-dimensional and has many nonlinear interdependencies. In addition to the improved prediction score, the algorithm offers a new deep-learning based approach for how computational resources can be focused in data exploration, since the algorithm points out areas of the input space that are more challenging to learn.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This work was supported by Vinnova and Jernkontoret under the project Dataflow. We would like to thank Andreas Persson at Outokumpu AB for the valuable collaboration.
References
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). ISSN 1573–0565. https://doi.org/10.1023/A:1010933404324
Cuadrado, A.A., Diaz, I., Diez, A.B., Obeso, F., Gonzalez, J.A.: Visual data mining and monitoring in steel processes. In: Conference Record of the 2002 IEEE Industry Applications Conference, 37th IAS Annual Meeting (Cat. No. 02CH37344), vol. 1, pp. 493–500 (2002). https://doi.org/10.1109/IAS.2002.1044131
Fritzke, B.: Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1994). https://doi.org/10.1016/0893-6080(94)90091-4
Fritzke, B.: A growing neural gas network learns topologies. In: Advances in Neural Information Processing Systems, pp. 625–632 (1995)
Fritzke, B.: Growing self-organizing networks-why? In: ESANN 1996, pp. 61–72 (1996)
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990). https://doi.org/10.1109/34.58871
He, H., Garcia, E.A.: A learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009). https://doi.org/10.1109/TKDE.2008.239
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kohonen, T.: Self-organized formation of topologically correct feature maps. Biol. Cybern. 43(1), 59–69 (1982). https://doi.org/10.1007/BF00337288
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539
Leshno, M., Lin, V.Y., Pinkus, A., Schocken, S.: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw. 6(6), 861–867 (1993). https://doi.org/10.1016/S0893-6080(05)80131-5
van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Martinetz, T.M., Berkovich, S.G., Schulten, K.J.: ‘Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Trans. Neural Netw. 4(4), 558–569 (1993)
Opitz, D.W., Maclin, R.: Popular ensemble methods. J. Artif. Intell. Res. (JAIR) 11, 169–198 (1999)
Poole, B., Lahiri, S., Raghu, M., Sohl-Dickstein, J., Ganguli, S.: Exponential expressivity in deep neural networks through transient chaos. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29, pp. 3360–3368. Curran Associates Inc. (2016)
Sagiroglu, S., Sinanc, D.: Big data: a review. In: 2013 International Conference on Collaboration Technologies and Systems (CTS), pp. 42–47. IEEE (2013). https://doi.org/10.1109/CTS.2013.6567202
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Ståhl, N., Falkman, G., Mathiason, G., Karlsson, A. (2018). A Self-Organizing Ensemble of Deep Neural Networks for the Classification of Data from Complex Processes. In: Medina, J., Ojeda-Aciego, M., Verdegay, J., Perfilieva, I., Bouchon-Meunier, B., Yager, R. (eds) Information Processing and Management of Uncertainty in Knowledge-Based Systems. Applications. IPMU 2018. Communications in Computer and Information Science, vol 855. Springer, Cham. https://doi.org/10.1007/978-3-319-91479-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-91479-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91478-7
Online ISBN: 978-3-319-91479-4
eBook Packages: Computer ScienceComputer Science (R0)