# A random-weighted plane-Gaussian artificial neural network

- 82 Downloads

## Abstract

Multilayer perceptron (MLP) and radial basis function network (RBFN) have received considerable attentions in data classification and regression. As a bridge between MLP and RBFN, plane-Gaussian (PG) network is capable of exhibiting globality and locality simultaneously by so-called PG activation function. Due to tuning network weight values by back propagation or clustering method in the training phase, they all confront with slow convergence rate, time-consuming, and easily dropping in local minima. To speed training networks, random projection technologies, for instance, extreme learning machine (ELM), have brightened up in recent decades. In this paper, we propose a random-weighted PG network, termed as RwPG. Instead of plane clustering in PG network, our RwPG adopts random values as network weight, and then analytically calculates network output by matrix inversion. Compared to PG and ELM, the advantages of the proposed RwPG list in fourfold: (1) It will be proved that the RwPG is also a universal approximator. (2) It inherits the geometrical interpretation of PG network, and is also suitable for capturing linearity in data, especially for plane distribution cases. (3) It has comparable training speed for ELM, but significantly faster than that of PG network. (4) Owing to random-weighted technology, RwPG is probably capable of breaking through local extremum problems. Finally, experiments on artificial and benchmark datasets will show its superiorities.

## Keywords

Matrix-generalized inverse Plane-Gaussian artificial neural network Random weight## Notes

### Acknowledgements

We would thank the anonymous editors and reviewers for their valuable comments and suggestions. We would thank Dr. Liyong Fu, the professor of Chinese Academy of Forestry, for his academic advice about deep networks in our revisions. This research was supported in part by the Central Public-interest Scientific Institution Basal Research Fund (Grant No. CAFYBB2019QD003), Natural Science Foundation of China under Grant 31670554 and 61871444, the Jiangsu Science Foundation under Grant BK20161527 and BK20171453, and Postgraduate Research and Practice Innovation Program of Jiangsu Province (SJKY19_0907).

### Author contribution

XY proposed learning method and wrote manuscript. HY and ZF designed experiments. XF, FZ, and QY analyzed experimental results and gave some advice for manuscript.

### Compliance with ethical standards

### Conflict of interest

The authors declared that they have no conflicts of interest to this work.

## References

- 1.Lu K, An X, Li J et al (2017) Efficient deep network for vision-based object detection in robotic applications. Neurocomputing 245:31–45CrossRefGoogle Scholar
- 2.Cox DD, Dean T (2014) Neural networks and neuroscience-inspired computer vision. Curr Biol 24(18):921–929CrossRefGoogle Scholar
- 3.Siniscalchi SM, Svendsen T, Lee C (2014) An artificial neural network approach to automatic speech processing. Neurocomputing 140(22):326–338CrossRefGoogle Scholar
- 4.Wu Y, Schuster M, Chen Z et al (2016) Google’s neural machine translation system: bridging the gap between human and machine translation. CoRR. Technical report, available at http://arxiv.org/abs/1609.08144
- 5.Varshney D, kumar S, Gupta V (2017) Predicting information diffusion probabilities in social networks: a Bayesian networks based approach. Knowl-Based Syst 133:66–76CrossRefGoogle Scholar
- 6.Mishra J, Anguera JA, Gazzaley A (2016) Video games for neuro-cognitive optimization. Neuron 90(2):214–218CrossRefGoogle Scholar
- 7.Yang X, Chen S, Chen B (2012) Plane-Gaussian artificial neural network. Neural Comput Appl 21(2):305–317CrossRefGoogle Scholar
- 8.Bradley PS, Mangasarian OL (2000) k-Plane Clustering. J Glob Optim 16(1):23–32MathSciNetzbMATHCrossRefGoogle Scholar
- 9.Bengio Y. (2012) Practical recommendations for gradient-based training of deep architectures. In: LNCS neural networks: tricks of the trade 2nd ed. Springer, Berlin, pp 437–478Google Scholar
- 10.Bengio Y, Courville A, Vincent P (2012) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828CrossRefGoogle Scholar
- 11.Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceeding of the 27th international conference on machine learning (ICML). Omnipress, Haifa, Israel, Wahpeton, ND, USA, June 21–24, pp 807–814Google Scholar
- 12.Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. NIPS 25:1097–1105Google Scholar
- 13.Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. NIPS 28:2377–2385Google Scholar
- 14.He KM, Zhang XY, Ren SQ et al (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE Press, Las Vegas, NV, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
- 15.He KM, Zhang XY, Ren SQ et al (2016) Identity mapping in deep residual networks. In: Proceedings of European conference on computer vision (ECCV), Amsterdam, Netherlands, October 8–16, pp 630–645. arXiv:1603.05027v3
- 16.Guo P, Zhou XL, Wang K (2018) PILAE: a non-gradient descent learning scheme for deep feedforward neural networks. arXiv:1811.01545
- 17.Pang S, Yang X (2016) Deep convolutional extreme learning machine and its application in handwritten digit classification. Comput Intell Neurosci. https://doi.org/10.1155/2016/3049632 CrossRefGoogle Scholar
- 18.Michel M, Abel G, Wellington P (2019) Deep convolutional extreme learning machines: filters combination and error model validation. Neurocomputing 329:359–369CrossRefGoogle Scholar
- 19.Tissera M, McDonnell M (2016) Deep extreme learning machines: supervised autoencoding architecture for classification. Neurocomputing 174(Part A):42–49CrossRefGoogle Scholar
- 20.Duan M, Li K, Yang C et al (2018) A hybrid deep learning CNN–ELM for age and gender classification. Neurocompuing 275:448–461CrossRefGoogle Scholar
- 21.Li J, Zhao X, Li Y et al (2018) Classification of hyperspectral imagery using a new fully convolutional neural network. IEEE Geosci Remote Sens Lett 99:1–5Google Scholar
- 22.Cao W, Wang X, Ming Z et al (2018) A review on neural networks with random weights. Neurocomputing 275:278–287CrossRefGoogle Scholar
- 23.Schmidt W, Kraaijveld M, Duin R (1992) Feedforward neural networks with random weights. In: Proceedings of 11th IAPR international conference on pattern recognition methodology and systems, vol 2, pp 1–4Google Scholar
- 24.Deng C, Huang G, Xu J, Tang J (2015) Extreme learning machines: new trends and applications. Sci China Inf Sci 58(2):1–16CrossRefGoogle Scholar
- 25.Huang G, Chen L, Siew C (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRefGoogle Scholar
- 26.Huang G (2015) What are extreme learning machines? Filling the gap between Frank Rosenblatt’s dream and John von Neumann’s puzzle. Cognit Comput 7:263–278CrossRefGoogle Scholar
- 27.Igelnik B, Pao Y-H (1995) Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans Neural Netw 6(6):1320–1329CrossRefGoogle Scholar
- 28.Li JY, Chow W, Igenik B, Pao YH (1997) Comments on “Stochastic choice of basis functions in adaptive function approximation and the functional-link net”. IEEE Trans Neural Netw 8(2):452–454Google Scholar
- 29.Kasun L, Zhou H, Huang G-B, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34Google Scholar
- 30.Huang G, Zhou H, Ding X et al (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B 42(2):513–529CrossRefGoogle Scholar
- 31.Dua D, Taniskidou EK (2017) UCI machine learning repository (http://archive.ics.uci.edu/ml). University of California, School of Information and Computer Science, Irvine
- 32.Moore AW, Crogan ML (2005) Discriminators for use in flow-based classification. Research reports: RR-05-13, Department of Computer Science, Queen Mary, University of LondonGoogle Scholar
- 33.Mygdalis V, Iosifidis A, Tefas A et al (2018) Semi-supervised subclass support vector data description for image and video classification. Neurocomputing 278:51–61CrossRefGoogle Scholar
- 34.Maronidis A, Tefas A, Pitas I (2015) Subclass graph embedding and a marginal fisher analysis paradigm. Pattern Recognit 48(12):4024–4035zbMATHCrossRefGoogle Scholar
- 35.Wan H, Wang H, Guo G et al (2018) Seperability-oriented subclass discriminant analysis. IEEE Trans Pattern Anal Mach Intell 40(2):409–422CrossRefGoogle Scholar
- 36.Baum EB, Haussler D (2014) What size net gives valid generalization? Neural Comput 1(1):151–160CrossRefGoogle Scholar