Abstract
Recently some novel strategies have been proposed for training of Single Hidden Layer Feedforward Networks, that set randomly the weights from input to hidden layer, while weights from hidden to output layer are analytically determined by Moore-Penrose generalised inverse. Such non-iterative strategies are appealing since they allow fast learning, but some care may be required to achieve good results, mainly concerning the procedure used for matrix pseudoinversion. This paper proposes a novel approach based on original determination of the initialization interval for input weights, a careful choice of hidden layer activation functions and on critical use of generalised inverse to determine output weights. We show that this key step suffers from numerical problems related to matrix invertibility, and we propose a heuristic procedure for bringing more robustness to the method. We report results on a difficult astronomical image analysis problem of chromaticity diagnosis to illustrate the various points under study.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
LeCun, Y.A., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient BackProp. In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, pp. 9–50. Springer, Heidelberg (1998)
Halawa, K.: A method to improve the performance of multilayer perceptron by utilizing various activation functions in the last hidden layer and the least squares method. Neural Processing Letters 34, 293–303 (2011)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme Learning Machine: Theory and applications. Neurocomputing 70, 489–501 (2006)
Nguyen, T.D., Pham, H.T.B., Dang, V.H.: An efficient Pseudo Inverse matrix-based solution for secure auditing. In: IEEE International Conference on Computing and Communication Technologies, Research, Innovation, and Vision for the Future (2010)
Kohno, K., Kawamoto, M., Inouye, Y.: A Matrix Pseudoinversion Lemma and its Application to Block-Based Adaptive Blind Deconvolution for MIMO Systems. IEEE Transactions on Circuits and Systems I 57(7), 1449–1462 (2010)
Ajorloo, H., Manzuri-Shalmani, M.T., Lakdashti, A.: Restoration of Damaged Slices in Images Using Matrix Pseudo Inversion. In: 22th International Symposium on Computer and Information Sciences, Ankara, pp. 98–104 (2007)
Bini, D., Capovani, M., Menchi, O.: Metodi numerici per l’algebra lineare. Ed. Zanichelli, Bologna (1988)
Bishop, C. M.: Pattern Recognition and Machine Learning. Ed. Springer, Berlin (2006)
Bartlett, P.L.: The sample complexity of pattern classification with neural networks: the size of the weights is more important that the size of the network. IEEE Trans. Inf. Theory 44(2), 525–536 (1998)
Ortega, J.M.: Matrix Theory. Plenum Press, New York (1987)
Golub, G., van Loan, C.: Matrix computations. The Johns Hopkins University Press, London (1996)
Le Gall, J.Y., Saisse, M.: Chromatic Aberration of an All-Reflective Telescope. In: Instrumentation in Astronomy V, London. SPIE, vol. 445, pp. 497–504 (1983)
Perryman, M.A.C., et al.: GAIA - Composition, formation and evolution of the galaxy. Concept and technology study, Rep. and Exec. Summary. In: European Space Agency, ESA-SCI, Munich, Germany, vol. 4 (2000)
Gai, M., Cancelliere, R.: Neural network correction of astrometric chromaticity. MNRAS 362(4), 1483–1488 (2005)
Cancelliere, R., Gai, M.: Efficient computation and Neural Processing of Astrometric Images. Computing and Informatics 28, 711–727 (2009)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: 13th International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia, Italy (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cancelliere, R., Gai, M., Artières, T., Gallinari, P. (2012). Matrix Pseudoinversion for Image Neural Processing. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7667. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34500-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-34500-5_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34499-2
Online ISBN: 978-3-642-34500-5
eBook Packages: Computer ScienceComputer Science (R0)