Least-Squares Regression with Unitary Constraints for Network Behaviour Classification

  • Antonio Robles-KellyEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10029)


In this paper, we propose a least-squares regression method [2] with unitary constraints with applications to classification and recognition. To do this, we employ a kernel to map the input instances to a feature space on a sphere. In a similar fashion, we view the labels associated with the training data as points which have been mapped onto a Stiefel manifold using random rotations. In this manner, the least-squares problem becomes that of finding the span and kernel parameter matrices that minimise the distance between the embedded labels and the instances on the Stiefel manifold under consideration. We show the effectiveness of our approach as compared to alternatives elsewhere in the literature for classification on synthetic data and network behaviour log data, where we present results on attack identification and network status prediction.


Support Vector Machine Radial Basis Function Linear Discriminant Analysis Network Behaviour Attack Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Math. 80(2), 199–220 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Anderson, T.W.: Estimating linear restrictions on regression coefficients for multivariate normal distributions. Ann. Math. Stat. 22(3), 327–351 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Comput. 12(10), 2385–2404 (2000)CrossRefGoogle Scholar
  4. 4.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Neural Information Processing Systems, vol. 14, pp. 634–640 (2002)Google Scholar
  5. 5.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  6. 6.
    Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, Cambridge (1975)zbMATHGoogle Scholar
  7. 7.
    Borg, I., Groenen, P.: Modern Multidimensional Scaling, Theory and Applications. Springer Series in Statistics. Springer, Heidelberg (1997)CrossRefzbMATHGoogle Scholar
  8. 8.
    Coleman, T.F., Li, Y.: An interior, trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  10. 10.
    De la Torre, F.: A least-squares framework for component analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1041–1055 (2012)CrossRefGoogle Scholar
  11. 11.
    Diaconis, P., Shahshahani, M.: The subgroup algorithm for generating uniform random variables. Probab. Eng. Inf. Sci. 1, 15–32 (1987)CrossRefzbMATHGoogle Scholar
  12. 12.
    Duda, R.O., Hart, P.E.: Pattern Classification. Wiley, Hoboken (2000)zbMATHGoogle Scholar
  13. 13.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugenics 7, 179–188 (1936)CrossRefGoogle Scholar
  14. 14.
    Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33(1), 1–22 (2010)CrossRefGoogle Scholar
  15. 15.
    Fu, Z., Robles-Kelly, A., Tan, R.T., Caelli, T.: Invariant object material identification via discriminant learning on absorption features. In: Object Tracking and Classification in and Beyond the Visible Spectrum (2006)Google Scholar
  16. 16.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, Cambridge (1990)zbMATHGoogle Scholar
  17. 17.
    Hassoun, M.H.: Fundamentals of Artificial Neural Networks. MIT Press, Cambridge (1995)zbMATHGoogle Scholar
  18. 18.
    James, I.M.: The Topology of Stiefel Manifolds. Cambridge University Press, Cambridge (1976)zbMATHGoogle Scholar
  19. 19.
    Khan, L., Awad, M., Thuraisingham, B.: A new intrusion detection system using support vector machines and hierarchical clustering. Int. J. Very Large Data Bases 16(4), 507–521 (2007)CrossRefGoogle Scholar
  20. 20.
    Landgrebe, D.: Hyperspectral image data analysis. IEEE Sig. Process. Mag. 19, 17–28 (2002)CrossRefGoogle Scholar
  21. 21.
    Li, H., Jiang, T., Zhang, K.: Efficient and robust feature extraction by maximum margin criterion. In: Neural Information Processing Systems, vol. 16 (2003)Google Scholar
  22. 22.
    Ma, Y., Fu, Y.: Manifold Learning Theory and Applications. CRC Press, Inc., Boca Raton (2011)Google Scholar
  23. 23.
    Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Muller, K.: Fisher discriminant analysis with kernels. In: IEEE Neural Networks for Signal Processing Workshop, pp. 41–48 (1999)Google Scholar
  24. 24.
    Neal, R.M.: Bayesian Learning for Neural Networks. Springer, New York (1996)CrossRefzbMATHGoogle Scholar
  25. 25.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  26. 26.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)CrossRefGoogle Scholar
  27. 27.
    Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)CrossRefzbMATHGoogle Scholar
  28. 28.
    Tang, Y., Salakhutdinov, R.R.: Learning stochastic feedforward neural networks. In: Advances in Neural Information Processing Systems, pp. 530–538 (2013)Google Scholar
  29. 29.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  30. 30.
    Van Gestel, T., Suykens, J.A.K., De Brabanter, J., Lambrechts, A., De Moor, B., Vandewalle, J.: Bayesian framework for least-squares support vector machine classifiers, Gaussian processes, and kernel fisher discriminant analysis. Neural Comput. 14(5), 1115–1147 (2002)CrossRefzbMATHGoogle Scholar
  31. 31.
    Wong, Y.C.: Differential geometry of Grassmann manifolds. Proc. Natl. Acad. Sci. United States Am. 57(3), 589–594 (1967)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.DATA61 - CSIROCanberra ACTAustralia
  2. 2.College of Engineering and Computer ScienceAustralian National UniversityCanberraAustralia

Personalised recommendations