Unsupervised Parameter Estimation of Non Linear Scaling for Improved Classification in the Dissimilarity Space

  • Mauricio Orozco-AlzateEmail author
  • Robert P. W. Duin
  • Manuele Bicego
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10029)


The non-linear scaling of given dissimilarities, by raising them to a power in the (0,1) interval, is often useful to improve the classification performance in the corresponding dissimilarity space. The optimal value for the power can be found by a grid search across a leave-one-out cross validation of the classifier: a procedure that might become costly for large dissimilarity matrices, and is based on labels, not permitting to capture the global effect of such a scaling. Herein, we propose an entirely unsupervised criterion that, when optimized, leads to a suboptimal but often good enough value of the scaling power. The criterion is based on a trade-off between the dispersion of data in the dissimilarity space and the corresponding intrinsic dimensionality, such that the concentrating effects of the power transformation on both the space axes and the spatial distribution of the objects are rationed.


Dissimilarity space Intrinsic dimensionality Dispersion Non linear scaling Nearest neighbor classification Power transformation 



Discussions for the proposal in this paper started while Mauricio Orozco-Alzate and Manuele Bicego visited the Pattern Recognition Laboratory, Delft University of Technology (Delft, The Netherlands) in September 2015 by a kind invitation from Robert P.W. Duin to attend the “Colors of dissimilarities” workshop.

This material is based upon work supported by Universidad Nacional de Colombia under project No. 32059 (Code Hermes) entitled “Consolidación de las líneas de investigación del Grupo de Investigación en Ambientes Inteligentes Adaptativos GAIA” within “Convocatoria interna de investigación de la Facultad de Administración 2015, para la formulación y ejecución de proyectos de consolidación y/o fortalecimiento de los grupos de investigación. Modalidad 1: Formulación y ejecución de proyectos de consolidación”.

The fist author also acknowledges travel funding to attend S+SSPR 2016 provided by Universidad Nacional de Colombia through “Convocatoria para la Movilidad Internacional de la Universidad Nacional de Colombia 2016–2018. Modalidad 2: Cofinanciación de docentes investigadores o creadores de la Universidad Nacional de Colombia para la presentación de resultados de investigación o representaciones artísticas en eventos de carácter internacional, o para la participación en megaproyectos y concursos internacionales, o para estancias de investigación o residencias artísticas en el extranjero”.


  1. 1.
    Bicego, M., Baldo, S.: Properties of the Box-Cox transformation for pattern classification. Neurocomputing (2016, in press)Google Scholar
  2. 2.
    Camastra, F.: Data dimensionality estimation methods: a survey. Pattern Recogn. 36(12), 2945–2954 (2003)CrossRefzbMATHGoogle Scholar
  3. 3.
    Camastra, F., Staiano, A.: Intrinsic dimension estimation: advances and open problems. Inf. Sci. 328, 26–41 (2016)CrossRefGoogle Scholar
  4. 4.
    Carli, A.C., Bicego, M., Baldo, S., Murino, V.: Non-linear generative embeddings for kernels on latent variable models. In: Proceedings of ICCV 2009 Workshop on Subspace Methods, pp. 154–161 (2009)Google Scholar
  5. 5.
    Carli, A.C., Bicego, M., Baldo, S., Murino, V.: Nonlinear mappings for generative kernels on latent variable models. In: Proceedings of International Conference on Pattern Recognition, pp. 2134–2137 (2010)Google Scholar
  6. 6.
    Duin, R.P.W., Bicego, M., Orozco-Alzate, M., Kim, S.-W., Loog, M.: Metric learning in dissimilarity space for improved nearest neighbor performance. In: Fränti, P., Brown, G., Loog, M., Escolano, F., Pelillo, M. (eds.) S+SSPR 2014. LNCS, vol. 8621, pp. 183–192. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-44415-3_19 Google Scholar
  7. 7.
    Duin, R.P.W., Pekalska, E.: The dissimilarity space: bridging structural and statistical pattern recognition. Pattern Recogn. Lett. 33(7), 826–832 (2012)CrossRefGoogle Scholar
  8. 8.
    Fahmy, A.A.: Using the Bees Algorithm to select the optimal speed parameters for wind turbine generators. J. King Saud Univ. Comput. Inf. Sci. 24(1), 17–26 (2012)Google Scholar
  9. 9.
    Guillot, D., Khare, A., Rajaratnam, B.: Complete characterization of Hadamard powers preserving Loewner positivity, monotonicity, and convexity. J. Math. Anal. Appl. 425(1), 489–507 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Kokoska, S., Zwillinger, D.: CRC Standard Probability and Statistics Tables and Formulae, Student edn. CRC Press, Boca Raton (2000)CrossRefzbMATHGoogle Scholar
  11. 11.
    Liu, C.L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: investigation of normalization and feature extraction techniques. Pattern Recogn. 37(2), 265–279 (2004)CrossRefzbMATHGoogle Scholar
  12. 12.
    Mariani, G., Palermo, G., Zaccaria, V., Silvano, C.: OSCAR: an optimization methodology exploiting spatial correlation in multicore design spaces. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 31(5), 740–753 (2012)CrossRefGoogle Scholar
  13. 13.
    Pekalska, E., Duin, R.P.W.: The Dissimilarity Representation for Pattern Recognition: Foundations and Applications (Machine Perception and Artificial Intelligence), vol. 64. World Scientific, Singapore (2005)zbMATHGoogle Scholar
  14. 14.
    Pelillo, M. (ed.): Similarity-Based Pattern Analysis and Recognition. Advances in Computer Vision and Pattern Recognition. Springer, London (2013)zbMATHGoogle Scholar
  15. 15.
    Pelillo, M., Hancock, E.R., Feragen, A., Loog, M. (eds.): SIMBAD. LNCS, vols. 7005, 7953, 9370. Springer, Heidelberg (2011, 2013, 2015)Google Scholar
  16. 16.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press, London (2009)zbMATHGoogle Scholar
  17. 17.
    Van Der Heiden, R., Groen, F.C.A.: The Box-Cox metric for nearest neighbour classification improvement. Pattern Recogn. 30(2), 273–279 (1997)CrossRefGoogle Scholar
  18. 18.
    Vapnik, V.N.: Statistical Learning Theory. Adaptive and Learning Systems for Signal Processing, Communication and Control. Wiley, New York (1998)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Mauricio Orozco-Alzate
    • 1
    Email author
  • Robert P. W. Duin
    • 2
  • Manuele Bicego
    • 3
  1. 1.Departamento de Informática y ComputaciónUniversidad Nacional de Colombia, Sede ManizalesManizalesColombia
  2. 2.Pattern Recognition LaboratoryDelft University of TechnologyDelftThe Netherlands
  3. 3.Dipartimento di InformaticaUniversità degli Studi di Verona, Cá Vignal 2VeronaItaly

Personalised recommendations