Advertisement

Nonlinear Multidimensional Data Projection and Visualisation

  • Hujun Yin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2690)

Abstract

Multidimensional data projection and visualisation are becoming increasingly important and have found wide applications in many fields such as decision support, bioinformatics and web/document organisation. Various methods and algorithms have been proposed as either nonparametric or semiparametric approaches. This paper provides an overview of the subject and reviews some recent developments. Relationships among various key methods such as Sammon mapping, Neuroscale, principal curve/surface, SOM, GTM and ViSOM are analysed and their advantages and limitations are highlighted in the context of nonlinear principal component analysis and independent component analysis.

Keywords

Independent Component Analysis Blind Source Separation Kernel Principal Component Analysis Nonlinear Principal Component Analysis Generative Topographic Mapping 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arciniegas, I., Daniel, B., Embrechts, M.J.: Exploring Financial Crises Data with selforganising maps. In: Allinson, N., Yin, H., Allinson, L., Slack, J. (eds.) Advances in Self-Organising Maps, pp. 39–46 (2001)Google Scholar
  2. 2.
    Banfield, J.D., Raftery, A.E.: Ice floe identification in satellite images using mathematical morphology and clustering about principal curves. Journal of the American Statistical Association 87, 7–16 (1992)CrossRefGoogle Scholar
  3. 3.
    Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Computation 10, 215–235 (1998)CrossRefGoogle Scholar
  4. 4.
    Bishop, C.M., Svensén, M., Williams, C.K.I.: Magnification factors for the SOM and GTM algorithms. In: Proceedings of Workshop on Self-Organizing Maps (WSOM 1997), pp. 333–338 (1997)Google Scholar
  5. 5.
    Biswas, G., Jain, A.K., Dubes, R.C.: Evaluation of project algorithms. IEEE Trans. On Pattern Analysis and Machine Intelligence PAMI-3, 701–708 (1981)CrossRefGoogle Scholar
  6. 6.
    Burel, G.: Blind separation of sources: A nonlinear neural algorithm. Neural Networks 5, 937–947 (1992)CrossRefGoogle Scholar
  7. 7.
    Chang, K.-Y., Ghosh, J.: A unified model for probabilistic principal surfaces. IEEE Trans. on Pattern Analysis and Machine Intelligence PAMI-23, 22–41 (2001)CrossRefGoogle Scholar
  8. 8.
    Condon, E., Golden, B., Lele, S., Raghavan, S., Wasil, E.: A visualization model based on adjacency data. Decision Support systems 33, 349–362 (2002)CrossRefGoogle Scholar
  9. 9.
    Cox, T.F., Cox, M.A.A.: Multidimensional Scaling. Chapman & Hall, Boca Raton (1994)zbMATHGoogle Scholar
  10. 10.
    De Ridder, D., Duin, R.P.W.: Sammon mapping using neural networks: a comparison. Pattern Recognition Letters 18, 1307–1316 (1997)CrossRefGoogle Scholar
  11. 11.
    Freeman, R., Yin, H.: Self-organising maps for hierarchical tree view document clustering using contextual information. In: Yin, H., Allinson, N.M., Freeman, R., Keane, J.A., Hubbard, S. (eds.) IDEAL 2002. LNCS, vol. 2412, pp. 123–128. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  12. 12.
    Girolami, M.: Self-Organising Neural Networks: Independent Component Analysis and Blind Source Separation. Springer, Heidelberg (1999)Google Scholar
  13. 13.
    Haritopoulos, M., Yin, H., Allinson, N.M.: Image denoising using self-organising map –based nonlinear independent component analysis. Neural Networks 15, 1085–1098 (2002)CrossRefGoogle Scholar
  14. 14.
    Hastie, T., Stuetzle, W.: Principal curves. Journal of the American Statistical Association 84, 502–516 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Herrmann, M., Yang, H.H.: Perspectives and limitations of self-organising maps inblind separation of source signals. In: Proc. ICONIP 1996, pp. 1211–1216 (1996)Google Scholar
  16. 16.
    Honkela, T., Kaski, S., Lagus, K., Kohonen, T.: WEBSOM-self-organizing maps of document collections. In: Proceedings of Workshop on Self-Organizing Maps (WSOM 1997), pp. 310–315 (1997)Google Scholar
  17. 17.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley &Sons, Inc. Chichester (2001)CrossRefGoogle Scholar
  18. 18.
    Hyvärinen, A., Pajunen, P.: Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks 12, 429–439 (1999)CrossRefGoogle Scholar
  19. 19.
    Kaban, A., Girolami, M.: A combined latent trait class and trait model for the analysis and visualisation of discrete data. IEEE Trans. on Pattern Analysis and Machine Intelligence PAMI-23, 859–872 (2001)CrossRefGoogle Scholar
  20. 20.
    Karhunen, J., Joutsensalo, J.: Generalisation of principal component analysis, optimization problems, and neural networks. Neural Networks 8, 549–562 (1995)CrossRefGoogle Scholar
  21. 21.
    Karhunen, J., Malaroiu, S.: Local independent component analysis using clusternig. In: Proc. 1st Int. Workshop on Independent Component Analysis and Signal Separation (ICA 1999), pp. 43–48 (1999)Google Scholar
  22. 22.
    Kaski, S., Kohonen, T.: Exploratory data analysis by the self-organizing map: Structures of welfare and poverty in the world. In: Refenes, A.-P.N., Abu-Mostafa, Y., Moody, J., Weigend, A. (eds.) Neural Networks in Financial Engineering, pp. 498–507. World Scientific, Singapore (1996)Google Scholar
  23. 23.
    Kegl, B., Krzyzak, A., Linder, T., Zeger, K.: A polygonal line algorithm for constructing principal curves. In: NIPS-WS 1996, vol. 11, pp. 501–507 (1998)Google Scholar
  24. 24.
    Kohonen, T.: Self-organised formation of topologically correct feature map. Biological Cybernetics 43, 56–69 (1982)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Kohonen, T.: Self-Organising Maps, 2nd edn. Springer, Berlin (1995)Google Scholar
  26. 26.
    Kraaijveld, M.A., Mao, J., Jain, A.K.: A nonlinear projection method based on Kohonen’s topology preserving maps. IEEE Trans. Neural Networks 6, 548–559 (1995)CrossRefGoogle Scholar
  27. 27.
    Kramer, M.A.: Nonlinear principal component analysis using autoassociative neural networks. AICHE Journal 37, 233–243 (1991)CrossRefGoogle Scholar
  28. 28.
    LeBlanc, M., Tibshirani, R.J.: Adaptive principal surfaces. J. Amer. Statist. Assoc. 89, 53–64 (1994)zbMATHCrossRefGoogle Scholar
  29. 29.
    Lee, R.C.T., Slagle, J.R., Blum, H.: A triangulation method for the sequential mapping of points from n-space to two-space. IEEE Trans. on Computers 27, 288–292 (1977)CrossRefGoogle Scholar
  30. 30.
    Lee, T.-W.: Independent Component Analysis: Theory and Applications. Kluwer Academic, Dordrecht (1998)zbMATHGoogle Scholar
  31. 31.
    Lee, T.-W., Koehler, B.-U., Orglmeister, R.: Blind source separation of nonlinear mixing models. In: Proc. IEEE Workshop on Neural Networks for Signal Processing (NNSP 1997), pp. 406–415 (1997)Google Scholar
  32. 32.
    Lee, T.-W., Lewicki, M.-S., Sejnowski, T.-J.: Unsupervised Classification with Non- Gaussian Mixture Models using ICA. Advances in Neural Information Processing Systems 11, 508–514 (1999)Google Scholar
  33. 33.
    Lowe, D., Tipping, M.E.: Feed-forward neural networks and topographic mappings for exploratory data analysis. Neural Computing and Applications 4, 83–95 (1996)CrossRefGoogle Scholar
  34. 34.
    Malthouse, E.C.: Limitations of nonlinear PCA as performed with generic neural networks. IEEE Trans. Neural Networks 9, 165–173 (1998)CrossRefGoogle Scholar
  35. 35.
    Mao, J., Jain, A.K.: Artificial Neural Networks for Feature Extraction and Multivariate Data Projection. IEEE Trans. on Neural Networks 6, 296–317 (1995)CrossRefGoogle Scholar
  36. 36.
    Mulier, F., Cherkassky, V.: Self-organisation as an iterative kernel smoothing process. Neural Computation 7, 1165–1177 (1995)CrossRefGoogle Scholar
  37. 37.
    Oja, E.: Neural networks, principal components, and subspaces. Int. Journal of Neural Systems 1, 61–68 (1989)CrossRefMathSciNetGoogle Scholar
  38. 38.
    Oja, E.: PCA, ICA, and nonlinear Hebbian learning. In: Proc. Int. Conf. on Artificial Neural Networks (ICANN 1995), pp. 89–94(1995)Google Scholar
  39. 39.
    Pajunen, P., Karhunen, J.: A maximum likelihood approach to nonlinear blind source separation. In: Proc. Int. Conf. on Artificial Neural Networks (ICANN 1997), pp. 541–546 (1997)Google Scholar
  40. 40.
    Pajunen, P., Hyvärinen, A., Karhunen, J.: Nonlinear blind source separation by self organising maps. In: Proc. ICONIP 1996, pp. 1207–1210 (1996)Google Scholar
  41. 41.
    Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)zbMATHGoogle Scholar
  42. 42.
    Ritter, H., Martinetz, T., Schulten, K.: Neural Computation and Self-organising Maps: An Introduction. Addison-Wesley Publishing Company, Reading (1992)Google Scholar
  43. 43.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  44. 44.
    Sanger, T.D.: Optimal unsupervised learning in a single-layer linear feedforward network. Neural Networks 2, 459–473 (1991)CrossRefGoogle Scholar
  45. 45.
    Sammon, J.W.: A nonlinear mapping for data structure analysis. IEEE Trans. on Computer 18, 401–409 (1969)CrossRefGoogle Scholar
  46. 46.
    Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  47. 47.
    Taleb, A., Jutten, C.: Source separation in postnonlinear mixtures. IEEE Trans. On Signal Processing 47, 2807–2820 (1999)CrossRefGoogle Scholar
  48. 48.
    Tan, Y., Wang, J., Zurada, J.M.: Nonlinear blind source separation using a radial basis function network. IEEE Trans. on Neural Networks 12, 124–134 (2001)CrossRefGoogle Scholar
  49. 49.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  50. 50.
    Tibshirani, R.: Principal curves revisited. Statistics and Computation 2, 183–190 (1992)CrossRefGoogle Scholar
  51. 51.
    Törönen, P., Kolehmainen, K., Wong, G., Castrén, E.: Analysis of gene expression data using self-organising maps. FEBS Letters 451, 142–146 (1999)CrossRefGoogle Scholar
  52. 52.
    Ultsch, A.: Self-organising neural networks for visualisation and classification. In: Opitz, O., Lausen, B., Klar, R. (eds.) Information and Classification, pp. 864–867 (1993)Google Scholar
  53. 53.
    Xu, L., Cheung, C.C., Amari, S.-I.: Learned parametric mixture based ICA algorithm. Neurocomputing 22, 69–80 (1998)zbMATHCrossRefGoogle Scholar
  54. 54.
    Yin, H.: Visualisation induced SOM (ViSOM). In: Allinson, N., Yin, H., Allinson, L., Slack, J. (eds.) Advances in Self-Organising Maps (Proc. WSOM 2001), pp. 81–88. Springer, Heidelberg (2001)Google Scholar
  55. 55.
    Yin, H.: ViSOM-A novel method for multivariate data projection and structure visualisation. IEEE Trans. on Neural Networks 13, 237–243 (2002)CrossRefGoogle Scholar
  56. 56.
    Yin, H.: Data visualisation and manifold mapping using the ViSOM. Neural Networks 15, 1005–1016 (2002)CrossRefGoogle Scholar
  57. 57.
    Yin, H., Allinson, N.M.: Interpolating self-organising map (iSOM). Electronics Letters 35, 1649–1650 (1999)CrossRefGoogle Scholar
  58. 58.
    Yin, H., Allinson, N.M.: Bayesian self-organising map for Gaussian mixtures. In: IEE Proc. –Vis. Image Signal Processing, vol. 148, pp. 234–240 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Hujun Yin
    • 1
  1. 1.Dept. of Electrical Engineering and ElectronicsUniversity of Manchester Institute of Science and TechnologyManchesterUK

Personalised recommendations