Skip to main content
SpringerLink
Log in

Adaptation Approaches in Unsupervised Learning: A Survey of the State-of-the-Art and Future Directions

  • Conference paper
  • First Online:
Image Analysis and Recognition (ICIAR 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9730))

Included in the following conference series:

Abstract

In real applications, data continuously evolve over time and change from one setting to another. This inspires the development of adaptive learning algorithms to deal with this data dynamics. Adaptation mechanisms for unsupervised learning have received an increasing amount of attention from researchers. This research activity has produced a lot of results in tackling some of the challenging problems of the adaptation process that are still open. This paper is a brief review of adaptation mechanisms in unsupervised learning focusing on approaches recently reported in the literature for adaptive clustering and novelty detection and discussing some future directions. Although these approaches have able to cope with different levels of data non-stationarity, there is a crucial need to extend these approaches to be able to handle large amount of data in distributed resource-limited environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chan, A., Gouvea, E., Singh, R., Mosur, R., Rosenfield, R., Sun, Y., Huggins-Daines, D.: (First Draft) Hieroglpyhs : Building speech applications using sphinx and related resources (2004)

    Google Scholar 

  2. Lee, C.H., Lin, C.H., Juang, B.H.: A study on speaker adaptation of the parameters of continuous density hidden Markov models. IEEE Trans. Sig. Process. 39(4), 806–814 (1991)

    Article  Google Scholar 

  3. Leggetter, C.J., Woodland, P.C.: Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models. Comput. Speech Lang. 9(2), 171–185 (1995)

    Article  Google Scholar 

  4. Miao, Y.Q.: Adaptive learning algorithms for non-stationary data, Ph.D. thesis, University of Waterloo (2015)

    Google Scholar 

  5. Miao, Y.Q., Farahat, A.K., Kamel, M.S.: Detecting emerging and evolving novelties with locally adaptive density ratio estimation. Knowl. Inf. Syst., 1–29 (2016)

    Google Scholar 

  6. Farahat, A.K., Kamel, M.S.: Statistical semantics for enhancing document clustering. Knowl. Inf. Syst. 28(2), 365–393 (2011)

    Article  Google Scholar 

  7. Pimentel, M.A.F., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Sig. Process. 99, 215–249 (2014)

    Article  Google Scholar 

  8. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–69 (2009)

    Article  Google Scholar 

  9. Sayed-Mouchaweh, M., Lughofer, E.: Learning in Non-Stationary Environments: Methods and Applications. Springer, New York (2012)

    Book  MATH  Google Scholar 

  10. Sugiyama, M., Kawanabe, M.: Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation (2012)

    Google Scholar 

  11. Chi, Y., Song, X., Zhou, D., Hino, K., Tseng, B.L.: Evolutionary spectral clustering by incorporating temporal smoothness. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 153–162 (2007)

    Google Scholar 

  12. Langone, R.: Clustering evolving data using kernel-based methods. Ph.D. thesis, KU Leuven (2014)

    Google Scholar 

  13. Langone, R., Alzate, C., Suykens, J.A.K.: Kernel spectral clustering with memory efect. Phy. A Stat. Mech. Appl. 392(10), 2588–2606 (2013)

    Article  MathSciNet  Google Scholar 

  14. Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2010)

    Article  Google Scholar 

  15. Chang, H., Yeung, D.Y.: Robust path-based spectral clustering. Pattern Recogn. 41(1), 191–203 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  16. Wang, L., Leckie, C., Kotagiri, R., Bezdek, J.: Approximate pairwise clustering for large data sets via sampling plus extension. Pattern Recogn. 44(2), 222–235 (2011)

    Article  Google Scholar 

  17. Cottrell, M., Fort, J.C., Pags, G.: Theoretical aspects of the SOM algorithm. Neurocomputing 21(1), 119–138 (1998)

    Article  MATH  Google Scholar 

  18. de Bodt, E., Verleysen, M., Cottrell, M.: Kohonen maps versus vector quantization for data analysis. In: ESANN, pp. 211–218 (1997)

    Google Scholar 

  19. Barreto, G.A.: Time series prediction with the self-organizing map: A review. In: Hitzler, P., Hammer, B. (eds.) Perspectives on Neural-Symbolic Integration, pp. 135–158. Springer, Berlin (2007)

    Chapter  Google Scholar 

  20. Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer, Berlin (2001)

    Book  MATH  Google Scholar 

  21. Sarlin, P.: Self-organizing time map: an abstraction of temporal multivariate patterns. Neurocomputing 99, 496–508 (2013)

    Article  Google Scholar 

  22. Chakraborty, S., Nagwani, N.K.: Analysis and Study of Incremental K-Means Clustering Algorithm. In: Mantri, A., Nandi, S., Kumar, G., Kumar, S. (eds.) HPAGC 2011. CCIS, vol. 169, pp. 338–341. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  23. Langone, R., Agudelo, O.M., De Moor, B., et al.: Incremental kernel spectral clustering for online learning of non-stationary data. Neurocomputing 139, 246–260 (2014)

    Article  Google Scholar 

  24. Valgren, C., Duckett, T., Lilienthal, A.: Incremental spectral clustering and its application to topological mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 4283–4288 (2007)

    Google Scholar 

  25. Valgren, C., Lilienthal, A.: Incremental spectral clustering and seasons: Appearance-based localization in outdoor environments. In: Robotics and Automation, ICRA, pp. 1856–1861 (2008)

    Google Scholar 

  26. Cao, F., Ester, M., Qian, W., Zhou, A.: Density-based clustering over an evolving data stream with noise. In: Proceedings of the Sixth SIAM International Conference on Data Mining (SDM), pp. 328–339 (2006)

    Google Scholar 

  27. Chen, H.L., Chen, M.S., Lin, S.C.: Catching the trend: a framework for clustering concept-drifting categorical data. IEEE Trans. Knowl. Data Eng. 21(5), 652–665 (2009)

    Article  Google Scholar 

  28. Cao, F.Y., Huang, Z.X., Liang, J.Y.: Trend analysis of categorical data streams with a concept change method. Inform. Sci. 276, 160–173 (2007)

    Article  Google Scholar 

  29. Nasraoui, O., Soliman, M., Saka, E., Badia, A., Germain, R.: A web usage mining framework for mining evolving user profiles in dynamic web sites. Trans. Knowl. Data Eng. 20(2), 202–215 (2008)

    Article  Google Scholar 

  30. Shental, N., Bar-Hillel, A., Hertz, T., et al.: Computing Gaussian mixture models with EM using equivalence constraints. Adv. Neural Inform. Process. Syst. (NIPS) 16(8), 465–472 (2004)

    Google Scholar 

  31. Park, C., Huang, J.Z., Ding, Y.: A computable plug-in estimator of minimum volume sets for novelty detection. Oper. Res. 58(5), 1469–1480 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  32. Breaban, M., Luchian, H.: Outlier detection with nonlinear projection pursuit. Int. J. Comput. Commun. Control 8(1), 30–36 (2013)

    Article  MathSciNet  Google Scholar 

  33. Zhang, K., Hutter, M., Jin, H.: A new local distance-based outlier detection approach for scattered real-word data. Adv. Knowl. Discov. Data Min. 5476, 813–822 (2009)

    Google Scholar 

  34. Cabral, G.G., Oliveira, A.L.I., Cahu, C.B.G.: Combining nearest neighbor data description and structural risk minimization for one-class classification. Neural Comput. Appl. 18(2), 175–183 (2009)

    Article  Google Scholar 

  35. Bay, S.D., Schwabacher, M.: Mining distance-based outlier sinnear linear time with randomization, a simplep runing rule. In: Proceedings of the 9th ACM International Conference on Knowl edge Discovery and Data Mining (SIGKDD), pp. 29–38. ACM (2003)

    Google Scholar 

  36. Miao, Y.Q., Farahat, A.K., Kamel, M.S.: Locally adaptive density ratio for detecting novelty in twitter streams. In: The 6th International Workshop on Modeling Social Media (MSM) - Behavioral Analytics in Social Media, Big Data and the Web, pp. 799–804 (2015)

    Google Scholar 

  37. Song, L., Teo, C.H., Smola, A.J.: Relative novelty detection. In: International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 536–543 (2009)

    Google Scholar 

  38. Scholkopf, B., Williamson, R.C., Smola, A.J., Shawe-Taylor, J., Platt, J.: Support vector method for novelty detection. Adv. Neural Inform. Process. Syst.(NIPS) 12, 582–588 (1999)

    Google Scholar 

  39. Hattori, Y., Inoue, S., Masaki, T., Hirakawa, G., Sudo, O.: Gathering large scale human activity information using mobile sensor devices. In: Proceedings of the Second International Workshop on Network Traffic Control, Analysis and Applications, pp. 708–713 (2010)

    Google Scholar 

  40. Sun, S., Shi, H., Wu, Y.: A survey of multi-source domain adaptation. Inform. Fusion 24, 84–92 (2015)

    Article  Google Scholar 

Download references

Acknowledgement

The paper is dedicated to the late Professor Mohamed Kamel, founding director of Centre for Pattern Analysis and Machine Intelligence (CPAMI) who gave us the chance to work on this interesting area of research. This contribution was made possible by NPRP grant # 06-1220-1-233 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors.

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Shanxi University, Taiyuan, 030006, China

    JunHong Wang & Jiye Liang

  2. Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, N2L 3G1, Canada

    JunHong Wang, Alaa Khamis & Fakhri Karray

  3. NCR Canada Ltd., Waterloo, N2V 1N3, Canada

    YunQian Miao

  4. Engineering Science Department, Suez University, Suez, Egypt

    Alaa Khamis

Authors
  1. JunHong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. YunQian Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alaa Khamis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fakhri Karray
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiye Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to JunHong Wang .

Editor information

Editors and Affiliations

  1. University of Porto, Porto, Portugal

    Aurélio Campilho

  2. Department of Electrical, University of Waterloo, Waterloo, Ontario, Canada

    Fakhri Karray

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Wang, J., Miao, Y., Khamis, A., Karray, F., Liang, J. (2016). Adaptation Approaches in Unsupervised Learning: A Survey of the State-of-the-Art and Future Directions. In: Campilho, A., Karray, F. (eds) Image Analysis and Recognition. ICIAR 2016. Lecture Notes in Computer Science(), vol 9730. Springer, Cham. https://doi.org/10.1007/978-3-319-41501-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41501-7_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41500-0

  • Online ISBN: 978-3-319-41501-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation