Advertisement

Parallel Implementation of Ant-Based Clustering Algorithm Based on Hadoop

  • Yan Yang
  • Xianhua Ni
  • Hongjun Wang
  • Yiteng Zhao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7331)

Abstract

Hadoop is a distributed system infrastructure of cloud computing. Based on the characteristics of ant-based clustering algorithm, the paper implements the parallelization of this algorithm using MapReduce on Hadoop. The Map function calculates the average similarity of the object with its neighborhood objects. The Reduce function processes the objects with the Map outputs and updates related information of both ants and the objects to get ready for the next job. Results on the Hadoop clusters show that our method can significantly improve the computational efficiency with the premise of maintaining clustering accuracy.

Keywords

Ant-based Clustering Parallelization Hadoop MapReduce model 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: Operating Systems Design and Implementation, pp. 137–149 (2004)Google Scholar
  2. 2.
    Apache Hadoop. Hadoop, http://hadoop.apache.org
  3. 3.
    Ghemawat, S., Gobioff, H., Leung, S.: The Google file system. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 29–43. ACM, Bolton Landing (2003)CrossRefGoogle Scholar
  4. 4.
    Borthakur, D.: The Hadoop Distributed File System: Architecture and Design. The Apache Software Foundation, http://hadoop.apache.org
  5. 5.
    Wei, J., Ravi, V.T., Agrawal, G.: Comparing map-reduce and FREERIDE for data-intensive applications. In: IEEE International Conference on Cluster Computing and Workshops. CLUSTER 2009, pp. 1–10 (2009)Google Scholar
  6. 6.
    Smith, A.E.: Swarm intelligence: from natural to artificial systems. IEEE Transactions on Evolutionary Computation 4, 192–193 (2000)CrossRefGoogle Scholar
  7. 7.
    Yang, Y., Kamel, M.: Clustering Ensemble Using Swarm Intelligence. In: IEEE Swarm Intelligence Symposium, pp. 65–71 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Yan Yang
    • 1
    • 2
  • Xianhua Ni
    • 1
    • 2
  • Hongjun Wang
    • 1
    • 2
  • Yiteng Zhao
    • 1
  1. 1.School of Information Science and TechnologySouthwest Jiaotong UniversityChengduP.R. China
  2. 2.Key Lab of Cloud Computing and Intelligent TechnologyChengduP.R. China

Personalised recommendations