Advertisement

A Framework for Data Processing at the Edges of Networks

  • Ichiro Satoh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8056)

Abstract

This paper proposes a distributed processing framework inspired by MapReduce processing. It is unique to other distributed processing approaches to large-scale data, i.e., so-called big data, because it can locally process data maintained in distributed nodes, including sensor or database nodes with non-powerful computing capabilities connected through low-bandwidth networks. It introduces mobile agent technology so that it distributes data processing tasks to distributed nodes as a map step and aggregates their results by returning them to specified servers as a reduce step. The paper describes the architecture of the framework, its basic performance, and its applications.

Keywords

Sensor Node Cloud Computing Mobile Agent Reducer Agent Worker Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bonomi, F., Milito, R., Zhu, J., Addepalli, S.: Fog computing and its role in the internet of things. In: Proceedings of MCC Workshop on Mobile Cloud Computing (MCC 2012), pp. 13–16. ACM Press (2012)Google Scholar
  2. 2.
    Bu, Y., Howe, B., Balazinska, M., Ernst, M.D.: HaLoop: Efficient Iterative Data Processing on Large Clusters. Proceedings of the VLDB Endowment 3(1) (2010)Google Scholar
  3. 3.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th Conference on Symposium on Opearting Systems Design and Implementation, OSDI 2004 (2004)Google Scholar
  4. 4.
    Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.H., Qiu, J., Fox, G.: Twister: a runtime for iterative MapReduce. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing (HPDC 2010). ACM (2010)Google Scholar
  5. 5.
    Grossman, R., Gu, Y.: Data mining using high performance data clouds: experimental studies using sector and sphere. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2008), pp. 920–927. ACM (2008)Google Scholar
  6. 6.
    Jiang, W., Ravi, V.T., Agrawal, G.: A Map-Reduce System with an Alternate API for Multi-Core Environments. In: Proceedings of 10th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing (2010)Google Scholar
  7. 7.
    Ogawa, H., Nakada, H., Takano, R., Kudoh, T.: SSS: An Implementation of Key-Value Store Based MapReduce Framework. In: Proceeding of IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pp. 754–761 (2010)Google Scholar
  8. 8.
    Satoh, I.: Mobile Agents. In: Handbook of Ambient Intelligence and Smart Environments, pp. 771–791. Springer (2010)Google Scholar
  9. 9.
    Sehrish, S., Mackey, G., Wang, J., Bent, J.: MRAP: A Novel MapReduce-based Framework to Support HPC Analytics Applications with Access Patterns. In: Proceedings of High Performance Distribute Computing, HPDC 2010 (2010)Google Scholar
  10. 10.
    Talbot, J., Yoo, R.M., Kozyrakis, C.: Phoenix++: modular MapReduce for shared-memory systems. In: Proceedings of 2nd International Workshop on MapReduce and its Applications (MapReduce 2011). ACM Press (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ichiro Satoh
    • 1
  1. 1.National Institute of InformaticsChiyoda-kuJapan

Personalised recommendations