Abstract
In large internet enterprise and data management system, Hadoop MapReduce is a popular framework. For data intensive batch jobs, MapReduce provided its impact. To build consistent, hi-availability (HA) and scalable data management system to serve peta bytes of data for the massive users, are the main focused objects. MapReduce is a programming model that enables easy development of scalable parallel applications to process vast amount of data on large cluster. Through a simple interface with two functions map and reduce, this model facilities parallel implementation of real world tasks such as data processing for search engine and machine learning. Earlier version of Hadoop MapReduce has several performance problems like connection between Map to Reduce task, data overload and time consumption. In this paper, we proposed a modified MapReduce architecture MRA (MapReduce Agent) which is a fusion of iSCSI protocol and the downloaded reference code of Hadoop*. Our developed MRA can reduce completion time, improve system utilization and give better performance.
This research (Grants NO. 2013-140-10047118) was supported by the 2013 Industrial Technology Innovation Project Funded by Ministry Of Science, ICT and Future Planning. The source code for HOP can be downloaded from http://code.google.com/p/hop
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
DEAN, J., AND GHEMAWAT, S. MapReduce: Simplified dataprocessing on large clusters. In OSDI (2004).
SAM-3 Information Technology – SCSI Architecture Model 3, Working Draft, T10 Project 1561-D, Revision7 (2003)
Allayear, S.M., Park, S.S.: iSCSI Multi-connection and Error Recovery Method for Remote Storage System in Mobile Appliance. In: Gavrilova, M.L., Gervasi, O., Kumar, V., Tan, C.J.K., Taniar, D., Laganá, A., Mun, Y., Choo, H. (eds.) ICCSA 2006. LNCS, vol. 3981, pp. 641–650. Springer, Heidelberg (2006)
Hadoop, HYPERLINK, http://hadoop.apache.org/mapreduce/
Condie, T., Conway, N., Alvaro, P., Hellerstein, J.M.: UC Berkeley: MapReduce Online. Khaled Elmeleegy, Russell Sears (Yahoo! Research)
Allayear, S.M., Park, S.S., No, J.: iSCSI Protocol Adaptation with 2-way TCP Hand Shake Mechanism for an Embedded Multi-Agent Based Health Care Service. In: Proceedings of the 10th WSEAS International Conference on Mathematical Methods, Computational Techniques and Intelligent Systems, Corfu, Greece (2008)
Allayear, S.M., Park, S.S.: iSCSI Protocol Adaptation With NAS System Via Wireless Environment. In: International Conference on Consumer Electronics (ICCE), Las Vegus, USA (2008)
Caceres, R., Iftode, L.: Improving the Performance of Reliable Transport Protocols in Mobile Computing Environments. IEEE JSAC
RFC 3270, http://www.ietf.org/rfc/rfc3720.txt
Verma, A., Zea, N., Cho, B., Gupta, I., Campbell, R.H.: Breaking the MapReduce Stage Barrier*
Yang, H., Dasdan, A., Hsiao, R., Parker, D.: Map-reduce-merge: simplified relational data processing on large clusters. In: Proc. of the 2007 ACM SIGMOD International Conference on Management of Data (January 2007)
Hellerstein, J.M., Haas, P.J., Wang, H.J.: Online aggregation. In: SIGMOD (1997)
Shah, M.A., Hellerstein, J.M., Brewer, E.A.: Highly-available, fault-tolerant, parallel dataflows. In: SIGMOD (2004)
Thusoo, A., Sarma, J.S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wyckoff, P., Murthy, R.: Hive—a warehousing solution over a Map-Reduce framework. In: VLDB (2009)
Wu, S., Jiang, S., Ooi, B.C., Tan, K.-L.: Distributed online aggregation. In: VLDB (2009)
Yang, C., Yen, C., Tan, C., Madden, S.: Osprey: Implementing MapReduce-style fault tolerance in a shared-nothing distributed database. In: ICDE (2010)
Chan, J.O.: An Architecture for Big Data Analytics
Daneshyar, S., Razmjoo, M.: Large-Scale Data Processing Using Mapreduce in Cloud Computing Environment
Ji, C., Li, Y., Qiu, W., Awada, U., Li, K.: Big Data Processing in Cloud Computing Environments
Padhy, R.P.: Big Data Processing with Hadoop-MapReduce in Cloud Systems
Stokely, M.: Histogram tools for distributions of large data sets
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Allayear, S.M., Salahuddin, M., Ahmed, F., Park, S.S. (2014). Introducing iSCSI Protocol on Online Based MapReduce Mechanism. In: Murgante, B., et al. Computational Science and Its Applications – ICCSA 2014. ICCSA 2014. Lecture Notes in Computer Science, vol 8583. Springer, Cham. https://doi.org/10.1007/978-3-319-09156-3_48
Download citation
DOI: https://doi.org/10.1007/978-3-319-09156-3_48
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09155-6
Online ISBN: 978-3-319-09156-3
eBook Packages: Computer ScienceComputer Science (R0)