Coordinated Placement of Meteorological Workflows and Data with Privacy Conflict Protection
- 16 Downloads
Cloud computing is cited by various industries for its powerful computing power to solve complex calculations in the industry. The massive data of meteorological department has typical big data characteristics. Therefore, cloud computing has been gradually applied to deal with a large number of meteorological -services. Cloud computing increases the computational speed of meteorological services, but data transmission between nodes also generates additional data transmission time. At the same time, based on cloud computing technology, a large number of computing tasks are cooperatively processed by multiple nodes, so improving the resource utilization of each node is also an important evaluation indicator. In addition, with the increase of data confidentiality, there are some data conflicts between some data, so the conflicting data should be avoided being placed on the same node. To cope with this challenge, the meteorological application is modeled and a collaborative placement method for tasks and data based on Differential Evolution algorithm (CPDE) is proposed. The Non-dominated Sorting Differential Evolution (NSDE) algorithm is used to jointly optimize the average data access time, the average resource utilization of nodes and the data conflict degree. Finally, a large number of experimental evaluations and comparative analyses verify the efficiency of our proposed CPDE method.
KeywordsMeteorological Coordinated placement NSDE Data access time Resource utilization Data conflict
This research is supported by the Scientific Research Project of Silicon Lake College under Grant No. 2018KY23.
- 6.Deng, K., Kong, L., Song, J., Ren, K., Dong, Y.: A weighted k-means clustering based co-scheduling strategy towards efficient execution of scientific workflows in collaborative cloud environments. In: IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing, DASC 2011, 12–14 December 2011, Sydney, Australia, pp. 547–554 (2011)Google Scholar
- 7.Kim, H., Kim, Y.: An adaptive data placement strategy in scientific workflows over cloud computing environments. In: NOMS 2018–2018 IEEE/IFIP Network Operations and Management Symposium, pp. 1–5 (2018)Google Scholar
- 8.Kashlev, A., Lu, S., Ebrahimi, M., Mohan, A.: BDAP: a big data placement strategy for cloud-based scientific workflows. In: 2015 IEEE First International Conference on Big Data Computing Service and Applications, pp. 813–820 (2015)Google Scholar
- 9.Liao, Z., Yu, B., Liu, K., Wang, J.: Learning-based adaptive data placement for low latency in data center networks. In: IEEE 43rd Conference on Local Computer Networks (2018)Google Scholar
- 10.Whaiduzzaman, M., Gani, A., Naveed, A. PEFC: performance enhancement framework for cloudlet in mobile cloud computing. In: IEEE-ROMA-2014, pp. 224–229 (2014)Google Scholar
- 11.Xu, X. et al.: A multi-objective data placement method for IoT applications over big data using NSGA-II. In: IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 503–509 (2018)Google Scholar
- 14.Kang, S., Veeravalli, B., Aung, K.M.M.: A security-aware data placement mechanism for big data cloud storage systems. In: IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC) and IEEE International Conference on Intelligent Data and Security (IDS), pp. 327–332 (2016)Google Scholar
- 16.Liu, L., Song, J., Wang, H.: BRPS: a big data placement strategy for data intensive applications. In: IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 813–820 (2016)Google Scholar