Advertisement

Task Scheduling for Streaming Applications in a Cloud-Edge System

  • Fei Yin
  • Xinjia Li
  • Xin LiEmail author
  • Yize Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11637)

Abstract

With the increasing popularity of ubiquitous smart devices, more and more IoT (Internet of Things) data processing applications are deployed. Due to the inherent defects of traditional data transmission networks and the low latency requirement of applications, effective use of bandwidth computing resources to support the efficient deployment of applications has become a very important issue. In this paper, we focus on how to deploy multi-source streaming data processing applications in a cloud-edge collaborative computing network and pay attention to make the overall application data processing delay lower. We abstract the application into a form of streaming data processing, formalize it as a Stream Processing Task Scheduling Problem. We present an efficient algorithm to solve the above problem. Simulation experiments show that our approach can significantly reduce the end-to-end latency of applications compared to commonly used greedy algorithms.

Keywords

Edge computing End to end delay Internet of Things Stream data processing Task scheduling 

References

  1. 1.
    L. Columbus Internet Of Things Market To Reach \$267B By 2020. https://www.forbes.com/sites/louiscolumbus/2017/01/29/%0Ainternet-of-things-market-to-reach-267b-by-2020/. Accessed 1 May 2019
  2. 2.
    Yang, L., Cao, J., Yuan, Y., Li, T., Han, A., Chan, C.: A framework for partitioning and execution of data stream applications in mobile cloud computing. In: International Conference on Cloud Computing 2012, vol. 40, pp. 23–32.  https://doi.org/10.1145/2479942.2479946CrossRefGoogle Scholar
  3. 3.
    Yang, L., Cao, J., Cheng, H., Ji, Y.: Multi-user computation partitioning for latency sensitive mobile cloud applications. IEEE Trans. Comput. 8(64), 2253–2266 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Soyata T., et al.: COMBAT: mobile-Cloud-based cOmpute/coMmunications infrastructure for BATtlefield applications. In: Proceedings of SPIE, vol. 8403, pp. 1–13.  https://doi.org/10.1117/12.919146
  5. 5.
    Akidau, T., et al.: The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. In: Very Large Data Bases 2015, vol. 8, pp. 1792–1803 (2015)CrossRefGoogle Scholar
  6. 6.
    Flink Home page. https://flink.apache.org/. Accessed 1 May 2019
  7. 7.
    Storm Home page. https://storm.apache.org/. Accessed 1 May 2019
  8. 8.
    Spark Home page. https://spark.apache.org/. Accessed 1 May 2019
  9. 9.
    Chintapalli, S., et al.: Benchmarking streaming computation engines: storm, flink and spark streaming. In: International Parallel and Distributed Processing Symposium 2016, pp. 1789–1792 (2016).  https://doi.org/10.1109/IPDPSW.2016.138
  10. 10.
    Pietzuch, P.R., Ledlie, J., Shneidman, J., Roussopoulos, M., Welsh, M., Seltzer, M.I.: Network-aware operator placement for stream-processing systems. In: International Conference on Data Engineering 2006, p. 49 (2006).  https://doi.org/10.1109/ICDE.2006.105
  11. 11.
    Jonathan, A., Chandra, A., Weissman, J.B.: Multi-query optimization in wide-area streaming analytics. In: Symposium on Cloud Computing 2018, pp. 412–425 (2018).  https://doi.org/10.1145/3267809.3267842
  12. 12.
    Heintz, B.: Optimizing Timeliness, Accuracy, and Cost in Geo-Distributed Data-Intensive Computing Systems (2016)Google Scholar
  13. 13.
    Heintz, B., Chandra, A., Sitaraman, R.K.: Optimizing grouped aggregation in geo-distributed streaming analytics. In: High Performance Distributed Computing 2015, pp. 133–144 (2015).  https://doi.org/10.1145/2749246.2749276
  14. 14.
    Heintz, B., Chandra, A., Sitaraman, R.K.: Trading timeliness and accuracy in geo-distributed streaming analytics. In: Symposium on Cloud Computing 2016, pp. 361–373 (2016).  https://doi.org/10.1145/2987550.2987580
  15. 15.
    Hwang, J., Cetintemel, U., Zdonik, S.B.: Fast and highly-available stream processing over wide area networks. In: International Conference on Data Engineering 2008, pp. 804–813 (2008).  https://doi.org/10.1109/ICDE.2008.4497489

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.JiangSu Frontier Electric Technology Co., LTDNanjingChina
  2. 2.CCSTNanjing University of Aeronautics and AstronauticsNanjingChina
  3. 3.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations