Design and Implementation of Sensory Data Collection and Storage Based on Hadoop Platform

  • Zhen BaiEmail author
  • Shaohua Cui
  • Chenglin Zhao
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 517)


At present, modern manufacturing and management concepts such as digitization, networking, and intelligence are widely used in the industry. Industry automation and information have been unprecedentedly improved, and therefore the entire life of industrial production link involves massive amounts of data, and the status monitoring data of industrial machine have large, multiple source, heterogeneous, and complex data characteristics. What is more, the traditional processing methods and tools could not meet the requirements for massive data, and may miss the best time to repair machine. So, to resolve the challenges that the industrial sensory big data faces, this paper proposes the sensory data collection and storage based on Hadoop platform.


Hadoop platform Sensory data Data collection Data storage 



This work was funded by the National Intelligent Manufacturing Project.


  1. 1.
    Liu, F.C., Shen, F.: Building a research data science platform from industrial machines. In: IEEE Conference (2015)Google Scholar
  2. 2.
    Manwal, M., Gupta, A.: Big Data and Hadoop A Technological Survey, Emerging Trends in Computing and Communication Technologies (ICETCCT), February (2018)Google Scholar
  3. 3.
    Lyu, Y., Fan, X., Liu, K.: An optimized strategy for small files storing and accessing in HDFS. In: IEEE International Conference on Computational Science and Engineering (CSE), vol. 1, pp. 611–614 (2017)Google Scholar
  4. 4.
    Sarnovsky, M., Bajus, D.: Building environment analysis based on clustering methods from sensory data on top of the Hadoop platform. In: IEEE Conference (2017)Google Scholar
  5. 5.
    Xie, J.: Construction for the city taxi trajectory data analysis system by Hadoop platform. In: IEEE Conference (2017)Google Scholar
  6. 6.
    Zhonghua, M.: Seismic data attribute extraction based on Hadoop platform. In: IEEE Conference (2017)Google Scholar
  7. 7.
    Longley, P.A., Goodchild, M.F., Maguire, D.J., Rhind, D.W.: Geographic Information Systems and Science. Wiley, New York (2001)Google Scholar
  8. 8.
    Mazumdar, S., Dhar, S.: Hadoop as big data operating system—the emerging approach for managing challenges of enterprise big data platform. In: IEEE Conference (2015)Google Scholar
  9. 9.
    Ding, X., Tian, B.: A scheme of structured data compression and query on Hadoop platform. In: IEEE Conference (2015)Google Scholar
  10. 10.
    Yan, H., Song, G.: Based on the Hadoop cloud computing experiment platform. J. Shenyang Norm. Univ. (Natural Sciences) 1, 85–89 (2013)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Key Laboratory of Universal Wireless Communications, MOEBeijing University of Posts and TelecommunicationsBeijingChina
  2. 2.China Petroleum Technology & Development CorporationBeijingChina

Personalised recommendations