Abstract
With the rapid development of Internet of things (IoT), a large number of IoT sensing devices produces amounts of sensing data in every second. These data should be processed in real-time to support IoT real-time services. The growth of IoT real-time services has been hampered due to the barriers of data storage efficiency and data processing performance with the traditional database system architecture. This paper proposes a lightweight time series main-memory database (TSMMDB) system for IoT real-time services. Firstly, we propose a tree structure of IoT sensing data model based on the IoT real-time monitoring business. The leaves of the tree are three-dimension tables. The data can be retrieved according to time, resource and measure. Based on the data model, we propose a customized virtual heap and virtual heap memory allocator. The applications can access the whole data in the database in their own processes based on shared memory without transferring data, and can achieve data persistence automatically based on memory mapping. The flexible data locality memory allocation makes the adjacent time series data storing in the continuous memory space which improves the data clustered analysis performance. The data access algorithm of TSMMDB has ideal time complexity, and experimental results show that TSMMDB has better performance significantly than the traditional main-memory database and disk-based relational database.
This work is supported by Key Research and Development Program for Guangdong Province under grant No. 2019B010137003, the Fundamental Research Funds for the Central Universities (Grant no. 24820192019RC56).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsChange history
09 June 2020
In the version of these papers that was originally published, the Grant no. should be “24820192019RC56”, not “2018RC56”. This has now been corrected.
References
Kunkle, D., Cooperman, G.: Solving Rubik’s cube: disk is the new RAM. Commun. ACM 51(4), 31–33 (2008)
Zhang, H., et al.: In-memory big data management and processing: a survey. IEEE Trans. Knowl. Data Eng. 27(7), 1920–1948 (2015)
Larson, P.Å., Levandoski, J.: Modern main-memory database systems. Proc. VLDB Endow. 9(13), 1609–1610 (2016)
Ailamaki, A.: The next 700 transaction processing engines. In: Proceedings of the 2017 ACM International Conference on Management of Data, pp. 1–2. ACM (2017)
Lahiri, T., et al.: Oracle database in-memory: a dual format in-memory database. In: 2015 IEEE 31st International Conference on Data Engineering (ICDE), pp. 1253–1258. IEEE (2015)
Kim, J., Salem, K., Daudjee, K.: Write amplification: an analysis of in-memory database durability techniques. In: Proceedings of the 3rd VLDB Workshop on In-Memory Data Management and Analytics. ACM (2015). Article No. 1
Plattner, H.: A common database approach for OLTP and OLAP using an in-memory column database. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pp. 1–2. ACM (2009)
Stonebraker, M., Weisberg, A.: The VoltDB main memory DBMS. IEEE Data Eng. Bull. 36(2), 21–27 (2013)
Xie, X., Chai, E., Zhang, X.: Hekaton: efficient and practical large-scale MIMO. In: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pp. 304–316. ACM (2015)
Färber, F., Cha, S.K., Primsch, J., Bornhövd, C., Sigg, S., Lehner, W.: SAP HANA database: data management for modern business applications. ACM SIGMOD Rec. 40(4), 45–51 (2011)
Färber, F., et al.: The SAP HANA database-an architecture overview. IEEE Data Eng. Bull. 35(1), 28–33 (2012)
Sikka, V., et al.: Efficient transaction processing in SAP HANA database: the end of a column store myth. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 731–742. ACM (2012)
Sikka, V., Färber, F., Goel, A., Lehner, W.: SAP HANA: the evolution from a modern main-memory data platform to an enterprise application platform. Proc. VLDB Endow. 6(11), 1184–1185 (2013)
Nica, A., Sherkat, R., Andrei, M., et al.: Statisticum: data statistics management in SAP HANA. Proc. VLDB Endow. 10(12), 1658–1669 (2017)
Lee, J., Moon, S.H., Kim, K.H., et al.: Parallel replication across formats in SAP HANA for scaling out mixed OLTP/OLAP workloads. Proc. VLDB Endow. 10(12), 1598–1609 (2017)
Andrei, M., Lemke, C., Radestock, G., et al.: SAP HANA adoption of non-volatile memory. Proc. VLDB Endow. 10(12), 1754–1765 (2017)
Pelkonen, T., et al.: Gorilla: a fast, scalable, in-memory time series database. Proc. VLDB Endow. 8(12), 1816–1827 (2015)
Meng, Q., Zhou, X., Wang, S.: Memory instant snapshot sharing mechanism and its application in database. Chin. J. Comput. 41(28), 1912–1927 (2018)
Lea, D.: http://g.oswego.edu/dl/html/malloc.html. Source code: ftp://g.oswego.edu/pub/misc/malloc.c
Kemper, A., Neumann, T.: HyPer: a hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: Proceedings of the 2011 IEEE 27th International Conference on Data Engineering, Hannover, Germany, pp. 195–206 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lan, L., Shi, R., Wang, B., Zhang, L., Shi, J. (2020). A Lightweight Time Series Main-Memory Database for IoT Real-Time Services. In: Hsu, CH., Kallel, S., Lan, KC., Zheng, Z. (eds) Internet of Vehicles. Technologies and Services Toward Smart Cities. IOV 2019. Lecture Notes in Computer Science(), vol 11894. Springer, Cham. https://doi.org/10.1007/978-3-030-38651-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-38651-1_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38650-4
Online ISBN: 978-3-030-38651-1
eBook Packages: Computer ScienceComputer Science (R0)