Research on Data Flow Partitioning Based on Dynamic Feature Extraction
With the rapid development of the Internet of things, social networks, and e-commerce, the era of big data has arrived. Although big data has great potential for many areas such as industry, education, and health care, getting valuable knowledge from big data can be a daunting task. Big data has the characteristics of high-speed change, and its content and distribution characteristics are in dynamic changes. Most current models are static learning models that do not support online updating, making it difficult to learn dynamically changing big data features in real time. In order to solve this problem, this paper proposed a method to support incremental recursive least squares (IRLS) regression parameter estimation and variable sliding window algorithm to analyze and judge the trends of dynamic characteristics of data streams, which can provide early warning, status assessment, and decision support for monitoring objects and improve the accuracy and adaptability of data flow classification. The real-time computational and analysis accuracy are obviously improved than the traditional algorithm, and the simulation results verify the effectiveness of the proposed algorithm.
KeywordsTrend analysis Dynamic data mining Incremental recursive least squares method Variable sliding window
This paper is supported by Natural Youth Science Foundation of China (61501326, 61401310) and Natural Science Foundation of China (61271411). It also supported by Tianjin Research Program of Application Foundation and Advanced Technology (15JCZDJC31500) and Tianjin Science Foundation (16JCYBJC16500).
- 3.Shatkay H, Zdonik S. Approximate queries and representations for large data sequences. In: Proceedings of 12th IEEE international conference on data engineering. Washington: IEEE Computer Society; 1996. p. 546–53.Google Scholar
- 4.Keogh E, Chu S, Hart D, et al. Segmenting time series: a survey and novel approach. In: Proceedings of IEEE international conference on data mining. Los Jose: IEEE Computer Society; 2001. p. 289–96.Google Scholar
- 6.Zhou Q, Wu T. Research and application of a data flow trend analysis method. Control Decis Mak. 2008;23(10):1182–5.Google Scholar