Adaptive Dimensionality Adjustment for Online “Principal Component Analysis”
Many applications in the Industrial Internet of Things and Industry 4.0 rely on large amounts of data which are continuously generated. The exponential growth in available data and the resulting storage requirements are often underestimated bottlenecks. Therefore, efficient dimensionality reduction gets more attention and becomes more relevant. One of the most widely used techniques for dimensionality reduction is “Principal Component Analysis” (PCA). A novel algorithm to determine the optimal number of meaningful principal components on a data stream is proposed. The basic idea of the proposed algorithm is to optimize the dimensionality adjustment process by taking advantage of several “natural” PCA features. In contrast to the commonly used approach to start with a maximal set of principal components and apply some sort of stopping rule, the proposed algorithm starts with a minimal set of principal components and uses a linear regression model in the natural logarithmic scale to approximate the remaining components. An experimental study is presented to demonstrate the successful application of the algorithm to noisy synthetic and real world data sets.
KeywordsPrincipal Component Analysis Adaptive dimensionality adjustment Stopping rule Big data Industry 4.0 Industrial IoT
This work was supported by the EFRE-NRW funding programme “Forschungsinfrastrukturen” (grant no. 34.EFRE-0300119).
- 1.Careercon 2019 - help navigate robots. https://www.kaggle.com/c/career-con-2019/data. Accessed 20 May 2019
- 7.Katal, A., Wazid, M., Goudar, R.H.: Big data: issues, challenges, tools and good practices. In: 2013 Sixth International Conference on Contemporary Computing (IC3), pp. 404–409 (2013)Google Scholar
- 11.Schenck, W.: Adaptive Internal Models for Motor Control and Visual Prediction. MPI Series in Biological Cybernetics, 1st edn. Logos Verlag Berlin, Berlin (2008)Google Scholar
- 13.Zhang, T., Yang, B.: Big data dimension reduction using PCA. In: 2016 IEEE International Conference on Smart Cloud (SmartCloud), pp. 152–157 (2016)Google Scholar