Abstract
With the booming of the Internet of Things, tremendous amount of sensors have been installed in different geographic locations, generating massive sensory data with both time-stamps and geo-tags. Such type of data usually have shown complex spatio-temporal correlation and are easily missing in practice due to communication failure or data corruption. In this paper, we aim to tackle the challenge – how to accurately and efficiently recover the missing values for corrupted spatio-temporal sensory data. Specifically, we first formulate such sensor data as a high-dimensional tensor that can naturally preserve sensors’ both geographical and time information, thus we call spatio-temporal Tensor. Then we model the sensor data recovery as a low-rank robust tensor completion problem by exploiting its latent low-rank structure and sparse noise property. To solve this optimization problem, we design a highly efficient optimization method that combines the alternating direction method of multipliers and accelerated proximal gradient to minimize the tensor’s convex surrogate and noise’s \(\ell _1\)-norm. In addition to testing our method by a synthetic dataset, we also use passive RFID (radio-frequency identification) sensors to build a real-world sensor-array testbed, which generates overall 115,200 sensor readings for model evaluation. The experimental results demonstrate the accuracy and robustness of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The rank of a matrix is often linked to the order, complexity, or dimensionality of the underlying system, which tends to be much smaller than the data size.
- 2.
We assume the additive noises are sufficiently sparse relative to the data tensor \(\mathcal {O}\).
- 3.
- 4.
Available in www.sandia.gov/~tgkolda/TensorToolbox/index-2.6.html.
- 5.
Available in http://sun.stanford.edu/~rmunk/PROPACK/.
- 6.
Available in au.mathworks.com/help/curvefit/smoothing-data.html.
- 7.
Available in www.cis.pku.edu.cn/faculty/vision/zlin/RPCA+MC_codes.zip.
- 8.
Available in www.cs.rochester.edu/u/jliu/code/TensorCompletion.zip.
- 9.
Available in https://github.com/ryotat/tensor.
- 10.
In each iteration, all the tensor completion based methods require to calculate 3 times SVD that is most time-consuming computation task so we use iteration number as evaluation metric for computation time.
- 11.
We collect over all one hour’s RSS readings, the sampling rate is 2 Hz. During the data collection, a participant is doing various activities between the RFID sensor-array and reader, including walking, sitting, standing, lying down as well as falling down etc. By doing so, the collected RSSI reading will reveal different patterns.
References
Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemometr. Intell. Lab. Syst. 106(1), 41–56 (2011)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)
Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)
Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 11 (2011)
Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)
Chen, Y., Jalali, A., Sanghavi, S., Caramanis, C.: Low-rank matrix recovery from errors and erasures. IEEE Trans. Info. Theor. 59(7), 4324–4337 (2013)
Chen, Y., Xu, H., Caramanis, C., Sanghavi, S.: Robust matrix completion with corrupted columns. arXiv preprint (2011). arXiv:1102.2254
Da Silva, C., Herrmann, F.J.: Hierarchical tucker tensor optimization-applications to tensor completion. In: Proceedings of 10th International Conference on Sampling Theory and Applications (2013)
Da Xu, L., He, W., Li, S.: Internet of things in industries: a survey. IEEE Trans. Industr. Inf. 10(4), 2233–2243 (2014)
Gandy, S., Recht, B., Yamada, I.: Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Prob. 27(2), 025010 (2011)
Goldfarb, D., Qin, Z.: Robust low-rank tensor recovery: models and algorithms. SIAM J. Matrix Anal. Appl. 35(1), 225–253 (2014)
Grewal, M.S.: Kalman Filtering. Springer, Heidelberg (2011)
Hazan, T., Polak, S., Shashua, A.: Sparse image coding using a 3D non-negative tensor factorization. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 50–57. IEEE (2005)
Hsieh, H.P., Lin, S.D., Zheng, Y.: Inferring air quality for station location recommendation based on urban big data. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 437–446 (2015)
Ji, H., Huang, S., Shen, Z., Xu, Y.: Robust video restoration by joint sparse and low rank matrix approximation. SIAM J. Imaging Sci. 4(4), 1122–1142 (2011)
Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)
Kreimer, N., Stanton, A., Sacchi, M.D.: Tensor completion based on nuclear norm minimization for 5D seismic data reconstruction. Geophysics 78(6), V273–V284 (2013)
Kressner, D., Steinlechner, M., Vandereycken, B.: Low-rank tensor completion by Riemannian optimization. BIT Numer. Math. 54(2), 447–468 (2014)
Lawrance, A., Lewis, P.: An exponential moving-average sequence and point process. J. Appl. Probab. 14, 98–113 (1977)
Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint (2010). arXiv:1009.5055
Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2114–2121. IEEE (2009)
Norcia, A.M., Clarke, M., Tyler, C.W.: Digital filtering and robust regression techniques for estimating sensory thresholds from the evoked potential. IEEE Eng. Med. Biol. Mag. 4(4), 26–32 (1985)
Ruan, W., Sheng, Q.Z., Yao, L., Gu, T., Ruta, M., Shangguan, L.: Device-free indoor localization and tracking through human-object interactions. In: 2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), pp. 1–9. IEEE (2016)
Ruan, W., Xu, P., Sheng, Q.Z., Tran, N.K., Falkner, N.J., Li, X., Zhang, W.E.: When sensor meets tensor: filling missing sensor values through a tensor approach. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 2025–2028. ACM (2016)
Ruan, W., Yao, L., Sheng, Q.Z., Falkner, N., Li, X., Gu, T.: TagFall: towards unobstructive fine-grained fall detection based on UHF passive RFID tags. In: Proceedings of the 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, pp. 140–149 (2015)
Sakurai, Y., Matsubara, Y., Faloutsos, C.: Mining and forecasting of big time-series data. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 919–922. ACM (2015)
Signoretto, M., De Lathauwer, L., Suykens, J.A.: Nuclear norms for tensors and their use for convex multilinear estimation. Linear Algebra Appl. 43 (2010)
Sun, J., Tao, D., Faloutsos, C.: Beyond streams and graphs: dynamic tensor analysis. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 374–383. ACM (2006)
Toh, K.C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac. J. Optim. 6(615–640), 15 (2010)
Tomioka, R., Hayashi, K., Kashima, H.: Estimation of low-rank tensors via convex optimization. arXiv preprint (2010). arXiv:1010.0789
Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Advances in Neural Information Processing Systems, pp. 2080–2088 (2009)
Xiuwen Yi, Y., Zheng, J.: ST-MVL: filling missing values in geo-sensory time series data. In: IJCAI 2016 (2016)
Yao, L., Ruan, W., Sheng, Q.Z., Li, X., Falkner, N.J.: Exploring tag-free RFID-based passive localization and tracking via learning-based probabilistic approaches. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1799–1802. ACM (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ruan, W., Xu, P., Sheng, Q.Z., Falkner, N.J.G., Li, X., Zhang, W.E. (2017). Recovering Missing Values from Corrupted Spatio-Temporal Sensory Data via Robust Low-Rank Tensor Completion. In: Candan, S., Chen, L., Pedersen, T., Chang, L., Hua, W. (eds) Database Systems for Advanced Applications. DASFAA 2017. Lecture Notes in Computer Science(), vol 10177. Springer, Cham. https://doi.org/10.1007/978-3-319-55753-3_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-55753-3_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55752-6
Online ISBN: 978-3-319-55753-3
eBook Packages: Computer ScienceComputer Science (R0)