Skip to main content

Forecasting Multivariate Time-Series Data Using LSTM and Mini-Batches

Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT,volume 45)

Abstract

Multivariate time-series data forecasting is a challenging task due to nonlinear interdependencies in complex industrial systems. It is crucial to model these dependencies automatically using the ability of neural networks to learn features by extraction of spatial relationships. In this paper, we converted non-spatial multivariate time-series data into a time-space format and used Recurrent Neural Networks (RNNs) which are building blocks of Long Short-Term Memory (LSTM) networks for sequential analysis of multi-attribute industrial data for future predictions. We compared the effect of mini-batch length and attribute numbers on prediction accuracy and found the importance of spatio-temporal locality for detecting patterns using LSTM.

Keywords

  • LSTM
  • Multivariate time-series
  • RNN
  • Sensors
  • Sequence data
  • Time-series

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-37309-2_10
  • Chapter length: 9 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-37309-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

References

  1. Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 95–104 (2018)

    Google Scholar 

  2. Langkvist, M., Karlsson, L., Loutfi, A.: A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn. Lett. 42, 11–24 (2014)

    CrossRef  Google Scholar 

  3. Jiang, Q., Tang, C., Chen, C., Wang, X., Huang, Q.: Stock price forecast based on LSTM neural network. In: International Conference on Management Science and Engineering Management, pp. 393–408. Springer (2018)

    Google Scholar 

  4. Varol, G., Laptev, I., Schmid, C.: Long-term temporal convolutions for action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1510–1517 (2018)

    CrossRef  Google Scholar 

  5. Laptev, N., Yosinski, J., Li, L.E., Smyl, S.: Time-series extreme event forecasting with neural networks at Uber. In: International Conference on Machine Learning, vol. 34, pp. 1–5 (2017)

    Google Scholar 

  6. Groß, W., Lange, S., Bödecker, J., Blum, M.: Predicting time series with space-time convolutional and recurrent neural networks. In: Proceeding of European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 71–76 (2017)

    Google Scholar 

  7. Lee, K.B., Cheon, S., Kim, C.O.: A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes. IEEE Trans. Semicond. Manuf. 30(2), 135–142 (2017)

    CrossRef  Google Scholar 

  8. Troiano, L., Villa, E.M., Loia, V.: Replicating a trading strategy by means of LSTM for financial industry applications. IEEE Trans. Ind. Inform. 14(7), 3226–3234 (2018)

    CrossRef  Google Scholar 

  9. Shih, S.Y., Sun, F.K., Lee, H.Y.: Temporal pattern attention for multivariate time series forecasting. arXiv preprint arXiv:1809.04206 (2018)

  10. TÜPRAŞ Refinery. http://tupras.com.tr/en/rafineries. Accessed 6 Dec 2018

  11. Box, G.E., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control. Wiley, Hoboken (2015)

    MATH  Google Scholar 

  12. Loganathan, G., Samarabandu, J., Wang, X.: Sequence to sequence pattern learning algorithm for real-time anomaly detection in network traffic. In: 2018 IEEE Canadian Conference on Electrical & Computer Engineering (CCECE), pp. 1–4 (2018)

    Google Scholar 

  13. Khodabakhsh, A., Ari, I., Bakir, M., Ercan, A.O.: Multivariate sensor data analysis for oil refineries and multi-mode identification of system behavior in real-time. IEEE Access 6, 64389–64405 (2018)

    CrossRef  Google Scholar 

  14. Khodabakhsh, A., Ari, I., Bakir, M., Alagoz, S.M.: Stream analytics and adaptive windows for operational mode identification of time-varying industrial systems. In: 2018 IEEE International Congress on Big Data (BigData Congress), pp. 242–246 (2018)

    Google Scholar 

  15. Abadi, M., Barham, P., Chen, J., Chen, Z., et al.: TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016, pp. 265–283 (2016)

    Google Scholar 

  16. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

Download references

Acknowledgments

This research was sponsored by a grant from TÜPRAŞ (Turkish Petroleum Refineries Inc.) R&D group. We would like to thank Burak Aydoğan and Mehmet Aydin for collecting and providing us with sensor data.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Athar Khodabakhsh .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Khodabakhsh, A., Ari, I., Bakır, M., Alagoz, S.M. (2020). Forecasting Multivariate Time-Series Data Using LSTM and Mini-Batches. In: Bohlouli, M., Sadeghi Bigham, B., Narimani, Z., Vasighi, M., Ansari, E. (eds) Data Science: From Research to Application. CiDaS 2019. Lecture Notes on Data Engineering and Communications Technologies, vol 45. Springer, Cham. https://doi.org/10.1007/978-3-030-37309-2_10

Download citation