Application of solely self-attention mechanism in CSI-fingerprinting-based indoor localization

Abstract

With the growth of IoT services, there has been an increased demand for indoor localization-based services. Wi-Fi access is omnipresent; its high accuracy and ability to use commodity devices makes it suitable to be widely adopted for localization in indoor environments. Recent sequence transduction models such as recurrent neural network (RNN) and long short-term memory (LSTM) mostly rely on recurrent and convolution. Both LSTM and RNN have achieved tremendous results in the localization tasks, but their sequential character prevents them from effectively performing parallel computing, therefore, limiting their performance in processing extremely long sequences. Lately, there have been different models developed for natural language processing transduction tasks that relied solely on attention mechanism, and they performed remarkably well with less computation. This paper is the first to propose the sole utilization of self-attention mechanism for localization time-series modeling. We introduce a self-attention mechanism fingerprinting-based model (SAMFI) which uses positional encoding and masking mechanism. To capture the temporal ordering information, we used the extended symbolic aggregate approximation strategy. Moreover, the proposed model utilizes calibrated channel state information as location fingerprints. SAMFI's pivotal concept is simple and empirically potent. The obtained results significantly minimized the location error on the collected dataset with an accuracy level score of 86.5% outperforming both RNN and LSTM models which scored 82.6 and 67.5%, respectively.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. 1.

    Tegou T, Kalamaras I, Tsipouras M, Giannakeas N, Votis K (2019) A low-cost indoor activity monitoring system for detecting frailty in older adults. Sensors. https://doi.org/10.3390/s19030452

    Article  Google Scholar 

  2. 2.

    Magnago V, Palopoli L, Passerone R, Fontanelli D, Macii D (2017) A nearly optimal landmark deployment for indoor localisation with limited sensing. In: 2017 international conference on indoor positioning and indoor navigation (IPIN), pp 1–8

  3. 3.

    Li H, Zeng X, Li Y, Zhou S, Wang J (2019) Convolutional neural networks based indoor wi-fi localization with a novel kind of CSI images. China Commun 16:250–260. https://doi.org/10.23919/JCC.2019.09.019

    Article  Google Scholar 

  4. 4.

    Comiter M (2018) Localization convolutional neural networks using angle of arrival images. In: 2018 IEEE global communications conference (GLOBECOM), pp 1–7

  5. 5.

    Hamza L, Nerguizian C (2009) Neural network and fingerprinting based localization Neural Network and Fingerprinting-based in dynamic channels localization in dynamic channels. In: 2009 IEEE international symposium on intelligent signal processing, pp 253–258. https://doi.org/10.1109/WISP.2009.5286554

  6. 6.

    Garcia-valverde T et al (2013) A fuzzy logic-based system for indoor localization using WiFi in ambient intelligent environments. IEEE Trans Fuzzy Syst 21(4):702–718. https://doi.org/10.1109/TFUZZ.2012.2227975

    Article  Google Scholar 

  7. 7.

    Wang X, Wang X, Mao S (2018) Deep convolutional neural networks for indoor localization with CSI images. IEEE Trans Netw Sci Eng. https://doi.org/10.1109/TNSE.2018.2871165

    Article  Google Scholar 

  8. 8.

    Trogh J, Joseph W, Martens L, Plets D (2019) An unsupervised learning technique to optimize radio maps for indoor localization. Sensors. https://doi.org/10.3390/s19040752

    Article  Google Scholar 

  9. 9.

    Mao S, Member S, Pandey S (2017) CSI-based fingerprinting for indoor localization: a deep learning approach. IEEE Trans Veh Technol 66(1):763–776. https://doi.org/10.1109/TVT.2016.2545523

    Article  Google Scholar 

  10. 10.

    Wu C, Yang Z, Zhou Z, Qian K, Liu Y, Liu M (2015) PhaseU: real-time LOS identification with WiFi. In: 2015 IEEE conference on computer communications, vol 978, pp 2038–2046. https://doi.org/10.1109/INFOCOM.2015.7218588

  11. 11.

    Qian K, Wu C, Yang Z, Liu Y, Zhou Z (2014) PADS: passive detection of moving targets with dynamic speed using PHY layer information. In: 2014 20th IEEE international conference on parallel and distributed systems, pp 1–8. https://doi.org/10.1109/PADSW.2014.7097784

  12. 12.

    Wang X, Gao L, Mao S (2017) BiLoc: bi-modal deep learning for indoor localization with commodity 5GHz WiFi. IEEE Access 5:4209–4220

    Article  Google Scholar 

  13. 13.

    Park CJ, Moon HW, Kim W, Yoon YJ (2014) WINNER channel model with geometric optics and probability for indoor environment. In: 2014 international symposium on antennas and propagation conference proceedings, vol 2, pp 253–254. https://doi.org/10.1109/ISANP.2014.7026626

  14. 14.

    Arruti E, Mendicute M, Thompson J (2013) A novel WINNER based model for wireless communications inside train carriages. In: 2013 European modelling symposium, vol 1, pp 584–589. https://doi.org/10.1109/EMS.2013.97

  15. 15.

    Khatab ZE, Hajihoseini A, Ghorashi SA (2018) A fingerprint method for indoor localization using autoencoder based deep extreme learning machine. IEEE Sens Lett 2(1):1–4

    Article  Google Scholar 

  16. 16.

    Bao G, Zhang Y, Zeng Z (2020) Memory analysis for memristors and memristive recurrent neural networks. IEEE/CAA J Autom Sin 7(1):96–105. https://doi.org/10.1109/JAS.2019.1911828

    Article  Google Scholar 

  17. 17.

    Chan S, Sohn G (2012) Indoor localization using Wi-Fi based fingerprinting and trilateration techniques for LBS applications. Int Arch Photogramm Remote Sens Spat Inf Sci XXXVIII:3–7

    Google Scholar 

  18. 18.

    Ren Z et al (2018) Deep scalogram representations for acoustic scene classification. IEEE/CAA J Autom Sin 5(3):662–669. https://doi.org/10.1109/JAS.2018.7511066

    Article  Google Scholar 

  19. 19.

    Celso A, Dantes JD, Ravelo LA, Limjoco WJ, Tiglao NM (2018) Improving WiFi indoor localization through a peer-to-peer architecture. In: TENCON 2018–2018 IEEE region 10 conference, October, pp 2060–2065

  20. 20.

    Song H, Rajan D, Thiagarajan JJ, Spanias A (2018) Attend and diagnose: clinical time series analysis using attention models. Mach Learn (stat.ML); Mach Learn

  21. 21.

    Zhang W, Sengupta R, Fodero J, Li X (2017) DeepPositioning: intelligent fusion of pervasive magnetic field and WiFi fingerprinting for smartphone indoor localization via deep learning. In: 2017 16th IEEE international conference on machine learning and applications, pp 1–7. https://doi.org/10.1109/ICMLA.2017.0-185

  22. 22.

    Gil-Castifieira JMPCF, GonzaJez-Castafio FJ (2004) Integration of indoor location networks into the UMTS architecture: assistant location networks. In: IEEE 60th vehicular technology conference 2004. VTC2004-Fall, vol 5, no. CA, pp 3565–3568. https://doi.org/10.1109/VETECF.2004.1404728

  23. 23.

    Bahl P, Padmanabhan VN (2000) RADAR: an in-building RF-based user location and tracking system. In: Proceedings of IEEE INFOCOM 2000, no. Cat. No.00CH37064

  24. 24.

    Song Q, Guo S, Liu X, Yang Y (2018) CSI amplitude fingerprinting-based NB-IoT indoor localization. IEEE Internet Things J 5(3):1494–1504. https://doi.org/10.1109/JIOT.2017.2782479

    Article  Google Scholar 

  25. 25.

    Vaswani A et al (2017) Attention is all you need. In: Advances in neural information processing systems, vol. 2017–Decem, no. Nips, pp 5999–6009

  26. 26.

    Wen Y, Tian X, Wang X, Lu S (2015) Fundamental limits of RSS fingerprinting based indoor localization. In: 2015 IEEE conference on computer communications (INFOCOM), pp 2479–2487. https://doi.org/10.1109/INFOCOM.2015.7218637

  27. 27.

    Shen T, Zhou T, Long G, Jiang J, Zhang C (2018) Bi-directional block self-attention for fast and memory-efficient sequence modeling. In: 6th international conference on learning representations (ICLR) 2018—conference track proceedings, pp 1–18

  28. 28.

    Yang B, Wang L, Wong DF, Chao LS, Tu Z (2019) Convolutional self-attention networks. In: NAACL HLT 2019—2019 conference of the North American chapter of the association for computational linguistics: human language technologiesProceedings conference, vol 1, pp 4040–4045. https://doi.org/10.18653/v1/n19-1407

  29. 29.

    Kenton MC, Kristina L, Devlin J (1953) BERT: pre-training of deep bidirectional transformers for language understanding. Mlm

  30. 30.

    Xu S, Chou W, Dong H (2019) A robust indoor localization system integrating visual localization aided by CNN-based image retrieval with Monte Carlo localization. Sensors 19(2):249. https://doi.org/10.3390/s19020249

    Article  Google Scholar 

  31. 31.

    Wu Z, Jiang L, Jiang Z, Chen B, Liu K, Xuan Q, Xiang Y (2018) Accurate indoor localization based on csi and visibility graph. Sensors 18:1–18. https://doi.org/10.3390/s18082549

    Article  Google Scholar 

  32. 32.

    Li X, Li S, Zhang D, Xiong J, Wang Y, Mei H (2016) {Dynamic-MUSIC}: accurate device-free indoor localization. In: Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing, pp 196–207. https://doi.org/10.1145/2971648.2971665

  33. 33.

    Wang J, Fang D, Yang Z, Jiang H, Member S (2017) E-HIPA: an Energy-efficient framework for high-precision multi-target-adaptive device-free localization. IEEE Trans Mob Comput 16(3):716–729. https://doi.org/10.1109/TMC.2016.2567396

    Article  Google Scholar 

  34. 34.

    Hsieh C, Chen J, Nien B (2019) Deep learning-based indoor localization using received signal strength and channel state information. IEEE Access 7:33256–33267. https://doi.org/10.1109/ACCESS.2019.2903487

    Article  Google Scholar 

  35. 35.

    Yu Y, Zhu Y, Wan D, Liu H, Zhao Q (2019) A novel symbolic aggregate approximation for time series. Adv Intell Syst Comput 935:805–822. https://doi.org/10.1007/978-3-030-19063-7_65

    Article  Google Scholar 

  36. 36.

    Taylor WL (1953) “Cloze procedure”: a new tool for measuring readability. Journal Q 30(4):415–433

    Article  Google Scholar 

  37. 37.

    Lkhagva B, Suzuki Y, Kawagoe K (2006) New time series data representation ESAX for financial applications. In: 22nd international conference on data engineering workshops, pp x115–x115. https://doi.org/10.1109/ICDEW.2006.99

  38. 38.

    Zalewski W, Silva F, Maletzke AG, Wu FC, Lee HD (2013) Symbolic representation based on temporal order information for time series classification. In: 2013 Brazilian conference on intelligent system, pp 95–100. https://doi.org/10.1109/BRACIS.2013.24

  39. 39.

    Hertel L, Collado J, Sadowski P, Ott J, Baldi P (2020) Sherpa: robust hyperparameter optimization for machine learning. NIPS Work, no. Nips. arXiv:2005.04048

  40. 40.

    Li L, Jamieson K, Rostamizadeh A, Gonina E, Moritz JB, Mar LG (2020) A system for massively parallel hyperparameter tuning. In: Conference on machine learning and systems

  41. 41.

    Liu L et al (2019) On the variance of the adaptive learning rate and beyond. Mach Learn (cs.LG); Comput Lang (cs.CL); Mach Learn, pp 1–14. arXiv preprint arXiv:1908.03265

Download references

Acknowledgements

I would like to thank the funder of this project and all those who contributed to this research project. This work was supported by the School of Computer and Communication Engineering University of Science and Technology Beijing, China.

Author information

Affiliations

Authors

Contributions

Kabo Poloko Nkabiti contributed to conceptualization, methodology, software, data curation, writing—original draft preparation, visualization, investigation, validation, formal analysis and investigation. Yue-Yun Chen contributed to supervision, writing—reviewing and editing, resources, project administration and funding acquisition.

Corresponding author

Correspondence to Yueyun Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nkabiti, K.P., Chen, Y. Application of solely self-attention mechanism in CSI-fingerprinting-based indoor localization. Neural Comput & Applic (2021). https://doi.org/10.1007/s00521-020-05681-1

Download citation

Keywords

  • Channel state information (CSI)
  • Fingerprinting (FP)
  • Indoor localization
  • Self-attention mechanism (SAM)
  • Deep learning