Skip to main content
SpringerLink
Log in
Book cover

Proceedings of the Computational Methods in Systems and Software

CoMeSySo 2019 2019: Intelligent Systems Applications in Software Engineering pp 69–82Cite as

Efficient Sensory Data Transformation: A Big Data Approach

  • Conference paper
  • First Online:

  • 550 Accesses

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1046))

Abstract

Big Data Analytics has immensely solved complex problems involving massive and complex data. Data filtering plays a vital role in helping the analysis processes to perform analytics with ease and in precise form. The data variety and veracity are some of the major problems that add enough complexity to data, creating the overall hindrance to an effective big data analytics process. The proposed work targets the data staging phase in a big data classification to tackle variety and veracity problems found in massive sensory data. The study adopts a novel approach for data storage, then designs and implements a simple algorithm to perform data transformation. The concept of cloud storage-bucket is used for effective storage and transformation of sensory data. Such an analytical approach is proven to retain the capability of an effective and faster data transformation. The algorithm performs conversion of unstructured data to semi-structured data in the first stage, then converts the semi-structured data to structured data in the second stage, and finally stores the resulting structured data into virtually localized distributed storage. The outcome of this study offers faster response time and higher data purity for data transformation process of data staging phase in any big data analytics application.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Trovati, M., Hill, R., Zhu, S., Liu, L.: Big-Data Analytics and Cloud Computing. Springer, Heidelberg (2015)

    Book  Google Scholar 

  2. Mazumder, S., Bhadoria, R.S., Deka, G.C.: Distributed Computing in Big Data Analytics: Concepts, Technologies and Applications. Springer, Heidelberg (2017)

    Book  Google Scholar 

  3. Zanoon, N., Al-Haj, A., Khwaldeh, S.M.: Cloud computing and big data is there a relation between the two: a study. Int. J. Appl. Eng. Res. 12(17), 6970–6982 (2017)

    Google Scholar 

  4. Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I.A.T., Siddiqa, A., Yaqoob, I.: Big IoT data analytics: architecture, opportunities, and open research challenges. IEEE Access 5, 5247–5261 (2017)

    Article  Google Scholar 

  5. Lv, Z., Song, H., Basanta-Val, P., Steed, A., Jo, M.: Next-generation big data analytics: state of the art, challenges, and future research topics. IEEE Trans. Ind. Inform. 13(4), 1891–1899 (2017)

    Article  Google Scholar 

  6. Marr, B.: Big Data: Using SMART Big Data, Analytics and Metrics to Makebetter Decisions and Improve Performance. Wiley, Hoboken (2015)

    Google Scholar 

  7. Li, K.C., Jiang, H., Zomaya, A.Y.: Big Data Management and Processing. CRC Press, Cambridge (2017)

    Book  Google Scholar 

  8. Puthal, D.: Lattice-modelled information flow control of big sensing data streams for smart health application. IEEE Internet of Things J. (2018)

    Google Scholar 

  9. Han, G., Guizani, M., Lloret, J., Chan, S., Wan, L., Guibene, W.: Emerging trends, issues, and challenges in big data and its implementation toward future smart cities: part 2. IEEE Commun. Mag. 56(2), 76–77 (2018)

    Article  Google Scholar 

  10. Habibzadeh, H., Boggio-Dandry, A., Qin, Z., Soyata, T., Kantarci, B., Mouftah, H.T.: Soft sensing in smart cities: handling 3Vs using recommender systems, machine intelligence, and data analytics. IEEE Commun. Mag. 56(2), 78–86 (2018)

    Article  Google Scholar 

  11. Raafat, H.M., Hossain, M.S., Essa, E., Elmougy, S., Tolba, A.S., Muhammad, G., Ghoneim, A.: Fog intelligence for real-time IoT sensor data analytics. IEEE Access 5, 24062–24069 (2017)

    Article  Google Scholar 

  12. Al-Ali, A., Zualkernan, I.A., Rashid, M., Gupta, R., Alikarar, M.: A smart home energy management system using IoT and big data analytics approach. IEEE Trans. Consum. Electron. 63(4), 426–434 (2017)

    Article  Google Scholar 

  13. ur Rehman, M.H., Ahmed, E., Yaqoob, I., Hashem, I.A.T., Imran, M., Ahmad, S.: Big data analytics in industrial IoT using a concentric computing model. IEEE Commun. Mag. 56(2), 37–43 (2018)

    Article  Google Scholar 

  14. Yang, C., Puthal, D., Mohanty, S.P., Kougianos, E.: Big-sensing-data curation for the cloud is coming: a promise of scalable cloud-data-center mitigation for next-generation IoT and wireless sensor networks. IEEE Consum. Electron. Mag. 6(4), 48–56 (2017)

    Article  Google Scholar 

  15. Zhang, D., He, T., Lin, S., Munir, S., Stankovic, J.A.: Taxi-passenger-demand modeling based on big data from a roving sensor network. IEEE Trans. Big Data 3(3), 362–374 (2017)

    Article  Google Scholar 

  16. Din, S., Ahmad, A., Paul, A., Rathore, M.M.U., Jeon, G.: A cluster-based data fusion technique to analyze big data in wireless multi-sensor system. IEEE Access 5, 5069–5083 (2017)

    Article  Google Scholar 

  17. Cheng, S., Cai, Z., Li, J., Gao, H.: Extracting kernel dataset from big sensory data in wireless sensor networks. IEEE Trans. Know. Data Eng. 29(4), 813–827 (2017)

    Article  Google Scholar 

  18. Ebner, K., Buhnen, T., Urbach, N.: Think big with big data: identifying suitable big data strategies in corporate environments. In: 47th Hawaii International Conference on System Sciences (HICSS), pp. 3748–3757. IEEE (2014)

    Google Scholar 

  19. Hu, G., Zhang, X., Duan, N., Gao, P.: Towards reliable online services analyzing mobile sensor big data. In: IEEE International Conference on Web Services (ICWS), pp. 849–852. IEEE (2017)

    Google Scholar 

  20. Ren, M., Li, J., Guo, L., Li, X., Fan, W.: Distributed data aggregation scheduling in multi-channel and multi-power wireless sensor networks. IEEE Access (2017)

    Google Scholar 

  21. Takaishi, D., Nishiyama, H., Kato, N., Miura, R.: Toward energy efficient big data gathering in densely distributed sensor networks. IEEE Trans. Emerg. Top. Comput. 2(3), 388–397 (2014)

    Article  Google Scholar 

  22. Karim, L., Al-kahtani, M.S.: Sensor data aggregation in a multi-layer big data framework. In: 7th IEEE Annual Conference on Information Technology, Electronics and Mobile Communication (IEMCON), pp. 1–7. IEEE (2016)

    Google Scholar 

  23. Jeong, M.H., Sullivan, C.J., Wang, S.: Complex radiation sensor network analysis with big data analytics. In: IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), pp. 1–4. IEEE (2015)

    Google Scholar 

  24. Kandah, F.I., Nichols, O., Yang, L.: Efficient key management for big data gathering in dynamic sensor networks. In: International Conference on Computing, Networking and Communications (ICNC), pp. 667–671. IEEE (2017)

    Google Scholar 

  25. Zhu, C., Shu, L., Leung, V.C., Guo, S., Zhang, Y., Yang, L.T.: Secure multimedia big data in trust-assisted sensor-cloud for smart city. IEEE Commun. Mag. 55(12), 24–30 (2017)

    Article  Google Scholar 

  26. Li, J., Guo, S., Yang, Y., He, J.: Data aggregation with principal component analysis in big data wireless sensor networks. In: 12th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN), pp. 45–51. IEEE (2016)

    Google Scholar 

  27. Miao, W., Zheng, D., Hangyu, G., Tao, Y.: Research on big data management and analysis method of multi-platform avionics system. In: 16th International Conference on Computer and Information Science (ICIS), pp. 757–761. IEEE (2017)

    Google Scholar 

  28. Onal, A.C., Sezer, O.B., Ozbayoglu, M., Dogdu, E.: Weather data analysis and sensor fault detection using an extended IoT framework with semantics, big data, and machine learning. In: IEEE International Conference on Big Data, pp. 2037–2046. IEEE (2017)

    Google Scholar 

  29. Latha, P., Vasantha, R.: MDS-WLAN: maximal data security in wlan for resisting potential threats. Int. J. Electr. Comput. Eng. 5(4), 859 (2015)

    Google Scholar 

  30. Wiska, R., Habibie, N., Wibisono, A., Nugroho, W.S., Mursanto, P.: Big sensor-generated data streaming using Kafka and impala for data storage in wireless sensor network for \({\rm CO}_2\) monitoring. In: International Workshop on Big Data and Information Security (IWBIS), pp. 97–102. IEEE (2016)

    Google Scholar 

  31. Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)

    Article  Google Scholar 

  32. Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Khan, S.U.: The rise of big data on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015)

    Article  Google Scholar 

  33. Cai, H., Xu, B., Jiang, L., Vasilakos, A.V.: IoT-based big data storage systems in cloud computing: perspectives and challenges. IEEE Internet of Things J. 4(1), 75–87 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. REVA University, Bengaluru, India

    Akram Pasha

  2. Sambhram Institute of Technology, Bengaluru, India

    P. H. Latha

Authors
  1. Akram Pasha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. H. Latha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akram Pasha .

Editor information

Editors and Affiliations

  1. Faculty of Applied Informatics, Tomas Bata University in Zlin, Zlin, Czech Republic

    Radek Silhavy

  2. Faculty of Applied Informatics, Tomas Bata University in Zlin, Zlin, Czech Republic

    Petr Silhavy

  3. Faculty of Applied Informatics, Tomas Bata University in Zlin, Zlin, Czech Republic

    Zdenka Prokopova

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pasha, A., Latha, P.H. (2019). Efficient Sensory Data Transformation: A Big Data Approach. In: Silhavy, R., Silhavy, P., Prokopova, Z. (eds) Intelligent Systems Applications in Software Engineering. CoMeSySo 2019 2019. Advances in Intelligent Systems and Computing, vol 1046. Springer, Cham. https://doi.org/10.1007/978-3-030-30329-7_7

Download citation

Publish with us

Policies and ethics

Search

Navigation