Internet of Things Based Smart Community Design and Planning Using Hadoop-Based Big Data Analytics

  • Muhammad BabarEmail author
  • Waseem Iqbal
  • Sarah Kaleem
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 69)


The current spreading out in big data is offering a hefty invention potential in itinerary of the fresh epoch of smart community. The foremost endeavor of smart community is to competently employ the asset of Big Data to manage and determine the issues face by recent smart cities for enhanced decision making. The applications of smart city fabricate a gigantic number of data that compose Big Data. This research proposes Big Data analytics architecture to address the challenges in Big Data analytics using Hadoop framework. The proposed framework is dealing particularly with data loading and processing. The proposal is consist of two parts that are Big Data loading (storage) in Hadoop file system and Big Data computation. The first part is liable for transferring Big Data from outer world and storing in Hadoop. The second part of the research deals with the data processing. YARN-based cluster management solution is provided to manage the cluster resource and process the data using Map-Reduce algorithm separately unlike traditional MapReduce architecture. The proposed architecture is tested with a variety of reliable datasets using Hadoop framework to verify and expose that the architecture offers precious imminent into the society organizations for development to improve the existing smart city architecture.


IoT Big data Hadoop Smart community 


  1. 1.
    Snijders, C., Matzat, U., Reips, U.D.: Big data: big gaps of knowledge in the field of internet science. Int. J. Internet Sci. 7(1), 1–5 (2012)Google Scholar
  2. 2.
    Hurwitz, J., Nugent, A., Halper, F., Kaufman, M.: Big Data for Dummies. Wiley, Hoboken (2013)Google Scholar
  3. 3.
    Villars, R.L., Olofson, C.W., Eastwood, M:. Big data: what it is and why you should care. White Paper, IDC (2011)Google Scholar
  4. 4.
    Gantz, J., Reinsel, D.: The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East, sponsored by EMC Corporation, December, 2012 white paper Big Data Meets Big Data AnalyticGoogle Scholar
  5. 5.
    Big Data: A New World of Opportunities, Networked European Software and Services Initiative (NESSI) White Paper, December 2012Google Scholar
  6. 6.
    Li, B.: Survey of Recent Research Progress and Issues in Big Data, December 2013Google Scholar
  7. 7.
    Gang, L.: Applications and development of Hadoop. Zhangtu Information Technology Inc., Beijing (2014)Google Scholar
  8. 8.
    Lublisnky, B., Smith, K.T., Yakubovich, A.: Professional Hadoop Solutions. Wros Press (2013)Google Scholar
  9. 9.
    White, T.: Hadoop: The Definitive Guide, 3rd edn. O’Reilly Press, Sebastopol (2012)Google Scholar
  10. 10.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), March 2010Google Scholar
  11. 11.
    Ahn, H.Y., Lee, K.H., Lee, S.H., Lee, Y.J., Lee, S.M., Kim, Y.K.: An efficient method for enhancing the storage efficiency in Hadoop DFS. J. KISS Comput. Pract. 19(3), 144–148 (2013)Google Scholar
  12. 12.
    Cheng, B., Longo, S.,Cirillo, F., Bauer, M., Kovacs, E.: Building a big data platform for smart cities: experience and lessons from santander. In: Proceedings of the 4th IEEE International Congress on Big Data (BigData Congress 2015), New York, NY, USA, pp. 592–599, July 2015Google Scholar
  13. 13.
    Sanchez, L., Muñoz, L., Galache, J.A., et al.: SmartSantander: IoT experimentation over a smart city testbed. Comput. Netw. 61, 217–238 (2014)CrossRefGoogle Scholar
  14. 14.
    Rong, W., Xiong, Z., Cooper, D., Li, C., Sheng, H.: Smartcity architecture: a technology guide for implementation and design challenges. China Commun. 11(3), 56–69 (2014)CrossRefGoogle Scholar
  15. 15.
    American Planning Association, Making Great Communities Happen, United States of America, (USA).
  16. 16.
    Rocky Mountain Institute, Colorado, United States.
  17. 17.
    World Resources Institute: Making Big Ideas Happen, Washington, D.C., United States, Founded: 1982.
  18. 18.
    Smart Cities Council, Livability, Workability, and Sustainability, Smart Cities Council, Inc 1900 Campus Commons Drive, Suite 100 Reston, VA 20191Google Scholar
  19. 19.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th Conference on Symposium on Opearting Systems Design and Implementation, vol. 6, p. 10 (2004)Google Scholar
  20. 20.
    Vavilapalli, V.K., Murthy, A.C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., Saha, B., Curino, C., O’Malley, O., Radia, S., Reed, B., Baldeschwieler, E.: Apache hadoop YARN: yet another resource negotiator. In: Proceedings of 4th ACM Symposium on Cloud Computing (SoCC 2013). ACM (2013)Google Scholar
  21. 21.
    He, B., Fang, W., Luo, Q., Govindaraju, N.K., Wang, T.: Mars: a MapReduce framework on graphics processors. In: Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques - PACT 2008, p. 260 (2008)Google Scholar
  22. 22.
    Lin, J.C., et al.: ABS-YARN: a formal framework for modeling Hadoop YARN clusters. In: International Conference on Fundamental Approaches to Software Engineering. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  23. 23.
    Kulkarni, A.P., Khandewal, M.: Survey on Hadoop and introduction to YARN. Int. J. Emerg. Technol. Adv. Eng. 4(5), 82–87 (2014)Google Scholar
  24. 24.
    Yang, G. (2011). The application of MapReduce in the cloud computing. In: 2011 2nd International Symposium on Intelligence Information Processing and Trusted Computing (IPTC), Hubei, RPC, 22–23 October 2011. IEEE (2011)Google Scholar
  25. 25.
    Uppoor, S., Trullols-Cruces, O., Fiore, M., Barcelo-Ordinas, J.M.: Generation and analysis of a large-scale urban vehicular mobility dataset. IEEE Trans. Mobile Comput. 13(5), 1061–1075 (2014)CrossRefGoogle Scholar
  26. 26.
    Ning, Huansheng, Wang, Ziou: Future Internet of Things architecture: like mankind neural system or social organization framework? Commun. Lett. IEEE 15(4), 461–463 (2011)CrossRefGoogle Scholar
  27. 27.
    Schatzinger, S., Lim, C.Y.R.: Taxi of the future: big data analysis as a framework for future urban fleets in smart cities. In: Smart and Sustainable Planning for Cities and Regions, pp. 83–98. Springer International Publishing (2017)Google Scholar
  28. 28.
    Nguyen, T.H., Nunavath, V., Prinz, A.: Big data metadata management in smart grids. In: Studies in Computational Intelligence, pp. 189–214. Springer Verlag (2014)Google Scholar
  29. 29.
    Le, X.H., Lee, S., Truc, P.T., Khattak, A.M., Han, M., Hung, D.V., Hassan, M.M., et al.: Secured WSN-integrated cloud computing for u-life care. In: Proceedings of the 7th IEEE Conference on Consumer Communications and Networking Conference, pp. 702–703. IEEE Press (2010)Google Scholar
  30. 30.
    Babar, Muhammad, Arif, Fahim: Smart urban planning using big data analytics to contend with the interoperability in Internet of Things. Future Gener. Comput. Syst. 77, 65–76 (2017)CrossRefGoogle Scholar
  31. 31.
    Babar, M., Rahman, A., Arif, F., Jeon, G.: Energy-harvesting based on internet of things and big data analytics for smart health monitoring. Sustainable Comput. Inform. Syst. 20, 155–164 (2017)CrossRefGoogle Scholar
  32. 32.
    Dataset, Dataset Collection. Accessed 12 Jan 2017

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Iqra UniversityIslamabadPakistan
  2. 2.National University of Sciences and TechnologyIslamabadPakistan
  3. 3.Iqra National UniversityPeshawarPakistan

Personalised recommendations