Skip to main content
SpringerLink
Log in

Big Data and the Internet of Things

  • Chapter
  • First Online:
Big Data Analysis: New Algorithms for a New Society

Part of the book series: Studies in Big Data ((SBD,volume 16))

Abstract

Advances in sensing and computing capabilities are making it possible to embed increasing computing power in small devices. This has enabled the sensing devices not just to passively capture data at very high resolution but also to take sophisticated actions in response. Combined with advances in communication, this results in an ecosystem of highly interconnected devices referred to as the Internet of Things—IoT. In conjunction, the advances in machine learning have allowed building models on this ever increasing amount of data. Consequently, devices all the way from heavy assets such as aircraft engines to wearables such as health monitors can all now not only generate massive amounts of data but can draw back on aggregate analytics to “improve” their performance over time. Big data analytics has been identified as a key enabler for the IoT. In this chapter, we discuss various avenues of the IoT where big data analytics either is already making a significant impact or is on the cusp of doing so. We also discuss social implications and areas of concern.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that we use the term “edge devices” loosely to encompass not just the devices such as RFID tags but also other sensors esp. MEMS, including embedded sensors, monitoring and diagnostic sensors aboard industrial assets and so on.

  2. 2.

    We do briefly cover some of these categories since they are indeed critical and the effectiveness of IoT applications and capabilities are highly contingent on effective solutions in these areas.

  3. 3.

    Cyber-physical systems consist of computational and physical components that are able to perceive real-time changes as a result of seamless integration [53].

  4. 4.

    Oil and gas industry can be viewed in three different segments: Upstream (concerned with exploration, drilling/development and production), Midstream (concerend with trading, transportation and refining) and Downstream (concerned with bulk distribution and retail). Refining step has components in both midstream and downstream sectors.

  5. 5.

    https://www.bosch-si.com/solutions/energy/virtual-power-plant/virtual-power-plant.html.

  6. 6.

    See http://www.mybraintech.com/.

  7. 7.

    http://www.delivering-tomorrow.com/.

  8. 8.

    Health Insurance Portability and Accountability Act.

References

  1. Aggarwal, C.C., Abdelzaher, T.: Integrating sensors and social networks. In: Aggarwal, C.C. (ed.) Social Network Data Analytics, pp. 379–412. Springer, US (2011). doi:10.1007/978-1-4419-8462-3_14; ISBN:978-1-4419-8461-6; http://dx.doi.org/10.1007/978-1-4419-8462-3_14

    Google Scholar 

  2. Aggarwal, C.C., Ashish, N., Sheth, A.: The internet of things: a survey from the data-centric perspective. In: Managing and Mining Sensor Data, pp. 383–428. Springer (2013)

    Google Scholar 

  3. Baaziz, A., Quoniam, L.: How to use big data technologies to optimize operations in upstream petroleum industry. Int. J. Innov. (IJI) 1(1), 30–42 (2013)

    Google Scholar 

  4. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learning Res. 3, 1137–1155 (2003)

    Google Scholar 

  5. Bettencourt, L.M.A.: The uses of big data in cities. Santa Fe Institute working paper 2013-09-029, September 2013. http://www.santafe.edu/media/workingpapers/13-09-029.pdf

  6. Bosch MongoDB white-paper: IoT and big data. Technical report, October 2014. http://info.mongodb.com/rs/mongodb/images/MongoDB_BoschSI_IoT_BigData.pdf

  7. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  8. Brasco, C., Eklund, N., Shah, M., Marthaler, D.: Predictive modeling of high-bypass turbofan engine deterioration. In: Proceedings of the Annual Conference of the Prognostics and Health Management Society (PHM 2013), vol. 4. PHM Society (2013). http://www.phmsociety.org/node/1104

  9. Bui, N., Zorzi, M.: Health care applications: a solution based on the internet of things. In: Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication Technologies, ISABEL ’11, pp. 131:1–131:5. ACM, New York, NY, USA (2011). http://doi.acm.org/10.1145/2093698.2093829

  10. Byrnes, N.: Cities find rewards in cheap technologies. MIT Technology Review, November 2014. http://www.technologyreview.com/news/532466/cities-find-rewards-in-cheap-technologies/

  11. Chui, M., Löffler, M., Roberts, R.: The internet of things. McKinsey Quarterly 2, 1–9 (2010). http://www.mckinsey.com/insights/high_tech_telecoms_internet/the_internet_of_things

  12. Cognizant Report: Reaping the benefits of the internet of things. Technical Report, May 2014. http://www.cognizant.com/InsightsWhitepapers/Reaping-the-Benefits-of-the-Internet-of-Things.pdf

  13. Crankshaw, D., Bailis, P., Gonzalez, J.E., Li, H., Zhang, Z., Franklin, M.J., Ghodsi, A., Jordan, M.I.: The missing piece in complex analytics: low latency, scalable model management and serving with velox. In: Conference on Innovative Data Systems Research (CIDR). Asilomar, CA (2014)

    Google Scholar 

  14. Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P., Yang, K., Le, Q.V., Ng, A.Y.: Large scale distributed deep networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1223–1231. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4687-large-scale-distributed-deep-networks.pdf

  15. Deb, B., Shah, M., Evans, S., Mehta, M., Gargulak, A., Lasky, T.: Towards systems level prognostics in the cloud. In: Proceedings of the IEEE Conference on Prognostics and Health Management (PHM), pp. 1–6. IEEE (2013). ISBN:978-1-4673-5722-7

    Google Scholar 

  16. Dohr, A., Modre-Opsrian, R., Drobics, M., Hayn, D., Schreier, G.: The internet of things for ambient assisted living. In: Seventh International Conference on Information Technology: New Generations (ITNG), 2010, pp. 804–809. IEEE (2010)

    Google Scholar 

  17. Doukas, C., Maglogiannis, I.: Bringing IoT and cloud computing towards pervasive healthcare. In: 2012 Sixth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS), pp. 922–926, July 2012. doi:10.1109/IMIS.2012.26

  18. Feblowitz, J.: The big deal about big data in upstream oil and gas. IDC Energy Insights, October 2012

    Google Scholar 

  19. Feigelson, E.D., Babu, G.J.: Big data in astronomy. Significance 9(4), 22–25 (2012)

    Article  Google Scholar 

  20. Fung, B.C.M., Wang, K., Chen, R., Yu, P.S.: Privacy-preserving data publishing: A survey of recent developments. ACM Comput. Surv. 42(4), 14:1–14:53, June 2010. doi:10.1145/1749603.1749605; ISSN:0360-0300; http://doi.acm.org/10.1145/1749603.1749605

    Google Scholar 

  21. Garcia, A.B., Bentes, C., de Melo, R.C., Zadrozny, B., Penna, T.J.P.: Sensor data analysis for equipment monitoring. Knowled. Inform. Syst. 28(2), 333–364 (2011). doi:10.1007/s10115-010-0365-1; ISSN:0219-1377; http://dx.doi.org/10.1007/s10115-010-0365-1

    Google Scholar 

  22. Ghose, A., Bhaumik, C., Das, D., Agrawal, A.K.: Mobile healthcare infrastructure for home and small clinic. In: Proceedings of the 2nd ACM International Workshop on Pervasive Wireless Healthcare, MobileHealth ’12, pp. 15–20. ACM, New York, NY, USA (2012). doi:10.1145/2248341.2248347; ISBN:978-1-4503-1292-9; http://doi.acm.org/10.1145/2248341.2248347

  23. Glas, B., Guajardo, J., Hacioglu, H., Ihle, M., Wehefritz, K., Yavuz, A.: Signal-based automotive communication security and its interplay with safety requirements. In: Proceedings of Embedded Security in Cars Conference, November 2012

    Google Scholar 

  24. Gonzalez, J.E., Xin, R.S., Dave, A., Crankshaw, D., Franklin, M.J., Stoica, I.: Graphx: graph processing in a distributed dataflow framework. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), pp. 599–613. USENIX Association, Broomfield, CO, October 2014. ISBN:978-1-931971-16-4; https://www.usenix.org/conference/osdi14/technical-sessions/presentation/gonzalez

  25. Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.: Internet of things (IoT): a vision, architectural elements, and future directions. Future Gen. Comput. Syst. 29, 1645–1660 (2013)

    Google Scholar 

  26. Hems, A., Soofi, A., Perez, E.: Drilling for new business value: how innovative oil and gas companies are using big data to outmaneuver the competition. A Microsoft White Pater, May 2013

    Google Scholar 

  27. Hesla, L.: Particle physics tames big data. Symmetry 1 (2012)

    Google Scholar 

  28. IBM White Paper: Predictive maintenance for manufacturing. IBM (2011)

    Google Scholar 

  29. Japkowicz, N., Shah, M.: Evaluating Learning Algorithms: A classification perspective. Cambridge University Press (2011)

    Google Scholar 

  30. Jeske, M., Grüner, M., Weiß, F.: Big data in logistics: A DHL perspective on how to move beyond the hype. DHL Customer Solutions and Innovation, December 2013. http://www.delivering-tomorrow.com/wp-content/uploads/2014/02/CSI_Studie_BIG_DATA_FINAL-ONLINE.pdf

  31. Joint DHL Bosch KIT Report: Self-driving vehicles in logistics: A DHL perspective on implications and use cases for the logistics industry. Technical report (2014). http://www.delivering-tomorrow.com/wp-content/uploads/2014/12/dhl_self_driving_vehicles.pdf

  32. Kleiner, A., Talwalkar, A., Sarkar, P., Jordan, M.I.: A scalable bootstrap for massive data. J. Royal Statis. Soc. 76, 795–816 (2013)

    Article  MathSciNet  Google Scholar 

  33. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

  34. Kurtz, J., Hoy, P., McHargue, L., Ward, J.: Improving operational and financial results through predictive maintenance. IBM Smarter Analytics Leadership Summit, Feb 2013

    Google Scholar 

  35. Lawson, S.: IoT groups are like an orchestra tuning up: the music starts in 2016. Computer World, Dec 2014. http://www.computerworld.com/article/2863498/networking-hardware/IoT-groups-are-like-an-orchestra-tuning-up-the-music-starts-in-2016.html

  36. Le, Q.V., Monga, R., Devin, M., Chen, K., Corrado, G.S., Dean, J., Ng, A.Y.: Building high-level features using large scale unsupervised learning. In: International Conference on Machine Learning (2012)

    Google Scholar 

  37. Lee, J., Lapira, E., Bagheri, B., Kao, H.: Recent advances and trends in predictive manufacturing systems in big data environment. Manuf. Lett. 1, 38–41 (2013)

    Google Scholar 

  38. Lee, J., Kao, H., Yang, S.: Service innovation and smart analytics for industry 4.0 and big data environment. Procedia CIRP 16, 3–8 (2014)

    Google Scholar 

  39. Leuth, K.L.: IoT market segments biggest opportunities in industrial manufacturing. IoT-Analytics (2014). http://IoT-analytics.com/IoT-market-segments-analysis/

  40. Lin, J., Keogh, E., Lonardi, S., Chiu, B.: A symbolic representation of time series, with implications for streaming algorithms. In: Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery, pp. 2–11. ACM (2003)

    Google Scholar 

  41. Mackey, L., Talwalkar, A., Jordan, M.I.: Distributed matrix completion and robust factorization. J. Mach. Learn. Res. (2014)

    Google Scholar 

  42. Markkanen, A., Shey, D.: The intersection of analytics and the internet of things. IEEE Internet of Things Newsletter, Nov 2014. http://IoT.ieee.org/newsletter/november-2014/the-intersection-of-analytics-and-the-internet-of-things.html

  43. Marz, N., Warren, J.: Big data: principles and best practices of scalable realtime data systems. Manning Publications Co. (2015)

    Google Scholar 

  44. Matwin, S.: Privacy-preserving data mining techniques: survey and challenges. In: Discrimination and Privacy in the Information Society, pp. 209–221. Springer (2013)

    Google Scholar 

  45. McKinsey Study: Connected car, automotive value chain unbound. Technical report (2014)

    Google Scholar 

  46. Metz, R.: Ces 2015: Wearables everywhere. MIT Technology Review, January 2015. http://www.technologyreview.com/news/533916/ces-2015-wearables-everywhere/

  47. Middleton, P., Kjeldsen, P., Tully, J.: Forecast: The Internet of Things, worldwide, 2013. Gartner, November 2013

    Google Scholar 

  48. Mind Commerce LLC Report: Big data in extraction and natural resource industries: Mining, water, timber, oil and gas 2014–2019. Technical report, July 2014. http://www.researchandmarkets.com/research/3qpj9t/big_data_in

  49. MIT Business Report: Cities get smarter. Technical report (2015)

    Google Scholar 

  50. Nambiar, R., Bhardwaj, R., Sethi, A., Vargheese, R.: A look at challenges and opportunities of big data analytics in healthcare. In: 2013 IEEE International Conference on Big Data, pp. 17–22. IEEE (2013)

    Google Scholar 

  51. Navarro-Arribas, G., Torra, V.: Advanced Research in Data Privacy (2014)

    Google Scholar 

  52. Nicholson, R.: Big data in the oil and gas industry. IDC Energy Insights, September 2012

    Google Scholar 

  53. NIST Report: Workshop report on foundations for innovation in cyber-physical systems. Technical report, Jan 2013. http://www.nist.gov/el/upload/CPS-WorkshopReport-1-30-13-Final.pdf

  54. Orts, E., Spigonardo, J.: Sustainability in the age of big data. Special Report, Initiative for Global Environmental Leadership (IGEL), Knowledge at Wharton, September 2014. http://knowledge.wharton.upenn.edu/article/the-big-data-and-energy-synergy/

  55. Páez, D., Aparicio, F., de Buenaga, M., Ascanio, J.R.: Big data and IoT for chronic patients monitoring. In: Ubiquitous Computing and Ambient Intelligence. Personalisation and User Adapted Services, pp. 416–423. Springer (2014)

    Google Scholar 

  56. Pan, X., Jegelka, S., Gonzalez, J., Bradley, J.K., Jordan, M.: Parallel double greedy submodular maximization. In: Advances in Neural Information Processing Systems 22, (2014)

    Google Scholar 

  57. Poulymenopoulou, M., Malamateniou, F., Vassilacopoulos, G.: Machine learning for knowledge extraction from phr big data. Stud. Health Technol. Inform. 202, 36–39 (2013)

    Google Scholar 

  58. Reddy, A.S.: Reaping the benefits of the internet of things. Cognizant Reports, May 2014

    Google Scholar 

  59. Salakhutdinov, R.: Learning deep generative models. Ph.D. thesis, University of Toronto, Toronto, Canada (2009)

    Google Scholar 

  60. Seshadri, M.: Big data science challenging the oil industry. Energyworld (2013). http://web.idg.no/app/web/online/event/energyworld/2013/emc.pdf

  61. Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’11, pp. 151–161. Association for Computational Linguistics, Stroudsburg, PA, USA, 2011. ISBN:978-1-937284-11-4. http://dl.acm.org/citation.cfm?id=2145432.2145450

  62. Sowe, S.K., Kimata, T., Mianxiong, D., Zettsu, K.: Managing heterogeneous sensor data on a big data platform: IoT services for data-intensive science. In: 2014 IEEE 38th International Computer Software and Applications Conference Workshops (COMPSACW), pp. 295–300, July 2014. doi:10.1109/COMPSACW.2014.52

  63. Tracey, D., Sreenan, C.: A holistic architecture for the internet of things, sensing services and big data. In: 2013 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 546–553, May 2013. doi:10.1109/CCGrid.2013.100

  64. Turner, V., Gantz, J.F., Reinsel, D., Minton, S.: The digital universe of opportunities: rich data and the increasing value of the internet of things. IDC White Paper, April 2014. http://idcdocserv.com/1678

  65. Vandermerwe, S., Rada, J.: Servitization of business: adding value by adding services. Eur. Manage J. 6(6), 314–324 (1989)

    Google Scholar 

  66. Vermesan, O., Friess, P.: Internet of Things: Converging Technologies for Smart Environments and Integrated Ecosystems. River Publishers (2013)

    Google Scholar 

  67. Waller, M.A., Fawcett, S.E.: Data science, predictive analytics, and big data: a revolution that will transform supply chain design and management. J. Bus. Logist. 34(2), 77–84 (2013)

    Article  Google Scholar 

  68. Wang, Y., Bai, H., Stanton, M., Chen, W., Chang, E.Y.: Plda: parallel latent dirichlet allocation for large-scale applications. In: Proceedings of the 5th International Conference on Algorithmic Aspects in Information and Management, AAIM ’09, pp. 301–314. Springer-Verlag, Berlin, Heidelberg (2009). doi:10.1007/978-3-642-02158-9_26; ISBN:978-3-642-02157-2; http://dx.doi.org/10.1007/978-3-642-02158-9_26

    Google Scholar 

  69. Witten, B.: Top 10 IoT security mishaps 2014. In: Industrial Internet Consortium Web blog post. IIC (2014). http://blog.iiconsortium.org/2014/12/top-10-IoT-security-mishaps-2014-.html

  70. Yashiro, T., Kobayashi, S., Koshizuka, N., Sakamura, K.: An internet of things (IoT) architecture for embedded appliances. In: Humanitarian Technology Conference (R10-HTC), 2013 IEEE. Region, vol. 10, pp. 314–319 (2013). doi:10.1109/R10-HTC.2013.6669062

  71. Yavuz, A.A.: Practical immutable signature bouquets (pisb) for authentication and integrity in outsourced databases. In: Data and Applications Security and Privacy XXVI, pp. 179–194. Springer (2013)

    Google Scholar 

  72. Zaki, M., Neely, A.: Optimising asset management within complex service networks: the role of data. Cambridge Service Alliance, working paper:1–11 (2014)

    Google Scholar 

  73. Zanella, A., Bui, N., Castellani, A., Vangelista, L., Zorzi, M.: Internet of things for smart cities. IoT J IEEE, 1(1):22–32 (2014). doi:10.1109/JIoT.2014.2306328; ISSN:2327-4662

    Google Scholar 

  74. Zaslavsky, A, Perera, C., Georgakopoulos, D.: Sensing as a service and big data. arXiv:1301.0159 (2013)

  75. Zhai, K., Boyd-Graber, J., Asadi, N., Alkhouja, M.L.: Mr. lda: A flexible large scale topic modeling package using variational inference in mapreduce. In: Proceedings of the 21st International Conference on World Wide Web, WWW ’12, pp. 879–888, ACM, New York, NY, USA (2012). doi:10.1145/2187836.2187955; ISBN:978-1-4503-1229-5; http://doi.acm.org/10.1145/2187836.2187955

  76. Zhou, Z., Chawla, N., Jin, Y., Williams, G.: Big data opportunities and challenges: discussions from data analytics perspectives [discussion forum]. IEEE Comput. Intell. Magaz. 9(4), 62–74 (2014)

    Article  Google Scholar 

  77. Zicari, R.V., Akerkar, R. (ed.): Big data computing. In: Big Data: Challenges and Opportunities, pp. 103–128. Chapman and Hall/CRC (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research and Technology Center - North America, Robert Bosch LLC, Palo Alto, USA

    Mohak Shah

Authors
  1. Mohak Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohak Shah .

Editor information

Editors and Affiliations

  1. University of Ottawa, Ottawa, Ontario, Canada

    Nathalie Japkowicz

  2. Institute of Computing Sciences, Poznań University of Technology, Poznań, Poland

    Jerzy Stefanowski

Appendix

Appendix

Links to entities referred to in the article (in alphabetical order):

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Shah, M. (2016). Big Data and the Internet of Things. In: Japkowicz, N., Stefanowski, J. (eds) Big Data Analysis: New Algorithms for a New Society. Studies in Big Data, vol 16. Springer, Cham. https://doi.org/10.1007/978-3-319-26989-4_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26989-4_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26987-0

  • Online ISBN: 978-3-319-26989-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation