Skip to main content
SpringerLink
Log in
Book cover

International Conference on Exploring Services Science

IESS 2016: Exploring Services Science pp 497–508Cite as

Queuing-Based Processing Platform for Service Delivery in Big Data Environments

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 247))

Abstract

Service Delivery is one of the most important aspects in every nowadays platforms. Big Data and all analytics processes and services are responsible for new models of service delivery. In this paper we propose an architecture based on message queues for communication between various data sources (e.g. sensors) and a central application, providing stability of delivered services in case of faults: if the central application does not work, messages from the sensors will remain unused in queue and be consumed when the application will be back on-line. Implementation was achieved with RabbitMQ. Also, we have proposed a web application that will generate statistics based on a large volume of data. When we add a new filter (that will generate new statistics), considered as a new task, it must be taken up by a scheduler. The interface is able to configure how many such tasks can run in parallel. Finally, we implemented the proposed architecture to support faults and to be scalable.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Snyder, B., Bosnanac, D., Davies, R.: ActiveMQ in Action. Manning (2011)

    Google Scholar 

  2. Kreps, J., Narkhede, N., Rao, J., et al.: Kafka: a distributed messaging system for log processing. In: Proceedings of the NetDB, pp. 1–7 (2011)

    Google Scholar 

  3. Lampkin, V., Leong, W.T., Olivera, L., Rawat, S., Subrahmanyam, N., Xiang, R., Kallas, G., Krishna, N., Fassmann, S., Keen, M., et al.: Building Smarter Planet Solutions with mqtt and IBM Websphere MQ Telemetry. IBM Redbooks (2012)

    Google Scholar 

  4. Krafzig, D., Banke, K., Slama, D.: Enterprise SOA: Service-Oriented Architecture Best Practices. Prentice Hall Professional, Upper Saddle River (2005)

    Google Scholar 

  5. Hohpe, G., Woolf, B.: Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley Professional, Upper Saddle River (2004)

    Google Scholar 

  6. Brydon, S.P., Singh, I.: Web services message broker architecture. US Patent 7, pp. 702–724, 20 April 2010

    Google Scholar 

  7. Fiosina, J., Fiosins, M.: Resampling based modelling of individual routing preferences in a distributed traffic network. Int. J. Artif. Intell. 12(1), 79–103 (2014)

    Google Scholar 

  8. Demirkan, H., Delen, D.: Leveraging the capabilities of service-oriented decision support systems: putting analytics and big data in cloud. Decis. Support Syst. 55(1), 412–421 (2013)

    Article  Google Scholar 

  9. Videla, A., Williams, J.: Rabbitmq in action: distributed messaging for everyone. Rabbit MQ in action (2012)

    Google Scholar 

  10. Dossot, D.: RabbitMQ Essentials. Packt Publishing Ltd (2014)

    Google Scholar 

  11. Krishnan, S., Gonzalez, J.L.U.: Google compute engine. In: Building Your Next Big Thing with Google Cloud Platform, pp. 53–81. Springer (2015)

    Google Scholar 

  12. Vasile, M.A., Pop, F., Tutueanu, R.I., Cristea, V., Kolodziej, J.: Resource-aware hybrid scheduling algorithm in heterogeneous distributed computing. Future Gener. Comput. Syst. 51(C), 61–71 (2015)

    Article  Google Scholar 

  13. Makpaisit, P., Marurngsith, W.: Griffon-gpu programming apis for scientific and general purpose computing (extended version). Int. J. Artif. Intell. 8(S12), 223–238 (2012)

    Google Scholar 

  14. Costa, Â., Novais, P.: Mobile sensor systems on outpatients. Int. J. Artif. Intell. 8(S12), 252–268 (2012)

    Google Scholar 

  15. Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: distributed stream computing platform. In: Proceedings of the 2010 IEEE International Conference on Data Mining Workshops. ICDMW 2010, pp. 170–177. IEEE, Washington, DC (2010)

    Google Scholar 

  16. Toshniwal, A., Taneja, S., Shukla, A., Ramasamy, K., Patel, J.M., Kulkarni, S., Jackson, J., Gade, K., Fu, M., Donham, J., Bhagat, N., Mittal, S., Ryaboy, D.: Storm@twitter. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. SIGMOD 2014, USA, pp. 147–156. ACM (2014)

    Google Scholar 

  17. Zaharia, M., Das, T., Li, H., Shenker, S., Stoica, I.: Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. In: Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing, p. 10. USENIX Association (2012)

    Google Scholar 

  18. Sun, Q., Yu, W., Kochurov, N., Hao, Q., Hu, F.: A multi-agent-based intelligent sensor and actuator network design for smart house and home automation. J. Sens. Actuator Netw. 2(3), 557–588 (2013)

    Article  Google Scholar 

  19. Oliveira, T.J.M., Costa, Â., Neves, J., Novais, P.: A comprehensive clinical guideline model and a reasoning mechanism for aal systems. Int. J. Artif. Intell. 11(A13), 57–73 (2013)

    Google Scholar 

  20. Benazzouz, Y., Chikhaoui, B., Abdulrazak, B.: An argumentation based approach for dynamic service composition in ambient intelligence environments. Int. J. Artif. Intell. 4(S10), 137–152 (2010)

    Google Scholar 

  21. Gomez, C., Paradells, J.: Wireless home automation networks: a survey of architectures and technologies. IEEE Comm. Mag. 48(6), 92–101 (2010)

    Article  Google Scholar 

  22. Shelby, Z., Bormann, C.: 6LoWPAN: The Wireless Embedded Internet. John Wiley & Sons, New York (2011)

    Google Scholar 

  23. Cao, H., Leung, V., Chow, C., Chan, H.: Enabling technologies for wireless body area networks: a survey and outlook. IEEE Comm. Mag. 47(12), 84–93 (2009)

    Article  Google Scholar 

  24. Hargreaves, T., Hauxwell-Baldwin, R., Coleman, M., Wilson, C., Stankovic, L., Stankovic, V., Murray, D., Liao, J., Kane, T., Firth, S., et al.: Smart homes, control and energy management: how do smart home technologies influence control over energy use and domestic life? European Council for an Energy Efficient Economy (ECEEE) 2015 Summer Study Proceedings, pp. 1022–1032 (2015)

    Google Scholar 

  25. Johansen, N.T.: Z-wave protocol overview (zensys), document no. sds 10243 (2006)

    Google Scholar 

  26. Gill, K., Yang, S.H., Yao, F., Lu, X.: A zigbee-based home automation system. IEEE Trans. Consum. Electron. 55(2), 422–430 (2009)

    Article  Google Scholar 

  27. Alliance, Z.: ZigBee Home Automation Public Application Profile (2007)

    Google Scholar 

  28. Ghaffarian Hoseini, A.H., Dahlan, N.D., Berardi, U., Ghaffarian Hoseini, A., Makaremi, N.: The essence of future smart houses: From embedding ict to adapting to sustainability principles. Renewable and Sustainable Energy Reviews 24, 593–607 (2013)

    Article  Google Scholar 

  29. Bessis, N., Sotiriadis, S., Pop, F., Cristea, V.: Optimizing the energy efficiency of message exchanging for service distribution in interoperable infrastructures. In: 2012 4th International Conference on Intelligent Networking and Collaborative Systems (INCoS), pp. 105–112. IEEE (2012)

    Google Scholar 

  30. Dragoicea, M., Patrascu, M., Serea, G.A.: Real time agent based simulation for smart city emergency protocols. In: 2014 18th International Conference on System Theory, Control and Computing (ICSTCC), pp. 187–192. IEEE (2014)

    Google Scholar 

  31. Patrascu, M., Dragoicea, M., Ion, A.: Emergent intelligence in agents: a scalable architecture for smart cities. In: 2014 18th International Conference on System Theory, Control and Computing (ICSTCC), pp. 181–186. IEEE (2014)

    Google Scholar 

  32. Skön, J.P., Johansson, M., Raatikainen, M., Haverinen-Shaughnessy, U., Pasanen, P., Leiviskä, K., Kolehmainen, M.: Analysing events and anomalies in indoor air quality using self-organizing maps. Int. J. of Artif. Intell. 9(A12), 79–89 (2012)

    Google Scholar 

  33. Ellis, C., Hazas, M., Scott, J.: Matchstick: A room-to-room thermal model for predicting indoor temperature from wireless sensor data. In: Proceedings of the 12th International Conference on Information processing in Sensor Networks, pp. 31–42. ACM (2013)

    Google Scholar 

  34. Yamazaki, T., Kamimura, K., Kurosu, S., Yamakawa, Y.: Air-conditioning PID control system with adjustable reset to offset thermal loads upsets. In: Yurkevich, V.D., (Ed.) Advances in PID Control, InTech.INTECH (2011)

    Google Scholar 

  35. Lachapelle, A.C., Love, J.A.: Simulink\({\textregistered }\) model of single co2 sensor location impact on CO2 levels in recirculating multiple-zone systems. In: Proceedings of eSim 2012: The Canadian Conference on Building Simulation, ESIM.CA, pp. 189–201 (2012)

    Google Scholar 

  36. Kuch, J.: Rabbitmq hits one million messages per second on google compute engine, point of view (2014). https://blog.pivotal.io/pivotal/products/rabbitmq-hits-one-million-messages-per-second-on-google-compute-engine

Download references

Acknowledgment

The research presented in this paper is supported by projects: DataWay: Real-time Data Processing Platform for Smart Cities: Making sense of Big Data - PN-II-RU-TE-2014-4-2731; CyberWater grant of the Romanian National Authority for Scientific Research, UEFISCDI, project 47/2012; clueFarm: Information system based on cloud services accessible through mobile devices, to increase product quality and business development farms - PN-II-PT-PCCA-2013-4-0870.

We would like to thank the reviewers for their time and expertise, constructive comments and valuable insight.

Author information

Authors and Affiliations

  1. Computer Science Department, Faculty of Automatic Control and Computets, University Politehnica of Bucharest, Splaiul Independentei 313, Sector 6, 060042, Bucharest, Romania

    Florin Stancu, Dan Popa, Loredana-Marsilia Groza & Florin Pop

Authors
  1. Florin Stancu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dan Popa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Loredana-Marsilia Groza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Florin Pop
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florin Pop .

Editor information

Editors and Affiliations

  1. Faculty of Automatic Control & Computers, University Politehnica of Bucharest, Bucharest, Romania

    Theodor Borangiu

  2. Faculty of Automatic Control & Computers, University Politehnica of Bucharest, Bucharest, Romania

    Monica Dragoicea

  3. Faculty of Engineering - FEUP, University of Porto, Porto, Portugal

    Henriqueta Nóvoa

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Stancu, F., Popa, D., Groza, LM., Pop, F. (2016). Queuing-Based Processing Platform for Service Delivery in Big Data Environments. In: Borangiu, T., Dragoicea, M., Nóvoa, H. (eds) Exploring Services Science. IESS 2016. Lecture Notes in Business Information Processing, vol 247. Springer, Cham. https://doi.org/10.1007/978-3-319-32689-4_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32689-4_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32688-7

  • Online ISBN: 978-3-319-32689-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation