Collaborative Web Platform for UNIX-Based Big Data Processing

  • Omar Castro
  • Hugo Sereno Ferreira
  • Tiago Boldt Sousa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8683)


UNIX-based operative systems were always empowered by scriptable shell interfaces, with a core set of powerful tools to perform manipulation over files and data streams. However those tools can be difficult to manage at the hands of a non-expert programmer.

This paper proposes the creation of a Collaborative Web Platform to easily create workflows using common UNIX command line tools for processing Big Data through a collaborative web GUI.


Big Data UNIX Shell End-User Programming Cooperative Programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Big data and the creative destruction of today’s business models, strategic it article, a.t. KearneyGoogle Scholar
  2. 2.
    Big Data Meets Big Data Analytics, SAS White Paper (2011)Google Scholar
  3. 3.
    Robbins, K.A., Robbins, S.: Practical UNIX Programming: A Guide to Concurrency, Communication, and Multithreading. Prentice-Hall, Inc., Upper Saddle River (1995)Google Scholar
  4. 4.
    Bourne, S.R.: An introduction to the UNIX shell. Bell Laboratories (1978)Google Scholar
  5. 5.
    Bartenstein, T.W., Liu, Y.D.: Green streams for data-intensive software. In: 2013 35th International Conference on Software Engineering (ICSE), pp. 532–541. IEEE Press (2013)Google Scholar
  6. 6.
    Scheibler, T., Leymann, F., Roller, D.: Executing pipes-and-filters with workflows. In: 2010 Fifth International Conference on Internet and Web Applications and Services (ICIW), pp. 143–148 (May 2010)Google Scholar
  7. 7.
    Johnston, W.M., Hanna, J.R.P., Millar, R.J.: Advances in dataflow programming languages. ACM Comput. Surv. 36, 1–34 (2004)CrossRefGoogle Scholar
  8. 8.
    LXC - Linux Containers, (accessed April 18, 2014)
  9. 9.
    Homepage - Docker: the Linux container engine, (accessed April 19, 2014)

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Omar Castro
    • 1
  • Hugo Sereno Ferreira
    • 1
    • 2
  • Tiago Boldt Sousa
    • 1
    • 2
  1. 1.Department of Informatics Engineering, Faculty of EngineeringUniversity of PortoPortugal
  2. 2.INESC Technology and Science (formerly INESC Porto)Portugal

Personalised recommendations