Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Data Warehousing in Cloud Environments

  • Christian Thomsen
  • Torben Bach Pedersen
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_80623

Synonyms

Cloud Data Warehousing; Cloud Warehousing; Data Warehousing as a Service

Definition

Data warehousing was born in business information systems environments dominated by relational databases running on traditional servers. Later, the types of source data and source systems widened, and the deployment environments increasingly included high-end MPP systems. Today, data warehousing has joined the cloud computing wave, running DW systems on both private, public, and hybrid clouds, based mainly on clusters of commodity machines. Cloud-based data warehouses employ components for cloud-based data storage, querying, and processing, often using file-based storage of complex, non-relational, types of data. A widely used platform is Hadoop, the open-source version of Google’s MapReduce platform for scalable dataflow processing on commodity clusters, which was among the earliest systems for cloud data warehousing. While Hadoop is scalable, fault tolerant, and versatile, it is not...

This is a preview of subscription content, log in to check access.

References

  1. 1.
    Abadi DJ. Data Management in the cloud: limitations and opportunities. IEEE Data Eng Bull. 2009;32(1):3–12.MathSciNetGoogle Scholar
  2. 2.
    Abouzeid A, Bajda-Pawlikowski K, Abadi D, Silberschatz A, Rasin A. HadoopDB: an architectural hybrid of MapReduce and DBMS technologies for analytical workloads. Proc VLDB Endow. 2009;2(1):922–933. https://doi.org/10.14778/1687627.1687731.CrossRefGoogle Scholar
  3. 3.
    Agarwal S, Mozafari B, Panda A, Milner H, Madden S, Stoica I. BlinkDB: queries with bounded errors and bounded response times on very large data. In: Proceedings of the 8th ACM SIGOPS/EuroSys European Conference on Computer Systems; 2013. https://doi.org/10.1145/2465351.2465355.
  4. 4.
    Armbrust M, Xin RS, Lian C, et al. Spark SQL: relational data processing in spark. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2015. https://doi.org/10.1145/2723372.2742797.
  5. 5.
    Chan L. Presto: interacting with petabytes of data at Facebook. 2016. https://www.facebook. com/notes/facebook-engineering/presto-interacting-with-petaby tes-of-data-at-facebook/10151786197628920. Accessed 28 June 2016.
  6. 6.
    Dean J, Ghemawat S. MapReduce: a flexible data processing tool. Commun ACM. 2010;53(1):72–77. https://doi.org/10.1145/1629175.1629198.CrossRefGoogle Scholar
  7. 7.
    Gupta A, Agarwal D, Tan D, et al. Amazon redshift and the case for simpler data warehouses. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2015. https://doi.org/10.1145/2723372.2742795.
  8. 8.
    Liu X, Thomsen C, Pedersen TB. ETLMR: a highly scalable dimensional ETL framework based on MapReduce. In: Proceedings of the 13th International Conference on Data Warehousing and Knowledge Discovery; 2011. https://doi.org/10.1007/978-3-642-23544-3_8.CrossRefGoogle Scholar
  9. 9.
    Liu X, Thomsen C, Pedersen TB. CloudETL: scalable dimensional ETL for hive. In: Proceedings of the 18th International Database Engineering & Applications Symposium; 2014. https://doi.org/10.1145/2628194.2628249.
  10. 10.
    Olston C, Reed B, Srivastava U, Kumar R, Tomkins A. Pig Latin: a not-so-foreign language for data processing. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2008. https://doi.org/10.1145/1376616.1376726.
  11. 11.
    Özcan F, Hoa D, Beyer KS, Balmin A, Liu CJ, Li Y. Emerging trends in the enterprise analytics: connecting Hadoop and DB2 warehouse. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2011. https://doi.org/10.1145/1989323.1989446.
  12. 12.
    Pavlo A, Paulson E, Rasin A, Abadi DJ, DeWitt DJ, Madden S, Stonebraker M. A comparison of approaches to large-scale data processing. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2009. https://doi.org/10.1145/1559845.1559865.
  13. 13.
    Pike R, Dorward S, Griesemer R, Quinlan S. Interpreting the data: parallel analysis with Sawzall. Sci Program. 2005;13(4):277–298.Google Scholar
  14. 14.
    Stonebreaker M, Abadi D, DeWitt DJ, Madden S, Paulson E, Pavlo A, Rasin A. MapReduce and parallel DBMSs: friends of foes? Commun ACM. 2010;53(1):64–71. https://doi.org/10.1145/1629175.1629197.CrossRefGoogle Scholar
  15. 15.
    Thusso A, Sarma JS, Jain N, Shao Z, Chakka P, Anthony S, et al. Hive – a warehousing solution over a Map-Reduce framework. In: Proceedings of the 35th International Conference on Very Large Data Bases; 2009. https://doi.org/10.14778/1687553.1687609.CrossRefGoogle Scholar
  16. 16.
    Xin R, Rosen J, Zaharia M, Franklin MJ, Shenker S, Stoica I. Shark: SQL and rich analytics at scale. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2013. https://doi.org/10.1145/2463676.2465288.
  17. 17.
    Zaharia M, Chowdhury M, Das T, et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Symposium on Networked Systems Design & Implementation; 2012.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceAalborg UniversityAalborgDenmark