Advertisement

Data Warehousing and Exploratory Analysis for Market Monitoring

  • Melanie Geiger
  • Kurt StockingerEmail author
Chapter

Abstract

With the growing trend of digitalization, many companies plan to use machine learning to improve their business processes or to provide new data-driven services. These companies often collect data from different locations with sometimes conflicting context. However, before machine learning can be applied, heterogeneous datasets often need to be integrated, harmonized, and cleaned. In other words, a data warehouse is often the foundation for subsequent analytics tasks.

In this chapter, we first provide an overview on best practices of building a data warehouse. In particular, we describe the advantages and disadvantage of the major types of data warehouse architectures based on Inmon and Kimball. Afterward, we describe a use case on building an e-commerce application where the users of this platform are provided with information about healthy products as well as products with sustainable production. Unlike traditional e-commerce applications, where users need to log into the system and thus leave personalized traces when they search for specific products or even buy them afterward, our application allows full anonymity of the users in case they do not want to log into the system. However, analyzing anonymous user interactions is a much harder problem than analyzing named users. The idea is to apply modern data warehousing, big data technologies, as well as machine learning algorithms to discover patterns in the user behavior and to make recommendations for designing new products.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgment

The work was funded by the Swiss Commission for Technology and Innovation (CTI) under grant 16053.2.

References

  1. Apache Mahout. Retrieved August 24, 2018., from http://mahout.apache.org/
  2. Bernstein, P. A. (1976). Synthesizing third normal form relations from functional dependencies. ACM Transactions on Database Systems, 1(4), 277–298.CrossRefGoogle Scholar
  3. Casper, J., & Olukotun, K. (2014). Hardware acceleration of database operations. In Proceedings of the 2014 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (pp. 151–160). ACM.Google Scholar
  4. Clifton, B. (2012). Advanced web metrics with Google analytics. Hoboken, NJ: Wiley.Google Scholar
  5. Ehrenmann, M., Pieringer, R., & Stockinger, K. (2012). Is there a cure-all for business analytics case studies of exemplary businesses in banking, telecommunications, and retail. Business Intelligence Journal, 17(3). TDWI.Google Scholar
  6. Ester, M., & Kriegel, H.P., & Sander, J. & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In International Conference on Knowledge Discovery and Data Mining, 1996.Google Scholar
  7. Feuerstein, S., & Pribyl, B. (2005). Oracle Pl/SQL programming. O’Reilly Media, Newton, MA.Google Scholar
  8. Hultgren, H. (2012). Modeling the agile data warehouse with data vault. Denver, CO: New Hamilton.Google Scholar
  9. Inmon, B. (1992). Building the data warehouse. Hoboken, NJ: Wiley.Google Scholar
  10. Ioannidis, Y. E. (1996). Query optimization. ACM Computing Surveys (CSUR), 28(1), 121–123.CrossRefGoogle Scholar
  11. JasperSoft. Retrieved July 21, 2017, from https://www.jaspersoft.com/
  12. Kimball, R. (2002). The data warehouse toolkit. Hoboken, NJ: Wiley.Google Scholar
  13. Larson, P. Å., & Levandoski, J. (2016). Modern main-memory database systems. Proceedings of the VLDB Endowment, 9(13), 1609–1610.CrossRefGoogle Scholar
  14. Lawton, G. (2005). LAMP lights enterprise development efforts. Computer, 38(9), 18–20.CrossRefGoogle Scholar
  15. MacQueen, J. B. (1967). Some methods for classification and analysis of multivariate observations. In Berkeley Symposium on Mathematical Statistics and Probability, University of California Press.Google Scholar
  16. McCallum, A., Nigam, K., & Ungar, L. H. (2000). Efficient clustering of high dimensional data sets with application to reference matching. In SIGKDD International Conference on Knowledge Discovery and Data Mining.Google Scholar
  17. Pentaho. Retrieved July 21, 2017, from http://www.pentaho.com/
  18. Postgres. Retrieved July 21, 2017, from https://www.postgresql.org/
  19. Talend. Retrieved July 21, 2017, from https://www.talend.com/
  20. Wang, W., Zhang, M., Chen, G., Jagadish, H. V., Ooi, B. C., & Tan, K. L. (2016). Database meets deep learning: Challenges and opportunities. ACM SIGMOD Record, 45(2), 17–22.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.ZHAW Zurich University of Applied SciencesWinterthurSwitzerland

Personalised recommendations