Advertisement

Journal of Intelligent Information Systems

, Volume 19, Issue 2, pp 169–190 | Cite as

Capturing Delays and Valid Times in Data Warehouses—Towards Timely Consistent Analyses

  • Robert M. Bruckner
  • A Min Tjoa
Article

Abstract

Real-world changes are generally discovered delayed by computer systems. The typical update patterns for traditional data warehouses on an overnight or even weekly basis increase this propagation delay until the information is available to knowledge workers. Typically, traditional data warehouses focus on summarized data (at some level) rather than detailed data.

For active data warehouse environments, detailed data about entities is required for checking the data conditions and triggering actions to automize routine decision tasks. Hence, keeping data current (by minimizing the latency from when data is captured until it is available to knowledge workers) and consistent in that context is a difficult task.

In this paper we present an approach for modeling conceptual time consistency problems and introduce a data model that deals with timely delays. It supports knowledge workers in finding out, why (or why not) an active system responded to acertain state of the data. Therefore, the model enables analytical processing of detailed data (enhanced by valid time) based on a knowledge state at a specific time. All states that were not yet known by the system at that point in time are consistently ignored. This enables timely consistent analyses by considering that the validity of detailed data and aggregates can be restricted to time intervals only, due to frequent updates and late-arriving information.

data modeling temporal data models propagation delays time semantics data warehouse 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. ACT-NET Consortium (1996). The Active Database Management System Manifesto: A Rule-Base of ADBMS Features. ACM SIGMOD Record, 25(3), 414–471.Google Scholar
  2. Ballou, D.P. and Tayi, G.K. (1999). Enhancing Data Quality in Data Warehouse Environments. Communications of the ACM, 42(1), 73–78.Google Scholar
  3. Bliujute, R., Saltenis, S., Slivinskas, G., and Jensen, C.S. (1998). Systematic Change Management in Dimensional Data Warehousing. In Proc. of the 3rd Intl. Baltic Workshop on Databases and Information Systems, Riga, Latvia, (pp. 27–41).Google Scholar
  4. Boehlen, M.H., Busatto, R., and Jensen, C.S. (1998). Point-Versus Interval-Based Temporal Data Models. In Proc. of 14th Intl. Conf. on Data Engineering (ICDE), Orlando, FL, (pp. 192–201). IEEE CS Press.Google Scholar
  5. Brobst, S. and Ballinger, C. (2000). Active Data Warehousing. Whitepaper EB-1327, NCR Corporation.Google Scholar
  6. Bruckner, R.M., List, B., Schiefer, J., and Tjoa, A.M. (2001): Modeling Temporal Consistency in Data Warehouses. In Proc. of the 12th Intl. Workshop on Database and Expert Systems Applications (DEXA), Munich, Germany (pp. 467–471). IEEE CS Press.Google Scholar
  7. Bruckner, R.M. and Tjoa, A.M. (2001). Managing Time Consistency for Active Data Warehouse Environments. In Proc. of the 3rd Intl. Conf. on Data Warehousing and Knowledge Discovery (DaWaK), Munich, Germany, Springer LNCS, Vol. 2114 (pp. 254–263).Google Scholar
  8. Chaudhuri, S. and Dayal, U. (1997). An Overview of DataWarehousing and OLAP Technology. SIGMOD Record, 26(1), 65–74.Google Scholar
  9. Dayal, U. (1988). Active Database Management Systems. In Proc. of 3rd Intl. Conf. on Data and Knowledge Bases, Jerusalem (pp. 150–169).Google Scholar
  10. Dittrich, K.R. and Gatziu, St. (2000). Aktive Datenbanksysteme, Heidelberg, Germany: Dpunkt. Verlag (in German).Google Scholar
  11. Dyreson, C.E., Evans, W.S., Lin, H., and Snodgrass, R.T. (2000). Efficiently Supporting Time Granularities. IEEE Trans. on Knowledge and Data Engineering, 12(4), 568–587.Google Scholar
  12. Eder, J. and Koncilia, C. (2001). Changes of Dimension Data in Temporal Data Warehouses. In Proc. of the 3rd Intl. Conf. on Data Warehousing and Knowledge Discovery (DaWaK), Munich, Germany, Springer LNCS, Vol. 2114 (pp. 284–293).Google Scholar
  13. English, L.P. (1999). Improving Data Warehouse and Business Information Quality. New York: J. Wiley.Google Scholar
  14. Etzion, O., Jajodia, S., and Sripada, S. (Eds.). (1998). Temporal Databases: Research and Practice, Heidelberg, Germany, Springer LNCS, Vol. 1399.Google Scholar
  15. Gregersen, H. and Jensen, C.S. (1999). Temporal Entity-Relationship Models—A Survey. IEEE Transactions on Knowledge and Data Engineering, 11(3), 464–497.Google Scholar
  16. Inmon, W.H. (1996). Building the Data Warehouse. New York: J. Wiley.Google Scholar
  17. Jarke, M., Jeusfeld, M.A., Quix, C., and Vassiliadis, P. (1998). Architecture and Quality in Data Warehouses. In Proc. of the 10th Intl. Conf. on Advanced Information Systems Engineering (CAiSE'98), Pisa, Italy, Springer LNCS, Vol. 1413 (pp. 93–114).Google Scholar
  18. Jensen, C.S. and Dyreson, C.E. (1998). The Consensus Glossary of Temporal Database Concepts. In O. Etzion, S. Jajodia, and S. Sripada (Eds.), Temporal Databases: Research and Practice, Springer LNCS, Vol. 1399 (pp. 367–405).Google Scholar
  19. Jensen, C.S. and Snodgrass, R.T. (1999). Temporal Data Management. IEEE Transactions on Knowledge and Data Engineering, 11(1), 36–44.Google Scholar
  20. Kimball, R. (1996). The Data Warehouse Toolkit. New York: J. Wiley.Google Scholar
  21. Kimball, R. (2000). Backward in Time. Intelligent Enterprise Magazine, 3(15).Google Scholar
  22. Melton, J. (Ed.). (1992). Database Language–SQL. ANSI X 3.135.Google Scholar
  23. Mendelzon, A.O. and Vaisman, A.A. (2000). Temporal Queries in OLAP. In Proc. of the 26th Intl. Conf. on Very Large Databases (VLDB), Cairo, Egypt (pp. 242–253). Morgan Kaufmann Publishing.Google Scholar
  24. Obermair, W. and Schrefl, M. (2000). Temporally Faithful Execution of Business Transactions. In Proc. of 12th Intl. Conf. on Advanced Information Systems Engineering (CAiSE), Stockholm, Sweden, Springer LNCS, Vol. 1789 (pp. 462–481).Google Scholar
  25. Pedersen, T.B. and Jensen, C.S. (1999). Multidimensional Data Modeling for Complex Data. In Proc. of the 15th Intl. Conf. on Data Engineering (ICDE), Sydney, Australia (pp. 336–345). IEEE CS Press.Google Scholar
  26. Pedersen, T.B., Jensen, C.S., and Dyreson, C.E. (2001). A Foundation for Capturing and Querying Complex Multidimensional Data. Information Systems, 26(5), 383–423.Google Scholar
  27. Roddick, J.F. and Schrefl, M. (2000). Towards an Accommodation of Delay in Temporal Active Databases. In Proc. of the 11th Australasien Database Conference (ADC), Canberra, Australia (pp. 115–119). IEEE CS Press.Google Scholar
  28. Rundensteiner, E.A., Koeller, A., and Zhang, X. (2000). Maintaining DataWarehouses over Changing Information Sources. Communications of the ACM, 43(6), 57–62.Google Scholar
  29. Salzberg, B. and Tsotras, V.J. (1999). Comparison of Access Methods for Time-Evolving Data. ACM Computing Surveys, 31(2), 158–221.Google Scholar
  30. Schrefl, M. and Thalhammer, T. (2000). On Making Data Warehouses Active. In Proc. of the 2nd Intl. Conf. on Data Warehousing and Knowledge Discovery (DaWaK), London, UK. Springer LNCS, Vol. 1874 (pp. 34–46).Google Scholar
  31. Shasha, D. and Zhu, Y. (2001). SpyTime—A Performance Benchmark for Bitemporal Databases. Available at http://cs.nyu.edu/cs/faculty/shasha/spytime/spytime.html.Google Scholar
  32. Thalhammer, T., Schrefl, M., and Mohania, M. (2001). Active Data Warehouses: Complementing OLAP with Analysis Rules. Data &; Knowledge Engineering, 39(3), 241–269.Google Scholar
  33. Torp, K., Jensen, C.S., and Snodgrass, R.T. (2000). Effective Timestamping in Databases. The VLDB Journal, 8(3/4), 267–288.Google Scholar
  34. Vaisman, A.A. and Mendelzon, A.O. (2001). A Temporal Query Language for OLAP: Implementation and a Case Study. In Proc. of the 8th BiennialWorkshop on Data Bases and Programming Languages (DBPL), Rome, Italy.Google Scholar
  35. Wang, R.Y. (1998). Total Data Quality Management. Communications of the ACM, 41(2), 58–65.Google Scholar
  36. Widom, J. (1995). Research Problems in Data Warehousing. In Proc. of the 5th Intl. Conference on Information and Knowledge Management (CIKM), Baltimore, Maryland (pp. 25–30). ACM-Press.Google Scholar
  37. Widom, J. and Ceri, S. (1996). Active Database Systems: Triggers and Rules For Advanced Database Processing. San Francisco: Morgan Kaufmann Publishers.Google Scholar
  38. Westerman, P. (2000). Data Warehousing: Using the Wal-Mart Model. San Francisco: Morgan Kaufmann Publishers.Google Scholar
  39. Yang, J. (2001). Temporal DataWarehousing. Ph.D. Thesis, Department of Computer Science, Stanford University.Google Scholar
  40. Yang, J. and Widom, J. (1998). Maintaining Temporal Views over Nontemporal Information Sources for Data Warehousing. In Proc. of the 6th Intl. Conf. On Extending Database Technology (EDBT), Valencia, Spain, Springer LNCS Vol. 1377 (pp. 389–403).Google Scholar
  41. Yang, J. and Widom, J. (2000). Temporal View Self-Maintenance. In Proc. of the 7th Intl. Conf. On Extending Database Technology (EDBT), Konstanz, Germany, Springer LNCS, Vol. 1777 (pp. 395–412).Google Scholar
  42. Yang, J. and Widom, J. (2001). Incremental Computation and Maintenance of Temporal Aggregates. In Proc. of the 17th Intl. Conf. on Data Engineering (ICDE), Heidelberg, Germany (pp. 51–60). IEEE CS Press.Google Scholar
  43. Zaniolo, C., Ceri, S., Faloutsos, C., Snodgrass, R.T., Subrahmanian, V.S., and Zicari, R. (1997). Advanced Database Systems. San Francisco: Morgan Kaufmann Publishers.Google Scholar
  44. Zhang, D., Markowetz, A., Tsotras, V.J., Gunopulos, D., and Seeger, B. (2001). Efficient Computation of Temporal Aggregates with Range Predicates. In Proc. of the 2001 ACM Symposium on Principles of Database Systems (PODS), Santa Barbara, CA.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Robert M. Bruckner
    • 1
  • A Min Tjoa
    • 1
  1. 1.Institute of Software Technology & Interactive Systems, Department of Information & Software EngineeringVienna University of TechnologyWienAustria

Personalised recommendations