Advertisement

Metadata Management in a Multiversion Data Warehouse

  • Robert Wrembel
  • Bartosz Bębel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4380)

Abstract

A data warehouse (DW) is a database that integrates data from external data sources (EDSs) for the purpose of advanced analysis. EDSs are production systems that often change not only their contents but also their structures. The evolution of EDSs has to be reflected in a DW that integrates the sources. Traditional DW systems offer a limited support for the evolution of their structures. Our solution to this problem is based on a multiversion data warehouse (MVDW). Such a DW is composed of the sequence of persistent versions, each of which describes a schema and data within a given time period. The management of the MVDW requires a metadata model that is much more complex than in traditional data warehouses. In our approach and prototype MVDW system, the metadata model contains data structures that support: (1) monitoring EDSs with respect to content and structural changes, (2) automatic generation of processes monitoring EDSs, (3) applying discovered EDS changes to a selected DW version, (4) describing the structure of every DW version, (5) querying multiple DW versions at the same time and presenting the results coming from multiple versions.

Keywords

Data Warehouse Level Version Integrity Constraint Data Cube Fact Table 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abdessalem, T., Jomier, G.: VQL: A query Language for Multiversion Databases. In: Cluet, S., Hull, R. (eds.) DBPL 1997. LNCS, vol. 1369, pp. 103–122. Springer, Berlin Heidelberg (1998)Google Scholar
  2. 2.
    Agrawal, R., Buroff, S., Gehani, N., Shasha, D.: Object Versioning in Ode. In: Proc. of Int. Conference on Data Engineering (ICDE), pp. 446-455 (1991)Google Scholar
  3. 3.
    Ahmed-Nacer, M., Estublier, J.: Schema Evolution in Software Engineering. Databases - A. new Approach in ADELE environment. Computers and Artificial Intelligence 19, 183–203 (2000)zbMATHGoogle Scholar
  4. 4.
    Balmin, A., Papadimitriou, T., Papakonstanitnou, Y.: Hypothetical Queries in an OLAP Environment. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 220–231 (2000)Google Scholar
  5. 5.
    Balmin, A., Papadimitriou, T., Papakonstanitnou, Y.: Optimization of Hypothetical Queries in an OLAP Environment. In: Proc. of Int. Conference on Data Engineering (ICDE), p. 311 (2000)Google Scholar
  6. 6.
    Barker, R.: Case*Method: Entity Relationship Modelling, ISBN 0201416964, Addison-Wesley (1990)Google Scholar
  7. 7.
    Bellahsene, Z.: View Adaptation in Data Warehousing Systems. In: Quirchmayr, G., Bench-Capon, T.J.M., Schweighofer, E. (eds.) DEXA 1998. LNCS, vol. 1460, pp. 300–309. Springer, Berlin Heidelberg New York (1998)CrossRefGoogle Scholar
  8. 8.
    Bębel, B.: Transactional Refreshing of Data Warehouses. PhD thesis, Poznań University of Technology, Institute of Computing Science (2005)Google Scholar
  9. 9.
    Bębel, B., Eder, J., Konicilia, C., Morzy, T., Wrembel, R.: Creation and Management of Versions in Multiversion Data Warehouse. In: Proc. of ACM Symposium on Applied Computing (SAC), pp. 717-723 (2004)Google Scholar
  10. 10.
    Bębel, B., Królikowski, Z., Wrembel, R.: Managing Multiple Real and Simulation Business Scenarios by Means of a Multiversion Data Warehouse. In: Proc. of Int. Conference on Business Information Systems (BIS), pp. 102-113, Lecture Notes in Informatics (2006)Google Scholar
  11. 11.
    Bębel, B., Wrembel, R., Czejdo, B.: Storage Structures for Sharing Data in Multiversion Data Warehouse. In: Proc. of Baltic Conference on Databases and Information Systems, pp. 218-231 (2004)Google Scholar
  12. 12.
    Blaschka, M., Sapia, C., Hofling, G.: On Schema Evolution in Multidimensional Databases. In: Mohania, M.K., Tjoa, A.M. (eds.) DaWaK 1999. LNCS, vol. 1676, pp. 153–164. Springer, Berlin Heidelberg New York (1999)Google Scholar
  13. 13.
    Body, M., Miquel, M., Bédard, Y., Tchounikine, A.: A Multidimensional and Multiversion Structure for OLAP Applications. In: Proc. of ACM Int. Workshop on Data Warehousing and OLAP (DOLAP), pp. 1-6 (2002)Google Scholar
  14. 14.
    Body, M., Miquel, M., Bédard, Y., Tchounikine, A.: Handling Evolutions in Multidimensional Structures. In: Proc. of Int. Conference on Data Engineering (ICDE), p. 581 (2003)Google Scholar
  15. 15.
    Bouguettaya, A., Benatallah, B., Elmargamid, A.: Interconnecting Heterogeneous Information Systems ISBN 0792382161. Kluwer Academic Publishers, Norwell, MA, USA (1998)Google Scholar
  16. 16.
    Buczkowski, P.: Błaszyk M., Chmiel J., Tucholski M., Wrembel R.: Design, Implementation, Evaluation of a Multiversion Join Index. Research report RA-009/05, Poznań University of TechnologyGoogle Scholar
  17. 17.
    Cellary, W., Jomier, G.: Consistency of Versions in Object-Oriented Databases. In:Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 432–441 (1990)Google Scholar
  18. 18.
    Chamoni, P., Stock, S.: Temporal Structures in Data Warehousing. In: Mohania, M.K., Tjoa, A.M. (eds.) DaWaK 1999. LNCS, vol. 1676, pp. 353–358. Springer, Berlin Heidelberg New York (1999)Google Scholar
  19. 19.
    Chaudhuri, S., Dayal, U.: An overview of data warehousing and OLAP technology. SIGMOD Record 26(1), 65–74 (1997)CrossRefGoogle Scholar
  20. 20.
    Chen, J., Chen, S., Rundensteiner, E.: A Transactional Model for Data Warehouse Maintenance. In: Spaccapietra, S., March, S.T., Kambayashi, Y. (eds.) ER 2002. LNCS, vol. 2503, pp. 247–262. Springer, Berlin Heidelberg New York (2002)Google Scholar
  21. 21.
    Chen, J., Chen, S., Zhang, X., Rundensteiner, E.: Detection and Correction of Conflicting Source Updates for View Maintenance, In: Proc. of Int. Conference on Data Engineering (ICDE), pp. 436-448 (2004)Google Scholar
  22. 22.
    Chen, J., Rundensteiner, E.: TxnWrap: A Transactional Approach to Data Warehouse Maintenance, Technical Report WPI-CS-TR-00-26, Worcester Polytechnic Institute, 2000, retrieved June 11 (2006) from http://citeseer.ist.psu.edu/384586.htmlGoogle Scholar
  23. 23.
    Chen, S., Zhang, X., Rundensteiner, E.: A Compensation-based Approach for Materialized View Maintenance in Distributed Environments. IEEE Transactions on Knowledge and Data Engineering 18(8), 1068–1081 (2006)CrossRefGoogle Scholar
  24. 24.
    Competing Data Warehousing Standards to Merge in the OMG. Retrieved August 10 (2005) from http://xml.coverpages.org/OMG-MDC-20000925.htmlGoogle Scholar
  25. 25.
    Czejdo, B., Messa, K., Morzy, T., Putonti, C.: Design of Data Warehouses with Dynamically Changing Data Sources. In: Proc. of Southern Conference on Computing, USA (2000)Google Scholar
  26. 26.
    Elmagarmid, A., Rusinkiewicz, M., Sheth, A.: Management of Heterogeneous and Autonomous Database Systems. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  27. 27.
    Eder, J., Koncilia, C.: Changes of Dimension Data in Temporal Data Warehouses. In: Kambayashi, Y., Winiwarter, W., Arikawa, M. (eds.) DaWaK 2001. LNCS, vol. 2114, pp. 284–293. Springer, Berlin Heidelberg New York (2001)Google Scholar
  28. 28.
    Eder, J., Koncilia, C., Mitsche, D.: Automatic Detection of Structural Changes in Data Warehouses. In: Kambayashi, Y., Mohania, M.K., Wöß, W. (eds.) DaWaK 2003. LNCS, vol. 2737, pp. 119–128. Springer, Berlin Heidelberg New York (2003)Google Scholar
  29. 29.
    Eder, J., Koncilia, C., Mitsche, D.: Analysing Slices of Data Warehouses to Detect Structural Modifications. In: Persson, A., Stirna, J. (eds.) CAiSE 2004. LNCS, vol. 3084, pp. 492–505. Springer, Berlin Heidelberg New York (2004)Google Scholar
  30. 30.
    Eder, J., Koncilia, C., Morzy, T.: The COMET Metamodel for Temporal Data Warehouses. In: Pidduck, A.B., Mylopoulos, J., Woo, C.C., Ozsu, M.T. (eds.) CAiSE 2002. LNCS, vol. 2348, pp. 83–99. Springer, Berlin Heidelberg New York (2002)Google Scholar
  31. 31.
    Etzion, O., Jajodia, S., Sripada, S. (eds.): Temporal Databases: Research and Practice. LNCS, vol. 1399. Springer, Berlin Heidelberg New York (1998)Google Scholar
  32. 32.
    S., G., G., J.: A framework for programming multiversion databases. Data Knowledge Engineering 36(1), 29–53 (2001)CrossRefGoogle Scholar
  33. 33.
    Golfarelli, M., Lechtenbörger, J., Rizzi, S., Vossen, G.: Schema Versioning in Data Warehouses. In: Wang, S., Tanaka, K., Zhou, S., Ling, T.-W., Guan, J., Yang, D.-q., Grandi, F., Mangina, E.E., Song, I.-Y., Mayr, H.C. (eds.) ER Workshops 2004. LNCS, vol. 3289, Springer, Berlin Heidelberg New York (2004)Google Scholar
  34. 34.
    Goralwalla, I.A., Tansel, A.U., Ozsu, M.T.: Experimenting with Temporal Relational Databases. In: Proc. of ACM Conference on Information and Knowledge Management (CIKM), pp. 296–303 (1995),Google Scholar
  35. 35.
    Gupta, A., Mumick, I.S. (eds.): Materialized Views: Techniques, Implementations, and Applications ISBN 0-262-57122-6. The MIT Press, Cambridge, MA (1999)Google Scholar
  36. 36.
    Gyssens, M., Lakshmanan, L.V.S.: A Foundation for Multi-Dimensional Databases. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 106–115 (1997)Google Scholar
  37. 37.
    Hurtado, C.A., Mendelzon, A.O., Vaisman, A.A.: Maintaining Data Cubes under Dimension Updates. In: Proc. of Int. Conference on Data Engineering (ICDE), pp. 346-355 (1999)Google Scholar
  38. 38.
    Hurtado, C.A., Mendelzon, A.O., Vaisman, A.A.: Updating OLAP Dimensions. In: Proc. of ACM Int. Workshop on Data Warehousing and OLAP (DOLAP), pp. 60–66 (1999)Google Scholar
  39. 39.
    Microsoft ImmortalDB. Retrieved November 25, 2005 from http://research.microsoft.com/db/ImmortalDB/ (2005)Google Scholar
  40. 40.
    Jarke, M., Lenzerini, M., Vassiliou, Y., Vassiliadis, P.: Fundamentals of Data Warehouses ISBN 3-540-42089-4. Springer-Verlag, Berlin Heidelberg New York (2003)Google Scholar
  41. 41.
    Jensen, C.S., Lomet, D.B.: Transaction Timestamping in (Temporal) Databases. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 441–450 (2001)Google Scholar
  42. 42.
    Kaas Ch.K., Pedersen T.B., Rasmussen B.D.: Schema Evolution for Stars and Snowflakes. In: Proc. of Int. Conference on Enterprise Information Systems (ICEIS), pp. 425-433 (2004)Google Scholar
  43. 43.
    Kang, H.G., Chung, C.W.: Exploiting Versions for On-line Data Warehouse Maintenance in MOLAP Servers. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 742–753 (2002)Google Scholar
  44. 44.
    Kim, W., Chou, H.: Versions of Schema for Object-Oriented Databases. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 148–159 (1988)Google Scholar
  45. 45.
    Kimball, R., Caserta, J.: The Data Warehouse ETL Tookit ISBN 0764567578. John Wiley & Sons, New York (2004)Google Scholar
  46. 46.
    Kimball, R., Ross, M.: The Data Warehouse Toolkit. ISBN 0-471-20024-7. John Wiley & Sons, New York (2002)Google Scholar
  47. 47.
    Kulkarni, S., Mohania, M.: Concurrent Maintenance of Views Using Multiple Versions. In: Proc. of the Int. Database Engineering and Application Symposium (IDEAS), pp. 254-259 (1999)Google Scholar
  48. 48.
    Lee, A., Nica, A., Rundensteiner, E.: The EVE Framework: View Synchronization in Evolving Environments. Technical Report WPI-CS-TR-97-4, Worcester Polytechnic Institute, 1997, retrieved (June 10, 2006) from http://citeseer.ist.psu.edu/ 100503.html (1997)Google Scholar
  49. 49.
    Letz, C., Henn, E.T., Vossen, G.: Consistency in Data Warehouse Dimensions. In: Proc. of Int. Database Engineering and Applications Symposium (IDEAS), pp. 224–232 (2002)Google Scholar
  50. 50.
    McBrien, P., Poulovassilis, A.: Automatic Migration and Wrapping of Database Applications - a Schema Transformation Approach. In: Akoka, J., Bouzeghoub, M., Comyn-Wattiau, I., Métais, E. (eds.) ER 1999. LNCS, vol. 1728, pp. 96–113. Springer, Berlin Heidelberg New York (1999)Google Scholar
  51. 51.
    McBrien, P., Poulovassilis, A.: Schema Evolution in Heterogeneous Database Architectures, A Schema Transformation Approach. In: Pidduck, A.B., Mylopoulos, J., Woo, C.C., Ozsu, M.T. (eds.) CAiSE 2002. LNCS, vol. 2348, pp. 484–499. Springer, Berlin Heidelberg New York (2002)Google Scholar
  52. 52.
    Mendelzon A.O., Vaisman A.A.: Temporal Queries in OLAP. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 242–253 (2000)Google Scholar
  53. 53.
    Morzy, T., Wrembel, R.: Modeling a Multiversion Data Warehouse: A Formal Approach. In: Proc. of Int. Conference on Enterprise Information Systems (ICEIS), pp. 120–127 (2003)Google Scholar
  54. 54.
    Morzy, T., Wrembel, R.: On Querying Versions of Multiversion Data Warehouse. In: Proc. ACM Int. Workshop on Data Warehousing and OLAP (DOLAP), pp. 92–101 (2004)Google Scholar
  55. 55.
    Nica, A., Lee, A., Rundensteiner, E.: CVS: The Complex Substitution Algorithm for View Synchronization. Technical Report WPI-CS-TR-97-8, Worcester Polytechnic Institute, 1997, retrieved (June 10, 2006) from http://citeseer.ist.psu.edu/ nica97cv.html (1997)Google Scholar
  56. 56.
    Nica, A., Rundensteiner, E.: Using Complex Substitution Strategies for View Synchronization. Technical Report, WPI-CS-TR-98-4, Worcester Polytechnic Institute, 1998, retrieved (June 11, 2006) from http://citeseer.ist.psu.edu/ 35922.html (1998)Google Scholar
  57. 57.
    Object Management Group. Common Warehouse Metamodel Specification, v1.1. Retrieved (August 10, 2005) from http://www.omg.org/cgi-bin/doc?formal/03-03-02Google Scholar
  58. 58.
    Overmars, M.H., van Leeuwen, J.: Dynamic multidimensional data structures based on Quad- and K-D trees. Acta Informatica 17, 267–285 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  59. 59.
    Panel discussion on ”Future trends in Data Warehousing and OLAP” at ACM Int. Workshop on Data Warehousing and OLAP (DOLAP) (2004)Google Scholar
  60. 60.
    Quass, D., Widom, J.: On-Line Warehouse View Maintenance. Proc. of ACM SIGMOD Int. Conference on Management of Data, pp. 393-404 (1997)Google Scholar
  61. 61.
    Quix, C.: Repository Support for Data Warehouse Evolution. In: Proc. of Design and Management of Data Warehouses (DMDW) (1999)Google Scholar
  62. 62.
    Rizzi, S.: Open Problems in Data Warehousing: 8 Years Later. Keynote speech at Design and Management of Data Warehouses (DMDW) (2003)Google Scholar
  63. 63.
    Roddick, J.: A Survey of Schema Versioning Issues for Database Systems. Information and Software Technology 37(7), 383–393 (1996)CrossRefGoogle Scholar
  64. 64.
    Roth, M.T., Schwarz, P.: Don’t scrap it, wrap it. A wrapper architecture for data sources. In: Proc. of Int. Conference on Very Large Data Bases (VLDB), pp. 266–275 (1997)Google Scholar
  65. 65.
    Rundensteiner, E., Koeller, A., Zhang, X.: Maintaining Data Warehouses over Changing Information Sources. Communications of the ACM, 43(6) (2000)Google Scholar
  66. 66.
    Salzberg, B., Jiang, L., Lomet, D., Barrena, M., Shan, J., Kanoulas, E.: A Framework for Access Methods for Versioned Data. In: Bertino, E., Christodoulakis, S., Plexousakis, D., Christophides, V., Koubarakis, M., Böhm, K., Ferrari, E. (eds.) EDBT 2004. LNCS, vol. 2992, pp. 730–747. Springer, Berlin Heidelberg New York (2004)Google Scholar
  67. 67.
    Sarawagi, S.: Indexing OLAP Data. IEEE Data Engineering Bulletin 20(1), 36–43 (1997)Google Scholar
  68. 68.
    Sarawagi, S., Stonebraker, M.: Efficient organization of large multidimensional arrays. In: Proc. of Int. Conference on Data Engineering (ICDE), pp. 328-336 (1994)Google Scholar
  69. 69.
    Schlesinger, L., Bauer, A., Lehner, W., Ediberidze, G., Gutzman, M.: Efficienlty Synchronizing Multidimensional Schema Data. In: Proc. of ACM Int. Workshop on Data Warehousing and OLAP (DOLAP), pp. 69-76 (2001)Google Scholar
  70. 70.
    Slowly Changing Dimension. Retrieved (July 24, 2006) from http://www.1keydata.com/datawarehousing/scd-type-1.htmlGoogle Scholar
  71. 71.
    Sjøberg, D.: Quantifying Schema Evolution. Information Software Technology 35(1), 35–54 (1993)CrossRefGoogle Scholar
  72. 72.
    Snodgrass, R. (ed.): The Temporal Query Language TSQL2. ISBN 0-7923-9614-6. Kluwer Academic Publishers, Norwell, MA, USA (1995)Google Scholar
  73. 73.
    Tansel, A., Gadia, J., Jajodia, S., Segev, A., Snodgrass, R.: (Eds.): Temporal Databases. Benjamin Cummings ISBN 0-8053-2413-5 (1993)Google Scholar
  74. 74.
    Teschke, M., Ulbrich, A.: Concurrent Warehouse Maintenance whithout Compromising Session Consistency. In: Quirchmayr, G., Bench-Capon, T.J.M., Schweighofer, E. (eds.) DEXA 1998. LNCS, vol. 1460, pp. 776–785. Springer, Berlin Heidelberg New York (1998)CrossRefGoogle Scholar
  75. 75.
    Vaisman, A., Mendelzon, A.: A Temporal Query Language for OLAP: Implementation and Case Study. In: Ghelli, G., Grahne, G. (eds.) DBPL 2001. LNCS, vol. 2397, pp. 78–96. Springer, Berlin Heidelberg New York (2002)Google Scholar
  76. 76.
    Valduriez, P.: Join Indices. ACM Transactions on Database Systems (TODS), 12(2), 218–246 (1987)CrossRefGoogle Scholar
  77. 77.
    Vetterli, T., Vaduva, A., Staudt, M.: Metadata Standards for Data Warehousing: Open Information Model vs. Common Warehouse Metadata. SIGMOD Record 29(3), 68–75 (2000)Google Scholar
  78. 78.
    Wrembel, R., Bębel, B.: Metadata Management in a Multiversion Data Warehouse. In: Meersman, R., Tari, Z. (eds.) OTM 2005: CoopIS, DOA, and ODBASE. LNCS, vol. 3761, pp. 1347–1364. Springer, Berlin Heidelberg New York (2005)CrossRefGoogle Scholar
  79. 79.
    Wrembel, R., Morzy, T.: Managing and Querying Versions of Multiversion Data Warehouse. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., Boehm, K., Kemper, A., Grust, T., Boehm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 1121–1124. Springer, Berlin Heidelberg New York (2006)CrossRefGoogle Scholar
  80. 80.
    Zhang, X., Rundensteiner, E.: Integrating the maintenance and synchronization of data warehouses using a cooperative framework. Information Systems 27, 219–243 (2002)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Robert Wrembel
    • 1
  • Bartosz Bębel
    • 1
  1. 1.Institute of Computing Science, Poznań University of Technology, PoznańPoland

Personalised recommendations