Use of HPC-Techniques for Large-Scale Data Migration

  • Jan Dünnweber
  • Valentin Mihaylov
  • René Glettler
  • Volker Maiborn
  • Holger Wolff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7155)


Any re-design of a distributed legacy system requires a migration which involves numerous complex data replication and transformation steps. Migration procedures can become quite difficult and time-consuming, especially when the setup (i.e., the employed databases, encodings, formats etc.) of the legacy and the target system fundamentally differ, which is often the case with finance data, grown over decades. We report on experiences from a real-world project: the recent migration of a customer loyalty system from a COBOL-operated mainframe to a modern service-oriented architecture. In this context, we present our easy-to-adopt solution for running most replication steps in a high-performance manner: the QuickApply HPC-software which helps minimizing the replication time, and, thereby, the overall downtime of the migration. Business processes can be kept up and running most of the time, while pre-extracted data already pass a variety of platforms and representations toward the target system. We combine the advantages of traditional migration approaches: transformations, which require the interruption of business processes are performed with static data only, they can be made undone in case of a failure and terminate quickly, due to the use of parallel processing.


Business Process Legacy System Target System Work Unit Loyalty Program 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hall, J., Hartline, J., et al.: On Algorithms for Efficient Data Migration. In: Proceedings of ACM Symposium on Discrete Algorithms, Washington, DC, USA (January 2001)Google Scholar
  2. 2.
    Allgaier, M., Heller, M.: Research Challenges for Seamless Service Integration in Enterprise Systems. In: Proceedings of IE4SOC: Workshop on Industrial Experiences for Service-Oriented Computing, Stockholm, Sweden (November 2009)Google Scholar
  3. 3.
    Brodie, M.L., Stonebraker, M.: Migrating Legacy Systems. Morgan Kaufmann, San Francisco (1995) ISBN 9781558603301Google Scholar
  4. 4.
    Wu, B., Lawless, D., et al.: The Butterfly Methodology: A Gateway-Free Approach for Migrating Legacy Information Systems. In: IEEE Conference on Engineering of Complex Computer Systems, Villa Olmo, Como, Italy (September 1997)Google Scholar
  5. 5.
    Pyla, P.S., Tungare, M., Homan, J., Pérez-Quiñones, M.A.: Continuous User Interfaces for Seamless Task Migration. In: Proceedings of HCII: IASTED Conference on Human-Computer Interaction, San Diego, CA, USA (July 2009)Google Scholar
  6. 6.
    Ebbers, M., O’Brien, W., Ogden, B.: Introduction to the New Mainframe: z/OS Basics. Vervanté books, Huntington Beach (2005) ISBN 9780738496740Google Scholar
  7. 7.
    DeWitt, D., Gray, J.: Parallel Database Systems: The Future of High Performance Database Systems. Comm. ACM 35(6), 85–98 (1992)CrossRefGoogle Scholar
  8. 8.
    Orlando, S., Orsini, R., Raffaetà, A., Roncato, A.: Trajectory Data Warehouses: Design and Implementation Issues. Journal of Computing Science and Engineering 1(2), 211–232 (2007)Google Scholar
  9. 9.
    Jäntti, J., Kerry, D., Kompalka, A., Long, R., Mitchell, G.: DataPropagator Implementation Guide. IBM Redbooks, Armonk (2002) ISBN 0738426342Google Scholar
  10. 10.
    Liang, S.: JavaTM Native Interface: Programmer’s Guide and Specification. Prentice Hall, Harlow (1999) ISBN 9780201325775Google Scholar
  11. 11.
    O’Neil, P.: Database: Principles Programming and Performance. Morgan Kaufmann Publishers (June 1994) ISBN 9781558602199Google Scholar
  12. 12.
    Dünnweber, J., Gorlatch, S.: Higher-Order Components for Grid Programming: Making Grids More Usable. Springer, Germany (2009) ISBN 9783642008405zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jan Dünnweber
    • 1
  • Valentin Mihaylov
    • 1
  • René Glettler
    • 1
  • Volker Maiborn
    • 1
  • Holger Wolff
    • 1
  1. 1.MaibornWolff GmbHMünchenGermany

Personalised recommendations