Design and realization of bank history data management system based on Hadoop 2.0
- 52 Downloads
With the continuous economic development, the total amount of data in banks is growing bigger and bigger and it is urgent and necessary to manage these data effectively. This paper studied Hadoop-based bank historical data management from the perspective of data migration, and designed and realized a data migration module. Map Reduce data migration was used in the process of structured data migration. An IO load-based scheduling algorithm was also designed. When scheduling, considering the consumption of resources, it is avoided to assign tasks to heavy-loaded IO nodes. An unstructured data migration tool was developed by smartly using FTP to concurrently migrate log files of online service platforms and other data to HDFS specified directories. The final system test results show that the system in this paper can work normally and meet the design requirements.
KeywordsHadoop Bank history data Management system
Meiwen Guo was supported by “13th Five-Year” plan research project of philosophy and social sciences of Guangdong province; “Research of the co-creation mechanism of shared economic dynamic value based on deep neural network” (Project number: GD17YGL03); Humanity and social science youth foundation of Ministry of Education of China “Business agglomeration attractiveness to consumers based on the “Internet plus”: an empirical study on the evolution process and mechanism” (Project number: 16YJCZH119).
- 1.Blixt, S.: Computer-supported gene bank management. Die Kulturpflanze 36(1), 912 (2017)Google Scholar
- 2.Paunović, M., Jovanović, T., Karapandža, B., Habijan-Mikeš, V.: Revision of bank vole Clethrionomys glareolus (Schreber, 1780) (Mammalia, Rodentia) distribution in Serbia and Montenegro. Arch. Biol. Sci. 57(1), 775–776 (2015)Google Scholar
- 3.Kittner, S.J., Sharkness, C.M., Price, T.R., Plotnick, G.D., Dambrosia, J.M., Wolf, P.A., Mohr, J.P., Hier, D.B., Kase, C.S., Tuhrim, S.: Infarcts with a cardiac source of embolism in the NINCDS Stroke data bank: historical features. Neurology 40(2), 291 (2016)Google Scholar
- 4.Berman, H.M.: The protein data bank: a historical perspective. Acta Cryst. A 64(1), 15–16 (2017)Google Scholar
- 5.Kittner, S.J., Sharkness, C.M., Price, T.R., Plotnick, G.D., Dambrosia, J.M., Wolf, P.A., Mohr, J.P., Hier, D.B., Kase, C.S., Tuhrim, S.: Infarcts with a cardiac source of embolism in the NINCDS stroke data bank: historical features. Neurology 40(2), 86–87 (2017)Google Scholar
- 6.Cabello, J.C., Lobillo, F.J.: Sound branch cash management for less: a low-cost forecasting algorithm under uncertain demand. Omega 32, 79–81 (2016)Google Scholar
- 7.Anonymous. Brocade: Brocade enhances data management at Van Lanschot Bank. M2 Presswire 19, 370–371 (2017)Google Scholar
- 8.Mora, M.C., Wong, K.E., Friderici, J., Bittner, K., Moriarty, K.P., Patterson, L.A., Gross, R.I., Tirabassi, M.V., Tashjian, D.B.: Operative versus nonoperative management of pediatric blunt pancreatic trauma: evaluation of the National trauma data bank. J. Am. Coll. Surg. 222(6), 97–98 (2016)CrossRefGoogle Scholar
- 10.Bourne, P.E., Westbrook, J., Berman, H.M.: The protein data bank and lessons in data management. Brief. Bioinform. 5(1), 99–100 (2017)Google Scholar