Big data framework for quantitative trading system

  • Shuji Dai (戴书吉)
  • Xing Wu (武 星)
  • Mengqi Pei (裴孟齐)
  • Zhikang Du (杜智康)


Massive trading data are produced in securities market every day. Besides, the amount of relevant social media data is also growing fast. It is a vital problem of making use of these data. Facing on the growing amount of data, using big data framework is a necessary and reasonable method. Then, a big data framework for quantitative trading system is proposed in this paper. In the framework, Apache Spark is chosen as the distributed computing framework to process trading data, and Apache HBase as the distributed database is used to store data. After introducing the whole framework, we discussed data sources and the structure of quantitative trading layer in detail.

Key words

big data framework quantitative trading Apache Spark 

CLC number

F 830.91 

Document code


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    KAUFFMAN R J, HU Y, MA D. Will high-frequency trading practices transform the financial markets in the Asia Pacific Region? [J]. Financial Innovation, 2015, 1(1): 1–27.CrossRefGoogle Scholar
  2. [2]
    KEARNS M, ORTIZ L. The Penn-Lehman automated trading project [J]. IEEE Intelligent Systems, 2003, 18(6): 22–31.CrossRefGoogle Scholar
  3. [3]
    TRELEAVEN P, GALAS M, LALCHAND V. Algorithmic trading review [J]. Communications of the ACM, 2013, 56(11): 76–85.CrossRefGoogle Scholar
  4. [4]
    ZAHARIA M, CHOWDHURY M, FRANKLIN M J, et al. Spark: Cluster computing with working sets [C]//Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing. Boston, MA: USENIX Association, 2010: 1765–1773.Google Scholar
  5. [5]
    SPARK A. Spark programming guide [EB/OL]. (2015-10-4). [2016-11-14]. docs/latest/programming-guide.html.Google Scholar
  6. [6]
    SPARK A. Apache spark–lightning-fast cluster computing [EB/OL]. (2014-4-21). [2016-11-14]. http:// Scholar
  7. [7]
    VORA M N. Hadoop-HBase for large-scale data [C]//Computer Science and Network Technology (ICCSNT), 2011 International Conference. Harbin: IEEE, 2011: 601–605.CrossRefGoogle Scholar
  8. [8]
    NARANG R K. Inside the black box: the simple truth about quantitative trading [M]. Canada: John Wiley & Sons, 2009.CrossRefGoogle Scholar
  9. [9]
    GRUNDY B D, KIM Y. Stock market volatility in a heterogeneous information economy [J]. Journal of Financial and Quantitative Analysis, 2002, 37(1): 1–27.CrossRefGoogle Scholar
  10. [10]
    KWON K Y, KISH R J. A comparative study of technical trading strategies and return predictability: An extension of Brock, Lakonishok, and LeBaron (1992) using NYSE and NASDAQ indices [J]. The Quarterly Review of Economics and Finance, 2002, 42(3): 611–631.CrossRefGoogle Scholar

Copyright information

© Shanghai Jiaotong University and Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Shuji Dai (戴书吉)
    • 1
  • Xing Wu (武 星)
    • 1
    • 2
  • Mengqi Pei (裴孟齐)
    • 1
  • Zhikang Du (杜智康)
    • 1
  1. 1.School of Computer Engineering and ScienceShanghai UniversityShanghaiChina
  2. 2.Shanghai Key Laboratory of Financial Information TechnologyShanghai University of Finance and EconomicsShanghaiChina

Personalised recommendations