Balanced Iterative Reducing and Clustering Using Hierarchies with Principal Component Analysis (PBirch) for Intrusion Detection over Big Data in Mobile Cloud Environment

  • Kai Peng
  • Lixin ZhengEmail author
  • Xiaolong Xu
  • Tao Lin
  • Victor C. M. Leung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11342)


With the development of big data, mobile cloud computing, cyber security issues have become more and more critical. Thus, enabling an intrusion detection method over big data in mobile cloud environment is of paramount importance. In our previous research, we proposed an approach named Mini Batch Kmeans with Principal Component Analysis (PMBKM) for big data which can effectively solve the clustering problem for intrusion detection of big data, but it needs to preset the number of clusters. The best clustering number is selected by comparing the clustering results of different clustering values multiple times. To address the above issue, we propose a new clustering method named Balanced Iterative Reducing and Clustering Using Hierarchies with Principal Component Analysis (PBirch) in this paper. Compared to PMBKM, the experimental results show that PBirch can obtain a good clustering result without presetting clustering values, and the clustering result can be further improved by optimizing the relevant parameters. The clustering time of PBirch decreases linearly with the increasing of the cluster numbers. Thus, the larger the number of clusters, the smaller the PBirch time cost. All in all, our proposed method can be widely used for big data in mobile cloud environment.


Mobile cloud environment Big data Intrusion detection Birch PCA 



This work is supported by The Natural Science Foundation of Fujian Province (Grant No. 2018J05106), Quanzhou Science and Technology Project (No. 2015Z115), the Scientific Research Foundation of Huaqiao University (No. 14BS316). The Education Scientific Research Project for Middle-age and Young Teachers of Fujian Province (JZ160084). China Scholarship Council awards to Kai Peng for one year’s research abroad at The University of British Columbia, Vancouver, Canada. The authors also wants to thank Jianping Liu, Zhiqiang Xu and etc. for sharing a lot of valuable information on his blog.


  1. 1.
    Anderson, J.P.: Computer security threat monitoring and surveillance. Technical Report, vol. 17. James P. Anderson Company, Pennsylvania (1980)Google Scholar
  2. 2.
    Denning, D.E.: An intrusion-detection model. IEEE Trans. Softw. Eng. 2, 222–232 (1987)CrossRefGoogle Scholar
  3. 3.
    Milenkoski, A., Vieira, M., Kounev, S., Avritzer, A., Payne, B.D.: Evaluating computer intrusion detection systems: a survey of common practices. ACM Comput. Surv. (CSUR) 48(1), 1–41 (2015)CrossRefGoogle Scholar
  4. 4.
    Wang, T., et al.: Fog-based storage technology to fight with cyber threat. Future Gener. Comput. Syst. 83, 208–218 (2018)CrossRefGoogle Scholar
  5. 5.
    Peng, K., Lin, R.H., Huang, B.B., Zou, H., Yang, F.C.: Link importance evaluation of data center network based on maximum flow. J. Internet Technol. 18(1), 23–31 (2017)Google Scholar
  6. 6.
    Wang, T., et al.: Data collection from WSNs to the cloud based on mobile fog elements. Future Gener. Comput. Syst. (2017). Scholar
  7. 7.
    Wang, T., Zhang, G.X., Bhuiyan, M.Z.A., Liu, A.F., Jia, W., Xie, M.: A novel trust mechanism based on fog computing in sensor-cloud system. Future Gener. Comput. Syst. (2018). Scholar
  8. 8.
    Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)CrossRefGoogle Scholar
  9. 9.
    Wang, T., Bhuiyan, M.Z.A., Wang, G.J., Rahman, A., Wu, J., Cao, J.N.: Big data reduction for smart city’s critical infrastructural health monitoring. IEEE Commun. Mag. 56(3), 128–133 (2018)CrossRefGoogle Scholar
  10. 10.
    Lee, W., Stolfo, S.J.: Data mining approaches for intrusion detection. In: 7th USENIX. USENIX Security Symposium, pp. 79–93 (1998)Google Scholar
  11. 11.
    Peng, K., Leung, V.C.M., Huang, Q.J.: Clustering approach based on mini batch Kmeans for intrusion detection system over big data. IEEE Access 6, 11897–11906 (2018)CrossRefGoogle Scholar
  12. 12.
    Peng, K., Leung, V.C.M., Zheng, L.X., Wang, S.G., Huang, C., Lin, T.: Intrusion detection system based on decision tree over big data in fog environment. Wirel. Commun. Mob. Comput. (2018). Scholar
  13. 13.
    Halko, N., Martinsson, P.G., Tropp, J.A.: Finding Structure with Randomness: Stochastic Algorithms for Constructing Approximate Matrix Decompositions.
  14. 14.
    Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analyzers. Neural Comput. 11(2), 443–482 (1999)CrossRefGoogle Scholar
  15. 15.
    Martinsson, P.G., Rokhlin, V., Tygert, M.: A randomized algorithm for the decomposition of matrices. Appl. Comput. Harmonic Anal. 30(1), 47–68 (2011)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Zhang, T., Ramakrishnan, R., Livny, M.: An efficient data clustering method for very large databases. In: Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data (SIGMOD 1996), pp. 103–114. ACM, New York (1996)Google Scholar
  17. 17.
    Calinski, T., Harabasz, J.: A dendrite method for cluster analysis. Commun. Stat.-Theory Methods 3(1), 1–27 (1974)MathSciNetCrossRefGoogle Scholar
  18. 18.
  19. 19.
  20. 20.

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.College of EngineeringHuaqiao UniversityQuanzhouChina
  2. 2.Department of Electrical and Computer EngineeringThe University of British ColumbiaVancouverCanada
  3. 3.Fujian Provincial Academic Engineering Research Centre in Industrial Intellectual Techniques and SystemsQuanzhouChina
  4. 4.School of Computer and SoftwareNanjing University of Information Science and TechnologyNanjingChina
  5. 5.State Key Laboratory of Networking and Switching TechnologyBeijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations