ASTracer: An Efficient Tracing Tool for HDFS with Adaptive Sampling

  • Yang Song
  • Yunchun Li
  • Shuhan Wu
  • Hailong YangEmail author
  • Wei Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11783)


Existing distributed tracing tools such as HTrace use static probabilistic samplers to collect the function call trees for performance analysis, which may fail to capture important but less executed function call trees and thus miss the opportunities for performance optimization. To address the above problem, we propose ASTracer, a new distributed tracing tool with two adaptive samplers. The advantage of adaptive samplers is that they can adjust the sampling rate dynamically, which is able to capture comprehensive function call trees and in the meanwhile maintain the size of trace file acceptable. In addition, we propose an auto-tuning mechanism to search for the optimal parameter settings of the adaptive samplers in ASTracer. The experiment results demonstrate the adaptive samplers are more effective in tracing the function call trees compared to probabilistic sampler. Moreover, we provide several case studies to demonstrate the usage of ASTracer in identifying potential performance bottlenecks.


HDFS Distributed tracing tool Adaptive sampling 



This work is supported by National Key Research and Development Program of China (Grant No. 2016YFB1000304) and National Natural Science Foundation of China (Grant No. 61502019). Hailong Yang is the corresponding author.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
    Borthakur, D.: The hadoop distributed file system: architecture and design. Hadoop Project Website 11(2007), 21 (2007)Google Scholar
  6. 6.
    Ehlers, J., van Hoorn, A., Waller, J., Hasselbring, W.: Self-adaptive software system monitoring for performance anomaly localization. In: Proceedings of the 8th ACM International Conference on Autonomic Computing, pp. 197–200. ACM (2011)Google Scholar
  7. 7.
    Fonseca, R., Porter, G., Katz, R.H., Shenker, S., Stoica, I.: X-trace: a pervasive network tracing framework. In: Proceedings of the 4th USENIX Conference on Networked Systems Design & Implementation, p. 20. USENIX Association (2007)Google Scholar
  8. 8.
    Huang, S., Huang, J., Dai, J., Xie, T., Huang, B.: The HiBench benchmark suite: characterization of the MapReduce-based data analysis. In: 2010 IEEE 26th International Conference on Data Engineering Workshops, ICDEW 2010, pp. 41–51. IEEE (2010)Google Scholar
  9. 9.
    Humayun, F., Babar, M.I.K., Zafar, M.H., Zuhairi, M.F., et al.: Performance analysis of a token bucket shaper for MPEG4 video and real audio signal. In: 2013 IEEE International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA), pp. 1–4. IEEE (2013)Google Scholar
  10. 10.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Liu, Y., Li, Y., Zhou, H., Zhang, J., Yang, H., Li, W.: A fine-grained performance bottleneck analysis method for HDFS. In: Zhang, F., Zhai, J., Snir, M., Jin, H., Kasahara, H., Valero, M. (eds.) NPC 2018. LNCS, vol. 11276, pp. 159–163. Springer, Cham (2018). Scholar
  12. 12.
    Mos, A., Murphy, J.: COMPAS: adaptive performance monitoring of component-based systems. In: Proceedings of 2nd ICSE Workshop on Remote Analysis and Measurement of Software Systems. Citeseer (2004)Google Scholar
  13. 13.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Sigelman, B.H., et al.: Dapper, a large-scale distributed systems tracing infrastructure (2010)Google Scholar
  15. 15.
    Wert, A., Schulz, H., Heger, C.: AIM: adaptable instrumentation and monitoring for automated software performance analysis. In: Proceedings of the 10th International Workshop on Automation of Software Test, pp. 38–42. IEEE Press (2015)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Yang Song
    • 1
  • Yunchun Li
    • 1
  • Shuhan Wu
    • 1
  • Hailong Yang
    • 1
    Email author
  • Wei Li
    • 1
  1. 1.School of Computer Science and EngineeringBeihang UniversityBeijingChina

Personalised recommendations