Abstract
Multi-Application onLine Profiling (MALP) is a performance tool which has been developed as an alternative to the trace-based approach for fine-grained event collection. Any performance and analysis measurement system must address the problem of data management and projection to meaningful forms. Our concept of a valorization chain is introduced to capture this fundamental principle. MALP is a dramatic departure from performance tool dogma in that is advocates for an online valorization architecture that integrates data producers with transformers, consumers, and visualizers, all operating in concert and simultaneously. MALP provides a powerful, dynamic framework for performance processing, as is demonstrated in unique performance analysis and application dashboard examples. Our experience with MALP has identified opportunities for data-query in MPI context, and more generally, creating a “constellation of services” that allow parallel processes and tools to collaborate through a common mediation layer.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Valorize means to give or ascribe value or validity to something.
- 2.
Dilation is a function of the duration ratio between events of interest and instrumentation cost.
References
Adhianto L, Banerjee S, Fagan M, Krentel M, Marin G, Mellor-Crummey J, Tallent NR (2010) HPCToolkit: tools for performance analysis of optimized parallel programs. Concurr. Comput.: Pract. Exp. 22(6):685–701
Arnold, D.C., Ahn, D.H., de Supinski, B.R., Lee, G.L., Miller, B.P., Schulz, M.: Stack trace analysis for large scale debugging. In: IEEE International Parallel and Distributed Processing Symposium, IPDPS. pp. 1–10. IEEE (2007)
Benedict, S., Petkov, V., Gerndt, M.: Periscope: an online-based distributed performance analysis tool. Tools for High Performance Computing 2009, pp. 1–16. Springer, Berlin (2010)
Besnard, J.B.: Profiling and Debugging by Efficient Tracing of Hybrid Multi-Threaded HPC Applications. Ph.D. thesis, Université de Versailles Saint Quentin en Yvelines (2014)
Besnard, J.B., Pérache, M., Jalby, W.: Event streaming for online performance measurements reduction. In: 42nd International Conference on Parallel Processing (ICPP), pp. 985–994. IEEE (2013)
Chan A, Gropp W, Lusk E (2008) An efficient format for nearly constant-time access to arbitrary time intervals in large trace files. Sci. Program. 16(2–3):155–165
Crockford, D.: The Application/Json Media Type for Javascript Object Notation (json) (2006)
von Eicken, T., Culler, D.E., Goldstein, S.C., Schauser, K.E.: Active messages: a mechanism for integrated communication and computation. In: Proceedings of the 19th Annual International Symposium on Computer Architecture, ISCA ’92, pp. 256–266. ACM, New York, NY, USA (1992). http://doi.acm.org/10.1145/139669.140382
Eschweiler D, Wagner M, Geimer M, Knüpfer A, Nagel WE, Wolf F (2011) Open trace format 2: the next generation of scalable trace formats and support libraries. PARCO. 22:481–490
Frings, W., Wolf, F., Petkov, V.: Scalable massively parallel i/o to task-local files. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, pp. 1–11. IEEE (2009)
Gansner ER, North SC (2000) An open graph visualization system and its applications to software engineering. Soft. -Pract. Exp. 30(11):1203–1233
Geimer M, Wolf F, Wylie BJ, Ábrahám E, Becker D, Mohr B (2010) The scalasca performance toolset architecture. Concurr. Comput. Pract. Exp. 22(6):702–719
Hilbrich, T., Müller, M.S., de Supinski, B.R., Schulz, M., Nagel, W.E.: GTI: A generic tools infrastructure for event-based tools in parallel systems. In: IEEE 26th International Parallel & Distributed Processing Symposium (IPDPS), pp. 1364–1375. IEEE (2012)
Hilbrich, T., Schulz, M., de Supinski, B.R., Müller, M.S.: MUST: A Scalable approach to runtime error detection in MPI programs. Tools for High Performance Computing 2009, pp. 53–66. Springer, Berlin (2010)
Knüpfer, A., Brendel, R., Brunst, H., Mix, H., Nagel, W.E.: Introducing the open trace format (OTF). Computational Science–ICCS 2006, pp. 526–533. Springer, Berlin (2006)
Knüpfer, A., Brunst, H., Doleschal, J., Jurenz, M., Lieber, M., Mickler, H., Müller, M.S., Nagel, W.E.: The vampir performance analysis tool-set. Tools for High Performance Computing, pp. 139–155. Springer, Berlin (2008)
Knüpfer, A., Rössel, C., an Mey, D., Biersdorff, S., Diethelm, K., Eschweiler, D., Geimer, M., Gerndt, M., Lorenz, D., Malony, A., et al.: Score-P: a joint performance measurement run-time infrastructure for periscope, scalasca, TAU, and vampir. Tools for High Performance Computing 2011, pp. 79–91. Springer, Berlin (2012)
Mainwaring, A.M., Culler, D.E.: Active message applications programming interface and communication subsystem organization. Technical Report UCB/CSD-96-918, EECS Department, University of California, Berkeley (Oct 1996). http://www.eecs.berkeley.edu/Pubs/TechRpts/1996/5768.html
Nataraj, A., Malony, A.D., Morris, A., Arnold, D., Miller, B.: A framework for scalable, parallel performance monitoring using TAU and MRnet. In: International Workshop on Scalable Tools for High-End Computing (STHEC 2008), Island of Kos, Greece (2008)
de Oliveira Stein, B., de Kergommeaux, J.C., Mounié, G.: Pajé Trace File Format. Technical report, ID-IMAG, Grenoble, France, 2002. http://www-id.imag.fr/Logiciels/paje/publications (2010)
Roth, P.C., Arnold, D.C., Miller, B.P.: MRNet: A Software-based multicast/reduction network for scalable tools. In: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, p. 21. ACM (2003)
Schulz, M., de Supinski, B.R.: P\(^n\)MPI tools: a whole lot greater than the sum of their parts. In: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, p. 30. ACM (2007)
Shende SS, Malony AD (2006) The TAU parallel performance system. Int. J. High Perform. Comput. Appl. 20(2):287–311
Vetter, J., Chambreau, C.: MPIP: Lightweight, scalable MPI profiling (2005)
Willcock, J.J., Hoefler, T., Edmonds, N.G., Lumsdaine, A.: AM++: a generalized active message framework. In: Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques, PACT ’10, pp. 401–410. ACM, New York, NY, USA (2010). http://doi.acm.org/10.1145/1854273.1854323
Zaki O, Lusk E, Gropp W, Swider D (1999) Toward scalable performance visualization with jumpshot. Int. J. High Perform. Comput. Appl. 13(3):277–288
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Besnard, JB., Malony, A.D., Shende, S., Pérache, M., Jaeger, J. (2016). Gleaming the Cube: Online Performance Analysis and Visualization Using MALP. In: Knüpfer, A., Hilbrich, T., Niethammer, C., Gracia, J., Nagel, W., Resch, M. (eds) Tools for High Performance Computing 2015. Springer, Cham. https://doi.org/10.1007/978-3-319-39589-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-39589-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-39588-3
Online ISBN: 978-3-319-39589-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)