Abstract
Following the job-centric monitoring concept, Job Provenance (JP) service organizes provenance records on the per-job basis. It is designed to manage very large number of records, as was required in the EGEE project where it was developed originally.
The quantitative aspect is also a focus of the presented demonstration. We show JP capability to retrieve data items of interest from a large dataset of full records of more than 1 million of jobs, to perform non-trivial transformation on those data, and organize the results in such a way that repeated interactive queries are possible.
The application area of the demo is derived from that of previous Provenance Challenges. Though the topic of the demo — a computational experiment — is arranged rather artificially, the demonstration still delivers its main message that JP supports non-trivial transformations and interactive queries on large data sets.
This work has been supported by Czech research intents MSM6383917201 and MSM0021622419. Job Provenance was developed in the EU EGEE-II project, INFSO-RI-031688.
Chapter PDF
References
Dvořák, F., et al.: gLite job provenance. In: Moreau, L., Foster, I. (eds.) IPAW 2006. LNCS, vol. 4145, pp. 246–253. Springer, Heidelberg (2006)
Křenek, A., et al.: gLite job provenance—a job-centric view. Concurrency and Computation: Practice and Experience 20(5) (2007) doi: 10.1002/cpe.1252
Křenek, A., et al.: Multiple ligand trajectory docking study —semiautomatic analysis of molecular dynamics simulations using EGEE gLite services. In: Proc. Euromicro Conference on Parallel Distributed and network-based Processing (2008)
Schovancová, J., et al.: VO AUGER large scale Monte Carlo simulations using the EGEE grid environment. In: 3rd EGEE User Forum, Clermont-Ferrand, France (2008)
Křenek, A., et al.: Experimental evaluation of job provenance in ATLAS environment. J. Phys.: Conf. Series (accepted, 2007)
Head, D., et al.: Frontal-hippocampal double dissociation between normal aging and Alzheimer’s disease. Celebral Cortex 15(6), 732–739 (2005)
Matyska, L., et al.: Job tracking on a grid—the Logging and Bookkeeping and Job Provenance services. Technical Report 9/2007, CESNET (2007), http://www.cesnet.cz/doc/techzpravy
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Křenek, A. et al. (2008). Job Provenance – Insight into Very Large Provenance Datasets. In: Freire, J., Koop, D., Moreau, L. (eds) Provenance and Annotation of Data and Processes. IPAW 2008. Lecture Notes in Computer Science, vol 5272. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89965-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-89965-5_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-89964-8
Online ISBN: 978-3-540-89965-5
eBook Packages: Computer ScienceComputer Science (R0)