Abstract
To analyze and understand the growing wealth of scientific data, complex workflows need to be assembled, often requiring the combination of loosely-coupled resources, specialized libraries, distributed computing infrastructure, and Web services. However, constructing these workflows is a non-trivial task, especially for users who do not have programming expertise. This problem is compounded for exploratory tasks, where the workflows need to be iteratively refined. In this paper, we introduce workflow medleys, a new approach for manipulating collections of workflows. We propose a workflow manipulation language that includes operations that are common in exploratory tasks and present a visual interface designed for this language. We briefly discuss how medleys have been applied in two (real) applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Yahoo! Pipes, http://pipes.yahoo.com
The Taverna Project, http://taverna.sourceforge.net
The VisTrails Project, http://www.vistrails.org
Aalst, W., Hee, K.: Workflow Management: Models, Methods, and Systems. MIT Press, Cambridge (2002)
Business process execution language for web services version 1.1 (February 2008), http://www.ibm.com/developerworks/library/specification/ws-bpel
Freire, J., Silva, C.T., Callahan, S.P., Santos, E., Scheidegger, C.E., Vo, H.T.: Managing rapidly-evolving scientific workflows. In: Moreau, L., Foster, I. (eds.) IPAW 2006. LNCS, vol. 4145, pp. 10–18. Springer, Heidelberg (2006)
Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)
The matplotlib library, http://matplotlib.sourceforge.net
The Kepler Project, http://kepler-project.org
Lee, E.A., Parks, T.M.: Dataflow Process Networks. Proceedings of the IEEE 83(5), 773–801 (1995)
The Chembiogrid web site, http://www.chembiogrid.org
Schroeder, W., Martin, K., Lorensen, B.: The Visualization Toolkit An Object-Oriented Approach To 3D Graphics. Kitware (2003)
Parker, S.G., Johnson, C.R.: SCIRun: a scientific programming environment for computational steering. In: Supercomputing (1995)
Deelman, E., Singh, G., Su, M.H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J.C., Katz, D.S.: Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Scientific Programming Journal 13(3), 219–237 (2005)
Microsoft Workflow Foundation, http://msdn2.microsoft.com/en-us/netframework/aa663322.aspx
Foster, I., Voeckler, J., Wilde, M., Zhao, Y.: Chimera: A virtual data system for representing, querying and automating data derivation. In: Statistical and Scientific Database Management (SSDBM), pp. 37–46 (2002)
Lawrence, P. (ed.): Workflow Handbook. Workflow Management Coalition. John Wiley and Sons, Chichester (1997)
van der Aalst, W.: Business process management: A personal view. Business Process Management Journal 10(2), 135–139 (2004)
Mohan, C., Alonso, G., Günthör, R., Kamath, M.: Exotica: A research perspective of workflow management systems. IEEE Data Engineering Bulletin 18(1), 19–26 (1995)
Deelman, E., Gil, Y.: NSF Workshop on Challenges of Scientific Workflows. Technical report, NSF (2006), http://vtcpc.isi.edu/wiki/index.php/Main_Page
Flickr, http://www.flickr.com
Facebook, http://www.facebook.com
Viegas, F.B., Wattenberg, M., van Ham, F., Kriss, J., McKeon, M.: Many eyes: A site for visualization at internet scale. IEEE Transactions on Visualization and Computer Graphics 13(6), 1121–1128 (2007)
Swivel, http://www.swivel.com
Social data analysis workshop (2008), http://researchweb.watson.ibm.com/visual/social_data_analysis_workshop
Myexperiment, http://www.myexperiment.org
Biton, O., Cohen-Boulakia, S., Davidson, S.B.: Zoom*userviews: querying relevant provenance in workflow systems. In: VLDB 2007: Proceedings of the 33rd international conference on Very large data bases, VLDB Endowment, pp. 1366–1369 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Santos, E., Koop, D., Vo, H.T., Anderson, E.W., Freire, J., Silva, C. (2009). Using Workflow Medleys to Streamline Exploratory Tasks. In: Winslett, M. (eds) Scientific and Statistical Database Management. SSDBM 2009. Lecture Notes in Computer Science, vol 5566. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02279-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-02279-1_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02278-4
Online ISBN: 978-3-642-02279-1
eBook Packages: Computer ScienceComputer Science (R0)