applications are often structured as workflows that execute a series of interdependent, distributed software modules to analyze large data sets. Passing very large data sets between workflow steps can incur unacceptable overheads with workflow engines that are not designed to efficiently handle large data transfers. In this paper, we describe how we have enhanced a Kepler workflow application with the MeDICi Integration Framework, which has been specifically designed to support the execution of data intensive workflows. The MeDICi technology provides a scalable, component-based architecture that efficiently handles integration with heterogeneous, distributed software systems. This paper describes the MeDICi Integration Framework and the mechanisms we used to integrate MeDICi components with Kepler workflow actors. We illustrate this solution with a workflow application for an atmospheric sciences application, which demonstrates the advantages of this approach.
Revised: March 1, 2011 |
Published: July 10, 2009
Citation
Chase J.M., I. Gorton, C. Sivaramakrishnan, J.P. Almquist, A.S. Wynne, G. Chin, and T.J. Critchlow. 2009.Kepler + MeDICi. In 2009 IEEE Congress on Services - I, July 6-10, Los Angeles, California, edited by LJ Zhang, 275 - 282. Los Alamitos, California:IEEE Computer Society.PNNL-SA-65047.