Workflow Fault Tolerance

Workflow Fault Tolerance

We built a fault tolerance framework for Kepler-based scientific workflows to support implementation of a range of comprehensive failure coverage and recovery options. The following publications  demonstrate the three major components (see below) this framework was divided into, and how the framework takes advantage of the provenance data collected by the Kepler-based workflows to detect failures and help in fault-tolerance decision making.

• a general contingency Kepler actor that provides a recovery block functionality at the workflow level
• an external monitoring module that tracks the underlying workflow components and monitors the overall health of the workflow execution
• a checkpointing mechanism that provides smart resume capabilities for cases in which an unrecoverable error occurs