Cloud computing workflow scheduling with maximum reduction of effective resources

Abstract

In order to solve the problem that resource utilization efficiency of large-scale scientific workflow is low in cloud computing environment, this paper proposes an algorithm of maximum effective resource reduction (MERR). The algorithm is mainly implemented in three steps. First, identify the delay limitation, finding the balance between the reduction of effective resource use and the increase of time. Second, task merge and merge the tasks with low resource utilization in original workflow scheduling. The third is resource consolidation, a best fitting method is adopted to combine resources that are not fully used, so as to improve the efficiency of resource utilization. Using CyberShake, Epigenomics, LIGO and Montage four kinds of scientific workflow to carry out simulation experiments. The results show that MERR has reduced using resources by 54%, and the average time increase is less than 10%, which are better than scheduling algorithm based on the critical path.