This paper presents a method for using distributed computing technology in scientific workflows, which can achieve parallel execution of parameter sweep workflows on ad-hoc network computing resources, thereby significantly improving workflow execution performance. The method implements a Master-Slave distributed execution framework in the Kepler scientific workflow environment, which can realize parallel and independent execution of sub-workflows, and verifies the usability and time efficiency of the method through a use case in the theoretical ecology domain.
The document presents a distributed execution framework for scientific workflows built on the Kepler environment. It introduces a Master‑Slave architecture where a Master node coordinates multiple Slave nodes to run sub‑workflows in parallel, and explains how this approach simplifies the transition from local to distributed execution for scientists.
A detailed ecological use case demonstrates the architecture’s performance benefits, comparing local execution with distributed execution under different computation models (SDF and Process Network). The paper also covers implementation details such as the DistributedCompositeActor, provenance collection, and authentication services.
It provides a framework for distributing sub‑workflows to multiple computing nodes, allowing parallel execution and reducing overall workflow runtime.
It enables concurrent independent sub‑workflow executions with minimal user configuration, leading to large productivity gains for large parameter sets.