The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: PMBS22: The 13th International Workshop on Performance Modeling, Benchmarking, and Simulation of High-Performance Computer Systems
Authors: Taina Coleman (University of Southern California (USC)); Henri Casanova (University of Hawaii at Manoa); Ketan Maheshwari (Oak Ridge National Laboratory (ORNL)); Loic Pottier (University of Southern California (USC)); Sean Wilkinson (Oak Ridge National Laboratory (ORNL)); Justin Wozniak (Argonne National Laboratory (ANL)); and Frederic Suter, Mallikarjun Shankar, and Rafael Ferreira da Silva (Oak Ridge National Laboratory (ORNL))
Abstract: The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class HPC clusters. To handle deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade, creating a need for workflow benchmarks to evaluate the performance of these WMSs on current and future software stacks and hardware platforms.
We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code and executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. Our experimental results show that our approach generates benchmarks representative of production workflows and conducts a case study to demonstrate the use/usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.