IOPathTune: Adaptive Online Parameter Tuning for Parallel File System I/O Path
DescriptionParallel file systems like Lustre contain complicated I/O paths from clients to storage servers. An efficient I/O path requires proper settings of multiple parameters as the default settings often fail to deliver optimal performance, especially for diverse workloads in the HPC environment. Existing tuning strategies are limited in being adaptive, timely, and flexible. We propose IOPathTune, which adaptively tunes PFS I/O Path online from the client side without characterizing the workloads, doing expensive profiling, and communicating with other machines. We leveraged CloudLab to conduct the evaluations with 20 different Filebench workloads under three different test conditions: single-client standalone tests, dynamic workload change, and multi-client executions. We observed either on par or better performance than the default configuration across all workloads. Some of the most considerable improvement includes 231%, 113%, 96%, 43%.
