The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: PDSW22: 7th International Parallel Data Systems Workshop
Authors: Jean Luca Bez, Hammad Ather, and Suren Byna (Lawrence Berkeley National Laboratory (LBNL))
Abstract: The complex software and hardware I/O stack of HPC platforms makes it challenging for end-users to extract performance and understand the root causes of I/O bottlenecks they encounter. Despite continuous efforts from the community to profile I/O performance and propose new optimization techniques and tuning options to improve performance, there is still a translation gap between profiling and tuning. In this paper, we propose Drishti, a solution to guide end-users in optimizing I/O in their applications by detecting typical I/O performance pitfalls and providing recommendations. We illustrate its applicability in two case studies and evaluate its robustness and performance by summarizing the issues detected in over a hundred thousand Darshan logs collected by the National Energy Research Scientific Computing Center on the Cori supercomputer. Drishti can empower end-users and guide them in the I/O optimization journey by shedding some light on everyday I/O performance pitfalls and how to fix them.
Back to PDSW22: 7th International Parallel Data Systems Workshop Archive Listing