SC22 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Workshops Archive

Drishti: Guiding End-Users in the I/O Optimization Journey

Workshop: PDSW22: 7th International Parallel Data Systems Workshop

Authors: Jean Luca Bez, Hammad Ather, and Suren Byna (Lawrence Berkeley National Laboratory (LBNL))

Abstract: The complex software and hardware I/O stack of HPC platforms makes it challenging for end-users to extract performance and understand the root causes of I/O bottlenecks they encounter. Despite continuous efforts from the community to profile I/O performance and propose new optimization techniques and tuning options to improve performance, there is still a translation gap between profiling and tuning. In this paper, we propose Drishti, a solution to guide end-users in optimizing I/O in their applications by detecting typical I/O performance pitfalls and providing recommendations. We illustrate its applicability in two case studies and evaluate its robustness and performance by summarizing the issues detected in over a hundred thousand Darshan logs collected by the National Energy Research Scientific Computing Center on the Cori supercomputer. Drishti can empower end-users and guide them in the I/O optimization journey by shedding some light on everyday I/O performance pitfalls and how to fix them.

Back to PDSW22: 7th International Parallel Data Systems Workshop Archive Listing

Back to Full Workshop Archive Listing