Workshop: 4th Workshop on Programming and Performance Visualization Tools
Authors: Kevin Huck (University of Oregon), Xingfu Wu and Anshu Dubey (Argonne National Laboratory (ANL)), Antigoni Georgiadou and J. Harris (Oak Ridge National Laboratory (ORNL)), Tom Klosterman (Argonne National Laboratory (ANL)), Matthew Trappett (University of Oregon), and Klaus Weide (Argonne National Laboratory (ANL))
Abstract: State-of-the-art multiphysics simulations running on large scale leadership computing platforms have many variables contributing to their performance and scaling behavior. We recently encountered an interesting performance anomaly in Flash-X, a multiphysics multicomponent simulation software, when characterizing its performance behavior on several large-scale platforms. The anomaly was tracked down to the interaction between the use of dynamic allocation of scratch data and data locality in the cache hierarchy. In this paper we present the details of unexpected performance variability we encountered, the extensive analysis using the performance measurement tool TAU to collect the data and Python data analysis libraries to explore the data, and our insights from this experience. In the process, we discovered and removed or mitigated two additional performance limiting bottlenecks for performance tuning.