The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: Women in HPC: Diversifying the HPC Community and Engaging Male Allies
Authors: Chris Brumgard, Anjus George, Rick Mohr, Ketan Maheshwari, James Simmons, and Sarp Oral (Oak Ridge National Laboratory (ORNL))
Abstract: Large scale parallel file systems with multiple storage tiers require policy driven data management to facilitate efficient storage and access of large-scale data. However, management of data across the tiers is challenging due to the massive scale of data being stored. In this talk, we present our initial work on QuickSilver, a light-weight flexible distributed policy engine. QuickSilver is composed of many single-purpose agents that handle tasks such as gathering file metadata, enforcing policy decisions, and executing policy actions like purging or data migration. These agents are designed to communicate using distributed message queues, while maintaining minimal state information. We will discuss the architectural details of the policy engine and its use of message queues to enable scaling. Examples of the initial implementation will be shown with preliminary performance numbers. Since this project is in its infancy, we will also discuss our plans for future work and areas of improvement.
Back to Women in HPC: Diversifying the HPC Community and Engaging Male Allies Archive Listing