Birds of a Feather Archive

The Challenge of Distributed Data in Scientific Computing for Earth Sciences

Authors: Ilene Carpenter (Hewlett Packard Enterprise (HPE)), Ann Dunkin (US Department of Energy)

Abstract: The explosion in scientific data volume, distributed from the edge to on-premise storage to storage on the cloud, is creating acute challenges and unique solutions within the High Performance Computing (HPC) community. Problems span workflows that produce and/or ingest distributed data, edge-to-cloud architectures, data distribution, data representation, and beyond. Join this BoF to meet individuals looking to address this challenge; hear from leaders from within this community; and be part of the discussion about your challenges, what you have done, and identifying the solutions that are needed.

Long Description: Distributed data is creating acute challenges and unique solutions within the High Performance Computing (HPC) community. Traditionally, the High Performance Scientific Computing community has been working within closed ecosystems. Today, scientists interact with a wide variety of data inputs (e.g. social media sources) and collaborators that may not have traditional high-performance capabilities or backgrounds. Unresolved, this creates issues that limit scientific breakthroughs that HPC is designed to resolve.

The panel will provide real-world examples that the earth sciences community is facing. Problems span data movement and sharing, highly complex workflows that are reliant (or built for) distributed data, real-time data requirements, edge-to-cloud architectures, data distribution, and beyond. Some solutions have caught our imagination – such as digital twins and operational platforms.

The discussion is relevant to all HPC communities that are working to incorporate distributed data challenges. Please bring your user stories and challenges to this BOF and hear from experts and fellow audience members that are on this journey.

This BOF is designed to bring together this community to understand the commonalities in our problems and to hear about the solutions that are being implemented.

URL:


Back to Birds of a Feather Archive Listing