Conduit: A Successful Strategy for Describing and Sharing Data In Situ
DescriptionData representation and coupling between scientific libraries is a key challenge to building a vibrant ecosystem of HPC simulation tools. From bespoke data structures to hundreds of file-based data models, the myriad of possible choices involved both enables key features and blocks adoption of others. Connecting data between code bases requires agreeing on or adapting between data representations. While in some cases this process is trivial, for more complicated cases, adapting data becomes a costly barrier. Conduit was designed within this context to help meet the key challenge of sharing data across HPC simulation tools by providing a dynamic API to describe in-memory data. It supports coupling simulations and connecting simulations to analysis and I/O libraries. This paper provides a broad overview of Conduit, background on the evolution of the project, and details on recently added features relevant to in situ use cases.
Event Type
Workshop
TimeSunday, 13 November 20229:10am - 9:35am CST
LocationC143-149
W
Accelerator-based Architectures
Data Analytics
In Situ Processing
Scientific Computing
Visualization
Workflows
Recorded