Automatic, Efficient, and Scalable Provenance Registration for FAIR HPC Workflows
DescriptionProvenance registration is becoming more and more important as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. We propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other environments with more relaxed restrictions, such as distributed ones). It also must be scalable in order to deal with large workflows, that are more typically used in HPC. We also target transparency for the user, shielding them from having to specify how provenance must be recorded. We implement our design using the COMPSs programming model as a Workflow Management System (WfMS) and use RO-Crate as a well-established standard to record provenance. Experiments are provided, demonstrating the efficiency and scalability of our solution.
Event Type
Workshop
TimeMonday, 14 November 20229:12am - 9:24am CST
LocationD222
W
Cloud and Distributed Computing
In Situ Processing
Scientific Computing
Workflows
Recorded
Author/Presenters
