Invited Talk: Bioinformatic Workflows
DescriptionContainers have become an essential tool for the deployment of scientific data pipelines in a portable and reproducible manner across clusters and clouds. The immutability of containers is an essential concept in this context because it guarantees that the environment in which pipeline tasks are executed is not altered over time and is precisely replicable. However, this principle collides with the continuously evolving nature of scientific data analyses. Modern scientific workflows can be composed of several dozens of tasks and corresponding containers, which developers need to modify and deploy promptly to validate experiment hypotheses. This may require re-building dozens of containers and uploading them to remote repositories, which forces developers to waste a significant amount of time on “infrastructure” tasks and adds considerable overhead to the overall development and deployment cycle. This talk will introduce a novel solution to the problem of container provisioning for data analysis pipelines based on the concept of container augmentation, which allows the extension of the container content at runtime without the need to store it in a persistent repository.
Event Type
Workshop
TimeMonday, 14 November 20221:30pm - 2:15pm CST
LocationC140-142
W
Recorded
Presenter