ROSS – Closing Invited Talk: Federated Function-as-a-Service to Power Distributed Computing Pipelines
DescriptionI report on experiences developing and deploying the funcX distributed function as a service (FaaS) platform and in employing this platform to support distributed computing pipelines that link instruments, computers (e.g., for analysis, simulation, AI model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. Both funcX and the Globus Flows system used to implement these pipelines combine cloud-hosted management, for reliability, with edge-hosted execution, for flexible and scalable execution. I discuss, in particular, the funcX and Globus Flows architectures; the container management strategies used in funcX to execute functions with high performance and efficiency on diverse funcX endpoints; and funcX’s integration with an in-memory data store and Globus for managing data that spans endpoints.
TimeSunday, 13 November 20224:20pm - 4:55pm CST