SC22 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Workshops Archive

ROSS – Closing Invited Talk: Federated Function-as-a-Service to Power Distributed Computing Pipelines


Workshop: 12th International Workshop on Runtime and Operating Systems for Supercomputers (ROSS)

Authors: Ian Foster (Argonne National Laboratory (ANL), University of Chicago)


Abstract: I report on experiences developing and deploying the funcX distributed function as a service (FaaS) platform and in employing this platform to support distributed computing pipelines that link instruments, computers (e.g., for analysis, simulation, AI model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. Both funcX and the Globus Flows system used to implement these pipelines combine cloud-hosted management, for reliability, with edge-hosted execution, for flexible and scalable execution. I discuss, in particular, the funcX and Globus Flows architectures; the container management strategies used in funcX to execute functions with high performance and efficiency on diverse funcX endpoints; and funcX’s integration with an in-memory data store and Globus for managing data that spans endpoints.


Website:






Back to 12th International Workshop on Runtime and Operating Systems for Supercomputers (ROSS) Archive Listing



Back to Full Workshop Archive Listing