Featured Talk: DAOS – Nextgen Storage Stack for HPC and AI
DescriptionDAOS is an open-source scale-out object store designed from the ground up to deliver extremely high bandwidth/IOPS and low latency I/Os to the most demanding data-intensive workloads. It aims at supporting nextgen scientific workflows combining simulation, big data and AI in a single storage tier. DAOS presents a rich and scalable storage interface that allows efficient storage of both structured and unstructured data. DAOS supports multiple application interfaces including a parallel filesystem, Hadoop/Spark connector, TensorFlow-IO, native Python bindings, HDF5, MPI-IO as well as domain-specific data models like SEGY. Many DAOS deployments are underway including a 230PB installation connected to the ALCF’s Aurora system and a 1PB DAOS system for LRZ’s SuperMUC-NG phase 2. In this presentation, we will provide an overview of the DAOS architecture, the software ecosystem, and the Aurora deployment.
Event Type
Workshop
TimeMonday, 14 November 20228:35am - 9:15am CST
LocationC143-149
W
Reliability and Resiliency
Recorded
Links
Presenter
