ADIOS-2: A Framework to Enable HPC Tools for Extreme Scale I/O, in situ Visualization, and Performance Analysis
DescriptionAs concurrency and complexity continue to increase on high-end machines, storage I/O performance is rapidly becoming a fundamental challenge to scientific discovery. At the exascale, online analysis will become a dominant form of data analytics, and thus scalable in situ workflows will become critical, along with high performance I/O to storage. The many components of a workflow running simultaneously pose another challenge of evaluating and improving the performance of these workflows. Therefore, performance data collection needs to be an integral part of the entire workflow.

In this tutorial, we present ADIOS-2 which allows for building in situ and file-based data processing workflows for extreme scale systems, including interactive, on-demand, in situ visualization of the data, and including performance profiling of the entire workflow. Half of this tutorial will be hands-on sessions, where we provide access to the software, and build together a complete MiniApp with in situ analytics and performance analysis that users can run on their laptop and supercomputers at large scale. We will show how ADIOS-2 is fully integrated into three popular visualization and performance tools: Jupyter Notebook, ParaView, and TAU, creating a software ecosystem for in situ processing of both performance and scientific data.
Event Type
Tutorial
TimeMonday, 14 November 20228:30am - 5pm CST
LocationD170
Registration Categories
TUT
Tags
Big Data
Cloud and Distributed Computing
Data Analytics
Data Mangement
Emerging Technologies
Exascale Computing
File Systems and I/O
In Situ Processing
Performance
Productivity Tools
Reliability and Resiliency
Resource Management and Scheduling
Software Engineering
Visualization
Session Formats
Recorded
Session Evaluation give feedback
Back To Top Button