Toward Efficient Oversubscription: On the Cost and Benefit of Event-Based Communication in MPI
DescriptionContemporary HPC systems run compute jobs on exclusively assigned hardware resources. During communication, polling for progress is used for minimal latency. Previous work on oversubscription and event-based communication shows these techniques can improve overall system utilization and reduce energy consumption. Despite these findings, neither of the techniques is commonly used in HPC systems today. We believe that the current lack of detailed studies of the low-level effects of event-based communication, a key enabler of efficient oversubscription, is a major obstacle to a wider adoption.
We demonstrate that the sched_yield system call, often used for oversubscription, is not best suited for this purpose on modern Linux systems. Furthermore, we incorporate event-based communication into Open MPI and evaluate the effects on latency and energy consumption using an MPI micro-benchmark. Our results indicate that event-base communication incurs significant latency overhead but also saves energy. Both effects grow with the imbalance of the application.
We demonstrate that the sched_yield system call, often used for oversubscription, is not best suited for this purpose on modern Linux systems. Furthermore, we incorporate event-based communication into Open MPI and evaluate the effects on latency and energy consumption using an MPI micro-benchmark. Our results indicate that event-base communication incurs significant latency overhead but also saves energy. Both effects grow with the imbalance of the application.
Event Type
Workshop
TimeSunday, 13 November 20222:35pm - 3pm CST
LocationD221
W
Runtime Systems
System Software
Recorded