MetaWBC: POSIX-Compliant Metadata Write-Back Caching for Distributed File Systems
DescriptionIn parallel and distributed file systems, caching can improve data performance and metadata operations. Currently, most distributed file systems adopt a write-back data cache for performance and a write-through metadata cache for simplifying consistency. However, with modern file systems scales and workloads, write-through metadata caching can impact overall file system performance, e.g., through lock contention and heavy RPC loads required for namespace synchronization and transaction serialization.

This paper proposes a novel metadata writeback caching (MetaWBC) mechanism to improve the performance of metadata operations in distributed environments. To achieve extreme metadata performance, we developed a fast, lightweight, and POSIX-compatible memory file system as a metadata cache. Further, we designed a file caching state machine and included other performance optimizations. We coupled MetaWBC with Lustre and evaluated that MetaWBC can outperform the native parallel file system by up to 8x for metadata-intensive benchmarks, and up to 7x for realistic workloads in throughput.
Event Type
TimeWednesday, 16 November 20224pm - 4:30pm CST
Registration Categories
File Systems and I/O
Session Formats
Back To Top Button