MinervaFS: A User-Space File System for Generalised Deduplication

TY - GEN

T1 - MinervaFS: A User-Space File System for Generalised Deduplication

AU - Nielsen, Lars

AU - Burihabwa, Dorian

AU - Schiavoni, Valerio

AU - Felber, Pascal

AU - Lucani Rötter, Daniel Enrique

N1 - Conference code: 40

PY - 2021

Y1 - 2021

N2 - Deduplication exploits the presence of similar data chunks to reduce storage overhead. Generalised deduplication (GD) uses transformation functions to split data into a basis (common to millions of chunks) and a deviation with respect to the basis. Doing so, it avoids computing additional hashes, comparing or differentiatingagainst previously stored chunks. MINERVAFS is the first FUSE-based file system for GD. We implement and evaluate it using several real-world datasets, e.g., satellite images and virtual machine images, comparing against classical deduplication approaches (ZFS, SDFS), delta compression (xdelta) or compression (Gzip). Compared to ZFS, MINERVAFS achieves up to 63.53% (average of 27.38%) saving in storage usage and a speedup of 16% in read-heavy workloads. For VM images, MINERVAFS’s data compression is on par with Gzip, while outperforming ZFS by severalfold. In contrast to ZFS’ growing RAM costs when more data is stored, MinervaFS’ RAM usage is independent from the amount of data stored, making it well suited to handle growing storage demands.

AB - Deduplication exploits the presence of similar data chunks to reduce storage overhead. Generalised deduplication (GD) uses transformation functions to split data into a basis (common to millions of chunks) and a deviation with respect to the basis. Doing so, it avoids computing additional hashes, comparing or differentiatingagainst previously stored chunks. MINERVAFS is the first FUSE-based file system for GD. We implement and evaluate it using several real-world datasets, e.g., satellite images and virtual machine images, comparing against classical deduplication approaches (ZFS, SDFS), delta compression (xdelta) or compression (Gzip). Compared to ZFS, MINERVAFS achieves up to 63.53% (average of 27.38%) saving in storage usage and a speedup of 16% in read-heavy workloads. For VM images, MINERVAFS’s data compression is on par with Gzip, while outperforming ZFS by severalfold. In contrast to ZFS’ growing RAM costs when more data is stored, MinervaFS’ RAM usage is independent from the amount of data stored, making it well suited to handle growing storage demands.

U2 - 10.1109/SRDS53918.2021.00033

DO - 10.1109/SRDS53918.2021.00033

M3 - Article in proceedings

BT - International Symposium on Reliable Distributed Systems (SRDS)

PB - IEEE

T2 - International Symposium on Reliable Distributed Systems

Y2 - 23 September 2021 through 23 September 2021

ER -