Extreme Bitmapping: Efficiently Sorting Data for Cloud Compression

TY - GEN

T1 - Extreme Bitmapping: Efficiently Sorting Data for Cloud Compression

AU - Mørup, Christian

AU - Kloborg, Anders

AU - Lucani Rötter, Daniel Enrique

PY - 2022

Y1 - 2022

N2 - This paper proposes Extreme Bitmapping, a new algorithm to concisely represent how a sorted sequence can be restored to its original, unsorted sequence. This concise representation is critical to the compression efficiency of newly proposed data compression and dual deduplication algorithms, e.g., Bonsai [1], but with potential implications in other fields. Extreme Bitmapping in the context of data compression leads to improved compression ratios while preserving the privacy-focused features of dual deduplication. Performance measurements on three datasets, including, HDFS log files, high-resolution TIF images, VDI images of different Linux distributions, show significant compression gains compared to Bonsai (up to three fold better compression) or similar state-of-the-art alternatives.

AB - This paper proposes Extreme Bitmapping, a new algorithm to concisely represent how a sorted sequence can be restored to its original, unsorted sequence. This concise representation is critical to the compression efficiency of newly proposed data compression and dual deduplication algorithms, e.g., Bonsai [1], but with potential implications in other fields. Extreme Bitmapping in the context of data compression leads to improved compression ratios while preserving the privacy-focused features of dual deduplication. Performance measurements on three datasets, including, HDFS log files, high-resolution TIF images, VDI images of different Linux distributions, show significant compression gains compared to Bonsai (up to three fold better compression) or similar state-of-the-art alternatives.

KW - Cloud Storage

KW - Compression

KW - Deduplication

UR - https://www.scopus.com/pages/publications/85146117480

U2 - 10.1109/CloudNet55617.2022.9978780

DO - 10.1109/CloudNet55617.2022.9978780

M3 - Article in proceedings

SN - 978-1-6654-8628-6

T3 - IEEE International Conference on Cloud Networking

SP - 247

EP - 251

BT - 2022 IEEE 11th International Conference on Cloud Networking (CloudNet)

PB - IEEE

ER -