Lossless Compression of Time Series Data with Generalized Deduplication

    Research output: Contribution to book/anthology/report/proceedingArticle in proceedingsResearchpeer-review

    308 Downloads (Pure)


    To provide compressed storage for large amounts of time series data, we present a new strategy for data deduplication. Rather than attempting to deduplicate entire data chunks, we employ a generalized approach, where each chunk is split into a part worth deduplicating and a part that must be stored directly. This simple principle enables a greater compression of the often similar, non-identical, chunks of time series data than is the case for classic deduplication, while keeping benefits such as scalability, robustness, and on-the-fly storage, retrieval, and search for chunks. We analyze the method's theoretical performance, and argue that our method can asymptotically approach the entropy limit for some data configurations. To validate the method's practical merits, we finally show that it is competitive when compared to popular universal compression algorithms on the MIT-BIH ECG Compression Test Database.
    Original languageEnglish
    Title of host publication2019 IEEE Global Communications Conference, GLOBECOM 2019 - Proceedings
    Publication date2019
    Article number9013957
    ISBN (Electronic)978-1-7281-0962-6
    Publication statusPublished - 2019
    Event2019 IEEE Global Communications Conference - Waikoloa, HI, USA, Waikoloa, United States
    Duration: 9 Dec 201913 Dec 2019


    Conference2019 IEEE Global Communications Conference
    LocationWaikoloa, HI, USA
    Country/TerritoryUnited States
    Internet address


    • Data compression
    • Deduplication
    • Storage
    • Time Series Data


    Dive into the research topics of 'Lossless Compression of Time Series Data with Generalized Deduplication'. Together they form a unique fingerprint.
    • Scale-loT

      Lucani Rötter, D. E. (Participant)


      Project: Research

    • Starting Grant

      Lucani Rötter, D. E. (Participant)

      Starting Grant


      Project: Research

    Cite this