Faculty of Technical Sciences

In the era of exponential growth in data, compression of data is needed to limit the growth in the required storage space and supporting infrastructure. Recently, generalized deduplication has been proposed as a better way of performing data deduplication. We use Golomb Rice transform for providing generalized deduplication. With this setup, we evalute the performance of generalized deduplication for the input conditions modeled using binomial, geometric, poisson and uniform distributions. We derive the closed form expressions for pmf of the data after generalized deduplication. For all the input conditions, generalized deduplicaton transforms the input pmf into a new pmf that is highly suitable for deduplication, reduces size of the deduplication table, provides comparable compression gain for fewer number of input chunks.