Research output: Research - peer-review › Journal article

- Pankaj K. Agarwal Pankaj K. Agarwal Duke UniversityUnited States
- Graham Graham Graham Graham University of WarwickUnited Kingdom
- Zengfeng HuangZengfeng HuangDenmark
- Jeff PhillipsJeff PhillipsUniversity of UtahUnited States
- Zhewei WeiZhewei WeiDenmark
- Ke YiKe Yi Tsinghua University and Hong Kong University of Science and TechnologyChina

We study the mergeability of data summaries. Informally speaking, mergeability requires that, given two summaries on two datasets, there is a way to merge the two summaries into a single summary on the two datasets combined together, while preserving the error and size guarantees. This property means that the summaries can be merged in a way akin to other algebraic operators such as sum and max, which is especially useful for computing summaries on massive distributed data. Several data summaries are trivially mergeable by construction, most notably all the sketches that are linear functions of the datasets. But some other fundamental ones, like those for heavy hitters and quantiles, are not (known to be) mergeable. In this article, we demonstrate that these summaries are indeed mergeable or can be made mergeable after appropriate modifications. Specifically, we show that for ϵ-approximate heavy hitters, there is a deterministic mergeable summary of size O(1/ϵ); for ϵ-approximate quantiles, there is a deterministic summary of size O((1/ϵ) log(ϵ n)) that has a restricted form of mergeability, and a randomized one of size O((1/ϵ) log3/2(1/ϵ)) with full mergeability. We also extend our results to geometric summaries such as ϵ-approximations which permit approximate multidimensional range counting queries. While most of the results in this article are theoretical in nature, some of the algorithms are actually very simple and even perform better than the previously best known algorithms, which we demonstrate through experiments in a simulated sensor network.

We also achieve two results of independent interest: (1) we provide the best known randomized streaming bound for ϵ-approximate quantiles that depends only on ϵ, of size O((1/ϵ) log3/2(1/ϵ)), and (2) we demonstrate that the MG and the SpaceSaving summaries for heavy hitters are isomorphic

We also achieve two results of independent interest: (1) we provide the best known randomized streaming bound for ϵ-approximate quantiles that depends only on ϵ, of size O((1/ϵ) log3/2(1/ϵ)), and (2) we demonstrate that the MG and the SpaceSaving summaries for heavy hitters are isomorphic

Original language | English |
---|---|

Article number | 26 |

Journal | A C M Transactions on Database Systems |

Volume | 38 |

Issue number | 4 |

Number of pages | 28 |

ISSN | 0362-5915 |

DOIs | |

State | Published - 2013 |

See relations at Aarhus University Citationformats

ID: 68455336