An Approximate L ^p-Difference Algorithm for Massive Data Streams

Abstract

Several recent papers have shown how to approximate the difference Σ_i |a _i − b _i| or Σ |a _i − b _i|² between two functions, when the function values a _i and b _i are given in a data stream, and their order is chosen by an adversary. These algorithms use little space (much less than would be needed to store the entire stream) and little time to process each item in the stream and give approximations with small relative error. Using different techniques, we show how to approximate the L ^p-difference Σ_i |a _i − b _i|^p for any rational-valued p ∈ (0,2], with comparable efficiency and error. We also show how to approximate Σ_i |a _i − b _i|^p for larger values of p but with a worse error guarantee. These results can be used to assess the difference between two chronologically or physically separated massive data sets, making one quick pass over each data set, without buffering the data or requiring the data source to pause.

Part of this work was done while the first author was visiting AT&T Labs.

An expanded version of this paper is available in preprint form at http://www.research.att.com/~mstrauss/pubs/lp.ps