Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Big Data Technologies pp 658–665Cite as

Delta Compression Techniques

  • Reference work entry
  • First Online:

  • 40 Accesses

Synonyms

Data differencing; Delta encoding; Differential compression

Definitions

Delta compression techniques encode a target file with respect to one or more reference files, such that a decoder who has access to the same reference files can recreate the target file from the compressed data. Delta compression is usually applied in cases where there is a high degree of redundancy between target and references files, leading to a much smaller compressed size than could be achieved by just compressing the target file by itself. Typical application scenarios include revision control systems and versioned file systems that store many versions of a file or software or content updates over networks where the recipient already has an older version of the data. Most work on delta compression techniques has focused on the case of textual and binary files, but the concept can also be applied to multimedia and structured data.

Delta compression should not be confused with Elias delta codes, a...

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   849.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   999.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Adler M, Mitzenmacher M (2001) Towards compressing web graphs. In: IEEE data compression conference

    Book  Google Scholar 

  • Agarwal R, Amalapuraru S, Jain S (2004) An approximation to the greedy algorithm for differential compression of very large files. In: IEEE data compression conference

    Book  Google Scholar 

  • Ajtai M, Burns R, Fagin R, Long D, Stockmeyer L (2002) Compactly encoding unstructured inputs with differential compression. J ACM 49(3):318–367

    Article  MathSciNet  MATH  Google Scholar 

  • Alakuijala J, Szabadka Z (2016) Rfc7932: Brotli compressed data format. Available at https://tools.ietf.org/html/rfc7932

  • Bagchi A, Bhargava A, Suel T (2006) Approximate maximum weighted branchings. Inf Process Lett 99(2): 54–58

    Article  MATH  Google Scholar 

  • Banga G, Douglis F, Rabinovich M (1997) Optimistic deltas for WWW latency reduction. In: USENIX annual technical conference

    Google Scholar 

  • Bentley J, McIlroy D (1999) Data compression using long common strings. In: IEEE data compression conference

    Book  MATH  Google Scholar 

  • Berliner B (1990) CVS II: Parallelizing software development. In: Winter 1990 USENIX conference

    Google Scholar 

  • Burrows M, Wheeler D (1994) A block-sorting lossless data compression algorithm. Technical report. 124, SRC. Digital Systems Research Center, Palo Alto

    Google Scholar 

  • Chan M, Woo T (1999) Cache-based compaction: a new technique for optimizing web transfer. In: INFOCOM conference

    Google Scholar 

  • Chang F, Dean J, Ghemawat S, Hsieh W, Wallach D, Burrows M, Chandra T, Fikes A, Gruber R (2006) Bigtable: a distributed storage system for structured data. In: Seventh symposium on operating system design and implementation

    Google Scholar 

  • Chen Y, Douglis F, Huang H, Vo K (2000) Topblend: an efficient implementation of HtmlDiff in Java. In: WebNet 2000 conference

    Google Scholar 

  • Douglis F, Haro A, Rabinovich M (1997) HPP: HTML macro-preprocessing to support dynamic document caching. In: USENIX symposium on internet technologies and systems

    Google Scholar 

  • Drago I, Bocchi E, Mellia M, Slatman H, Pras A (2013) Benchmarking personal cloud storage. In: Internet measurement conference

    Book  Google Scholar 

  • Ferragina P, Manzini G (2010) On compressing the textual web. In: ACM international conference on web search and data mining

    Book  Google Scholar 

  • Gailly J (2017) zlib compression library, version 1.2.11. Available at https://zlib.net

  • Housel B, Lindquist D (1996) WebExpress: a system for optimizing web browsing in a wireless environment. In: ACM conference on mobile computing and networking, pp 108–116

    Google Scholar 

  • Hunt J, Vo KP, Tichy W (1998) Delta algorithms: an empirical analysis. ACM Trans Softw Eng Methodol 7:192–213

    Article  Google Scholar 

  • Korn D, Vo KP (2002) Engineering a differencing and compression data format. In: USENIX annual technical conference, pp 219–228

    Google Scholar 

  • Kulkarni P, Douglis F, LaVoie J, Tracey JM (2014) Redundancy elimination within large collections of files. In: USENIX annual technical conference

    Google Scholar 

  • MacDonald J (2000) File system support for delta compression. MS thesis, University of California, Berkeley

    Google Scholar 

  • Mogul JC, Douglis F, Feldmann A, Krishnamurthy B (1997) Potential benefits of delta-encoding and data compression for HTTP. In: ACM SIGCOMM conference, pp 181–196

    Google Scholar 

  • Molfetas A, Wirth A, Zobel J (2014a) Scalability in recursively stored delta compressed collections of files. In: Second Australasian web conference

    Google Scholar 

  • Molfetas A, Wirth A, Zobel J (2014b) Using inter-file similarity to improve intra-file compression. In: IEEE international congress on big data

    Book  Google Scholar 

  • Motta G, Gustafson J, Chen S (2007) Differential compression of executable code. In: IEEE data compression conference

    Book  Google Scholar 

  • Nakanishi T, Shih H, Hisazumi K, Fukuda A (2013) A software update scheme by airwaves for automotve equipment. In: International conference on information, electronics, and vision

    Google Scholar 

  • Ouyang Z, Memon N, Suel T, Trendafilov D (2002) Cluster-based delta compression of a collection of files. In: Third international conference on web information systems engineering

    Book  Google Scholar 

  • Percival C (2006) Matching with mismatches and assorted applications. PhD thesis, University of Oxford

    Google Scholar 

  • Rochkind M (1975) The source code control system. IEEE Trans Softw Eng 1:364–370

    Article  Google Scholar 

  • Samteladze N, Christensen K (2012) Delta: delta encoding for less traffic for apps. In: IEEE conference on local computer networks

    Google Scholar 

  • Savant A, Suel T (2003) Server-friendly delta compression for efficient web access. In: 8th international workshop on web content caching and distribution

    Google Scholar 

  • Shilane P, Huang M, Wallace G, Hsu W (2012) WAN optimized replication of backup datasets using stream-informed delta compression. In: USENIX symposium on file and storage technologies

    Book  Google Scholar 

  • Tate S (1997) Band ordering in lossless compression of multispectral images. IEEE Trans Comput 46(45): 211–320

    MathSciNet  Google Scholar 

  • Tichy W (1984) The string-to-string correction problem with block moves. ACM Trans Comput Syst 2(4): 309–321

    Article  Google Scholar 

  • Tichy W (1985) RCS: a system for version control. Softw Pract Exp 15:637–654

    Article  Google Scholar 

  • Trendafilov D, Memon N, Suel T (2002) zdelta: a simple delta compression tool. Technical report. Polytechnic University, CIS Department

    Google Scholar 

  • Trendafilov D, NMemon, Suel T (2004) Compressing file collections with a TSP-based approach. Technical report TR-CIS-2004-02. Polytechnic University

    Google Scholar 

  • Tridgell A (2000) Efficient algorithms for sorting and synchronization. PhD thesis, Australian National University

    Google Scholar 

  • Wagner RA, Fisher MJ (1974) The string-to-string correction problem. J ACM 21(1):168–173

    Article  MathSciNet  MATH  Google Scholar 

  • Wang J, Guo Y, Huang B, Ma J, Mo Y (2008) Delta compression for information push services. In: International conference on advanced information networking and applications – workshops

    Book  Google Scholar 

  • Xia W, Jiang H, Feng D, Tian L (2014a) Combining deduplication and delta compression to achieve low-overhead data reduction on backup datasets. In: IEEE data compression conference

    Book  Google Scholar 

  • Xia W, Jiang H, Feng D, Tian L, Fu M, Zhou Y (2014b) Ddelta: a deduplication-inspired fast delta compression approach. Perform Eval 79:258–272

    Article  Google Scholar 

  • Xia W, Li C, Jiang H, Feng D, Hua Y, Qin L, Zhang Y (2015) Edelta: a word-enlarging based fast delta compression approach. In: USENIX workshop on hot topics in storage and file systems

    Google Scholar 

  • Xiao C, Bing B, Chang GK (2005) Delta compression for fast wireless internet downloads. In: IEEE GlobeCom

    Google Scholar 

  • Ziv J, Lempel A (1977) A universal algorithm for data compression. IEEE Trans Inf Theory 23(3):337–343

    Article  MathSciNet  MATH  Google Scholar 

  • Ziv J, Lempel A (1978) Compression of individual sequences via variable-rate coding. IEEE Trans Inf Theory 24(5):530–536

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Tandon School of Engineering, New York University, Brooklyn, NY, USA

    Torsten Suel

Authors
  1. Torsten Suel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Torsten Suel .

Editor information

Editors and Affiliations

  1. Institute of Computer Science, University of Tartu, Tartu, Estonia

    Sherif Sakr

  2. School of Information Technologies, Sydney University, Sydney, Australia

    Albert Y. Zomaya

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Suel, T. (2019). Delta Compression Techniques. In: Sakr, S., Zomaya, A.Y. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-77525-8_63

Download citation

Publish with us

Policies and ethics

Search

Navigation