Skip to main content

Delta Compression Techniques

  • Living reference work entry
  • First Online:

Synonyms

Data differencing; Delta encoding; Differential compression

Definition

Delta compression techniques encode a target file with respect to one or more reference files, such that a decoder who has access to the same reference files can recreate the target file from the compressed data. Delta compression is usually applied in cases where there is a high degree of redundancy between target and references files, leading to a much smaller compressed size than could be achieved by just compressing the target file by itself. Typical application scenarios include revision control systems and versioned file systems that store many versions of a file or software or content updates over networks where the recipient already has an older version of the data. Most work on delta compression techniques has focused on the case of textual and binary files, but the concept can also be applied to multimedia and structured data.

Delta compression should not be confused with Elias delta codes, a...

This is a preview of subscription content, log in via an institution.

References

  • Adler M, Mitzenmacher M (2001) Towards compressing web graphs. In: IEEE data compression conference

    Google Scholar 

  • Agarwal R, Amalapuraru S, Jain S (2004) An approximation to the greedy algorithm for differential compression of very large files. In: IEEE data compression conference

    Google Scholar 

  • Ajtai M, Burns R, Fagin R, Long D, Stockmeyer L (2002) Compactly encoding unstructured inputs with differential compression. J ACM 49(3):318–367

    Article  MathSciNet  Google Scholar 

  • Alakuijala J, Szabadka Z (2016) Rfc7932: Brotli compressed data format. Available at https://tools.ietf.org/html/rfc7932

  • Bagchi A, Bhargava A, Suel T (2006) Approximate maximum weighted branchings. Inf Process Lett 99(2): 54–58

    Google Scholar 

  • Banga G, Douglis F, Rabinovich M (1997) Optimistic deltas for WWW latency reduction. In: USENIX annual technical conference

    Google Scholar 

  • Bentley J, McIlroy D (1999) Data compression using long common strings. In: IEEE data compression conference

    Google Scholar 

  • Berliner B (1990) CVS II: Parallelizing software development. In: Winter 1990 USENIX conference

    Google Scholar 

  • Burrows M, Wheeler D (1994) A block-sorting lossless data compression algorithm. Technical report. 124, SRC. Digital Systems Research Center, Palo Alto

    Google Scholar 

  • Chan M, Woo T (1999) Cache-based compaction: a new technique for optimizing web transfer. In: INFOCOM conference

    Google Scholar 

  • Chang F, Dean J, Ghemawat S, Hsieh W, Wallach D, Burrows M, Chandra T, Fikes A, Gruber R (2006) Bigtable: a distributed storage system for structured data. In: Seventh symposium on operating system design and implementation

    Google Scholar 

  • Chen Y, Douglis F, Huang H, Vo K (2000) Topblend: an efficient implementation of HtmlDiff in Java. In: WebNet 2000 conference

    Google Scholar 

  • Douglis F, Haro A, Rabinovich M (1997) HPP: HTML macro-preprocessing to support dynamic document caching. In: USENIX symposium on internet technologies and systems

    Google Scholar 

  • Drago I, Bocchi E, Mellia M, Slatman H, Pras A (2013) Benchmarking personal cloud storage. In: Internet measurement conference

    Google Scholar 

  • Ferragina P, Manzini G (2010) On compressing the textual web. In: ACM international conference on web search and data mining

    Google Scholar 

  • Gailly J (2017) zlib compression library, version 1.2.11. Available at https://zlib.net

  • Housel B, Lindquist D (1996) WebExpress: a system for optimizing web browsing in a wireless environment. In: ACM conference on mobile computing and networking, pp 108–116

    Google Scholar 

  • Hunt J, Vo KP, Tichy W (1998) Delta algorithms: an empirical analysis. ACM Trans Softw Eng Methodol 7:192–213

    Article  Google Scholar 

  • Korn D, Vo KP (2002) Engineering a differencing and compression data format. In: USENIX annual technical conference, pp 219–228

    Google Scholar 

  • Kulkarni P, Douglis F, LaVoie J, Tracey JM (2014) Redundancy elimination within large collections of files. In: USENIX annual technical conference

    Google Scholar 

  • MacDonald J (2000) File system support for delta compression. MS thesis, University of California, Berkeley

    Google Scholar 

  • Mogul JC, Douglis F, Feldmann A, Krishnamurthy B (1997) Potential benefits of delta-encoding and data compression for HTTP. In: ACM SIGCOMM conference, pp 181–196

    Article  Google Scholar 

  • Molfetas A, Wirth A, Zobel J (2014a) Scalability in recursively stored delta compressed collections of files. In: Second Australasian web conference

    Google Scholar 

  • Molfetas A, Wirth A, Zobel J (2014b) Using inter-file similarity to improve intra-file compression. In: IEEE international congress on big data

    Google Scholar 

  • Motta G, Gustafson J, Chen S (2007) Differential compression of executable code. In: IEEE data compression conference

    Google Scholar 

  • Nakanishi T, Shih H, Hisazumi K, Fukuda A (2013) A software update scheme by airwaves for automotve equipment. In: International conference on information, electronics, and vision

    Google Scholar 

  • Ouyang Z, Memon N, Suel T, Trendafilov D (2002) Cluster-based delta compression of a collection of files. In: Third international conference on web information systems engineering

    Google Scholar 

  • Percival C (2006) Matching with mismatches and assorted applications. PhD thesis, University of Oxford

    Google Scholar 

  • Rochkind M (1975) The source code control system. IEEE Trans Softw Eng 1:364–370

    Article  Google Scholar 

  • Samteladze N, Christensen K (2012) Delta: delta encoding for less traffic for apps. In: IEEE conference on local computer networks

    Google Scholar 

  • Savant A, Suel T (2003) Server-friendly delta compression for efficient web access. In: 8th international workshop on web content caching and distribution

    Google Scholar 

  • Shilane P, Huang M, Wallace G, Hsu W (2012) WAN optimized replication of backup datasets using stream-informed delta compression. In: USENIX symposium on file and storage technologies

    Article  Google Scholar 

  • Tate S (1997) Band ordering in lossless compression of multispectral images. IEEE Trans Comput 46(45): 211–320

    Article  MathSciNet  Google Scholar 

  • Tichy W (1984) The string-to-string correction problem with block moves. ACM Trans Comput Syst 2(4): 309–321

    Article  Google Scholar 

  • Tichy W (1985) RCS: a system for version control. Softw Pract Exp 15:637–654

    Google Scholar 

  • Trendafilov D, Memon N, Suel T (2002) zdelta: a simple delta compression tool. Technical report. Polytechnic University, CIS Department

    Google Scholar 

  • Trendafilov D, NMemon, Suel T (2004) Compressing file collections with a TSP-based approach. Technical report TR-CIS-2004-02. Polytechnic University

    Google Scholar 

  • Tridgell A (2000) Efficient algorithms for sorting and synchronization. PhD thesis, Australian National University

    Google Scholar 

  • Wagner RA, Fisher MJ (1974) The string-to-string correction problem. J ACM 21(1):168–173

    Article  MathSciNet  Google Scholar 

  • Wang J, Guo Y, Huang B, Ma J, Mo Y (2008) Delta compression for information push services. In: International conference on advanced information networking and applications – workshops

    Google Scholar 

  • Xia W, Jiang H, Feng D, Tian L (2014a) Combining deduplication and delta compression to achieve low-overhead data reduction on backup datasets. In: IEEE data compression conference

    Google Scholar 

  • Xia W, Jiang H, Feng D, Tian L, Fu M, Zhou Y (2014b) Ddelta: a deduplication-inspired fast delta compression approach. Perform Eval 79:258–272

    Article  Google Scholar 

  • Xia W, Li C, Jiang H, Feng D, Hua Y, Qin L, Zhang Y (2015) Edelta: a word-enlarging based fast delta compression approach. In: USENIX workshop on hot topics in storage and file systems

    Google Scholar 

  • Xiao C, Bing B, Chang GK (2005) Delta compression for fast wireless internet downloads. In: IEEE GlobeCom

    Google Scholar 

  • Ziv J, Lempel A (1977) A universal algorithm for data compression. IEEE Trans Inf Theory 23(3):337–343

    Article  MathSciNet  Google Scholar 

  • Ziv J, Lempel A (1978) Compression of individual sequences via variable-rate coding. IEEE Trans Inf Theory 24(5):530–536

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Torsten Suel .

Editor information

Editors and Affiliations

Section Editor information

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Suel, T. (2018). Delta Compression Techniques. In: Sakr, S., Zomaya, A. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-63962-8_63-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63962-8_63-1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63962-8

  • Online ISBN: 978-3-319-63962-8

  • eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering

Publish with us

Policies and ethics