RFC1951: DEFLATE Compressed Data Format Specification version 1.31996 RFC
- Author:
- P. Deutsch
Skip Abstract Section
Abstract
This specification defines a lossless compressed data format that compresses data using a combination of the LZ77 algorithm and Huffman coding, with efficiency comparable to the best currently available general-purpose compression methods. The data can be produced or consumed, even for an arbitrarily long sequentially presented input data stream, using only an a priori bounded amount of intermediate storage. The format can be implemented readily in a manner not covered by patents.
RFC Downloads
ZIP
Cited By
- Hahnfeld J, Blomer J and Kollegger T Parallel Writing of Nested Data in Columnar Formats Euro-Par 2024: Parallel Processing, (18-31)
- Weißenberger A and Schmidt B Massively Parallel Inverse Block-sorting Transforms for bzip2 Decompression on GPUs Proceedings of the 53rd International Conference on Parallel Processing, (856-865)
- Zillien S, Schmidbauer T, Kubek M, Keller J and Wendzel S Look What’s There! Utilizing the Internet’s Existing Data for Censorship Circumvention with OPPRESSION Proceedings of the 19th ACM Asia Conference on Computer and Communications Security, (80-95)
- Wei K, Amano H, Niwase R and Yamaguchi Y A data compressor for FPGA-based state vector quantum simulators Proceedings of the 14th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies, (63-70)
- Gao R, Li Z, Tan G and Li X BeeZip: Towards An Organized and Scalable Architecture for Data Compression Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, (133-148)
- Lascorz A, Mahmoud M, Zadeh A, Nikolic M, Ibrahim K, Giannoula C, Abdelhadi A and Moshovos A Atalanta: A Bit is Worth a “Thousand” Tensor Values Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, (85-102)
- Rifai M and Johnsson L (2023). VxH: A Systematic Determination of Efficient Hierarchical Voxel Structures, ACM Transactions on Spatial Algorithms and Systems, 10:1, (1-34), Online publication date: 31-Mar-2024.
- Zhang B, Tian J, Di S, Yu X, Feng Y, Liang X, Tao D and Cappello F FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing, (129-142)
- Knespel M and Brunst H Rapidgzip: Parallel Decompression and Seeking in Gzip Files Using Cache Prefetching Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing, (295-307)
- Zhang B, Tian J, Di S, Yu X, Swany M, Tao D and Cappello F GPULZ: Optimizing LZSS Lossless Compression for Multi-byte Data on Modern GPUs Proceedings of the 37th International Conference on Supercomputing, (348-359)
- Karandikar S, Udipi A, Choi J, Whangbo J, Zhao J, Kanev S, Lim E, Alakuijala J, Madduri V, Shao Y, Nikolic B, Asanovic K and Ranganathan P CDPU: Co-designing Compression and Decompression Processing Units for Hyperscale Systems Proceedings of the 50th Annual International Symposium on Computer Architecture, (1-17)
- Gao Z, Xie Y and Yu W Efficient and Effective Digital Waveform Compression for Large-scale Logic Simulation of Integrated Circuit Proceedings of the Great Lakes Symposium on VLSI 2023, (373-377)
- Liu Z, Zhang Y, Zhu Y, Zhang R, Yang T, Xie K, Wang S, Li T and Cui B (2023). TreeSensing: Linearly Compressing Sketches with Flexibility, Proceedings of the ACM on Management of Data, 1:1, (1-28), Online publication date: 26-May-2023.
- Moon T, Kim H and Hyun S (2022). Mutexion: Mutually Exclusive Compression System for Mitigating Compression Side-Channel Attacks, ACM Transactions on the Web, 16:4, (1-20), Online publication date: 30-Nov-2022.
- Aycock J, Ganesh S, Biittner K, Newell P and Therrien C The Sincerest Form of Flattery: Large-Scale Analysis of Code Re-Use in Atari 2600 Games Proceedings of the 17th International Conference on the Foundations of Digital Games, (1-10)
- May M (2022). Donag: Generating Efficient Patches and Diffs for Compressed Archives, ACM Transactions on Storage, 18:3, (1-41), Online publication date: 31-Aug-2022.
- Chiosa M, Maschi F, Müller I, Alonso G and May N (2022). Hardware acceleration of compression and encryption in SAP HANA, Proceedings of the VLDB Endowment, 15:12, (3277-3291), Online publication date: 1-Aug-2022.
- Gao R, Li X, Li Y, Wang X and Tan G MetaZip Proceedings of the 59th ACM/IEEE Design Automation Conference, (319-324)
- Soule N, Kalashian B, Rock C and Tomcho C Software Acoustic Modem for TAK Communications with Analog Radios at the Tactical Edge MILCOM 2021 - 2021 IEEE Military Communications Conference (MILCOM), (347-353)
- Wang J, Gu X and Yu X Early Analysis and Detection of Compressed Traffic Proceedings of the 2020 International Conference on Cyberspace Innovation of Advanced Technologies, (471-476)
- Vaucher S, Yazdani N, Felber P, Lucani D and Schiavoni V ZipLine Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies, (399-405)
- Müller J, Ising F, Mainka C, Mladenov V, Schinzel S and Schwenk J Office document security and privacy Proceedings of the 14th USENIX Conference on Offensive Technologies, (12-12)
- Göttel C, Nielsen L, Yazdani N, Felber P, Lucani D and Schiavoni V Hermes Proceedings of the 14th ACM International Conference on Distributed and Event-based Systems, (133-136)
- Ilkhechi A, Crotty A, Galakatos A, Mao Y, Fan G, Shi X and Cetintemel U DeepSqueeze: Deep Semantic Compression for Tabular Data Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, (1733-1746)
- Jiang H, Liu C, Jin Q, Paparrizos J and Elmore A (2020). PIDS, Proceedings of the VLDB Endowment, 13:6, (925-938), Online publication date: 1-Feb-2020.
- Monaco J What are you searching for? a remote keylogging attack on search engine autocomplete Proceedings of the 28th USENIX Conference on Security Symposium, (959-976)
- Fifield D A better zip bomb Proceedings of the 13th USENIX Conference on Offensive Technologies, (9-9)
- Calvo-Palomino R, Cordobés H, Ricciato F, Giustiniano D and Lenders V Collaborative wideband signal decoding using non-coherent receivers Proceedings of the 18th International Conference on Information Processing in Sensor Networks, (37-48)
- Singhal S, Sharma P, Aggarwal R and Passricha V (2018). A Global Survey on Data Deduplication, International Journal of Grid and High Performance Computing, 10:4, (43-66), Online publication date: 1-Oct-2018.
- Petersen D and Böhmer M Understanding mobile information supply Proceedings of the 10th Nordic Conference on Human-Computer Interaction, (195-202)
- Weißenberger A and Schmidt B Massively Parallel Huffman Decoding on GPUs Proceedings of the 47th International Conference on Parallel Processing, (1-10)
- Arimilli L, Blaner B, Drerup B, Marino C, Williams D, Lais E, Campisano F, Guthrie G, Floyd M, Leavens R, Willenborg S, Kalla R and Abali B (2018). IBM POWER9 processor and system features for computing in the cognitive era, IBM Journal of Research and Development, 62:4-5, (1:1-1:11), Online publication date: 1-Jul-2018.
- Ladra S, Param J and Silva-Coira F (2017). Scalable and queryable compressed storage structure for raster data, Information Systems, 72:C, (179-204), Online publication date: 1-Dec-2017.
- Agrawal A, Loh G and Tuck J Leveraging near data processing for high-performance checkpoint/restart Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, (1-12)
- Yamamoto K, Tsujikawa T and Oku K Exploring HTTP/2 Header Compression Proceedings of the 12th International Conference on Future Internet Technologies, (1-5)
- Lee S, Kim K, Koo G, Jeon H, Annavaram M and Ro W (2017). Improving Energy Efficiency of GPUs through Data Compression and Compressed Execution, IEEE Transactions on Computers, 66:5, (834-847), Online publication date: 1-May-2017.
- Horn D, Elkabany K, Lesniewski-Laas C and Winstein K The design, implementation, and deployment of a system to transparently compress hundreds of petabytes of image files for a file-storage service Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation, (1-15)
- Canet M, Kumar A, Lauradoux C, Rakotomanga M and Safavi-Naini R Decompression Quines and Anti-Viruses Proceedings of the Seventh ACM on Conference on Data and Application Security and Privacy, (23-34)
- Kong W, Wu J, Hu Z, Anisetti M, Damiani E and Jeon G (2017). Lossless compression for aurora spectral images using fast online bi-dimensional decorrelation method, Information Sciences: an International Journal, 381:C, (33-45), Online publication date: 1-Mar-2017.
- Wu R, Xiao X, Cheung S, Zhang H and Zhang C (2016). Casper: an efficient approach to call trace collection, ACM SIGPLAN Notices, 51:1, (678-690), Online publication date: 8-Apr-2016.
- Wu R, Xiao X, Cheung S, Zhang H and Zhang C Casper: an efficient approach to call trace collection Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, (678-690)
- Kuhn M, Kunkel J and Ludwig T (2016). Data Compression for Climate Data, Supercomputing Frontiers and Innovations: an International Journal, 3:1, (75-94), Online publication date: 1-Jan-2016.
- LaPre J, Gonsiorowski E, Carothers C, Jenkins J, Carns P and Ross R Time warp state restoration via delta encoding Proceedings of the 2015 Winter Simulation Conference, (3025-3036)
- Sultan A, Benbadis F, Gauthier V and Afifi H Mobile Data Network Analysis Platform Proceedings of the 6th International Workshop on Hot Topics in Planet-Scale Measurement, (13-18)
- Qasem M and Pustina L ITFComp Proceedings of the 8th International Conference on Simulation Tools and Techniques, (200-207)
- Pellegrino G, Balzarotti D, Winter S and Suri N In the compression hornet's nest Proceedings of the 24th USENIX Conference on Security Symposium, (801-816)
- Chłopkowski M and Walkowiak R (2015). A general purpose lossless data compression method for GPU, Journal of Parallel and Distributed Computing, 75:C, (40-52), Online publication date: 1-Jan-2015.
- Xiao T, Guo Z, Zhou H, Zhang J, Zhao X, Ye C, Wang X, Lin W, Chen W and Zhou L (2014). Cybertron, ACM SIGPLAN Notices, 49:10, (895-908), Online publication date: 31-Dec-2015.
- Zohar E and Cassuto Y Automatic and dynamic configuration of data compression for web servers Proceedings of the 28th USENIX conference on Large Installation System Administration, (97-108)
- Iglesias N, Bulacio P and Tapia E (2014). Improving ISO 11783 file transfers into mobile farm equipments using on-the-fly data compression, Computers and Electronics in Agriculture, 109:C, (101-108), Online publication date: 1-Nov-2014.
- Xiao T, Guo Z, Zhou H, Zhang J, Zhao X, Ye C, Wang X, Lin W, Chen W and Zhou L Cybertron Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, (895-908)
- Crochemore M, Langiu A and Mignosi F (2014). Note on the greedy parsing optimality for dictionary-based text compression, Theoretical Computer Science, 525, (55-59), Online publication date: 1-Mar-2014.
- Lin X, Lu G, Douglis F, Shilane P and Wallace G Migratory compression Proceedings of the 12th USENIX conference on File and Storage Technologies, (257-271)
- Chandra S, Boreczky J and Rowe L (2014). High performance many-to-many intranet screen sharing with DisplayCast, ACM Transactions on Multimedia Computing, Communications, and Applications, 10:2, (1-22), Online publication date: 1-Feb-2014.
- Phokas T, Efstathiades H, Pallis G and Dikaiakos M Feel the World Proceedings of the 10th International Conference on Mobile Web Information Systems - Volume 8093, (143-156)
- Cicirello V and Regli W (2013). A flexible and extensible approach to automated CAD/CAM format classification, Computers and Graphics, 37:5, (484-495), Online publication date: 1-Aug-2013.
- Golander A, Taharlev S, Glass L, Biran G and Manole S Leveraging predefined huffman dictionaries for high compression rate and ratio Proceedings of the 6th International Systems and Storage Conference, (1-6)
- Qian F, Huang J, Erman J, Mao Z, Sen S and Spatscheck O How to reduce smartphone traffic volume by 30%? Proceedings of the 14th international conference on Passive and Active Measurement, (42-52)
- Garfinkel S (2013). Digital media triage with bulk data analysis and bulk_extractor, Computers and Security, 32:C, (56-72), Online publication date: 1-Feb-2013.
- Bernas T, Starosolski R, Robinson J and Rajwa B (2012). Application of detector precision characteristics and histogram packing for compression of biological fluorescence micrographs, Computer Methods and Programs in Biomedicine, 108:2, (511-523), Online publication date: 1-Nov-2012.
- Chandra S, Biehl J, Boreczky J, Carter S and Rowe L Understanding screen contents for building a high performance, real time screen sharing system Proceedings of the 20th ACM international conference on Multimedia, (389-398)
- Ali F, Feaster Y, Wahba S and Hallstrom J A metadata encoding for memory-constrained devices Proceedings of the 49th annual ACM Southeast Conference, (191-196)
- Colantonio A and Di Pietro R (2010). Concise, Information Processing Letters, 110:16, (644-650), Online publication date: 1-Jul-2010.
- Otten F, Irwin B and Thinyane H Evaluating text preprocessing to improve compression on maillogs Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, (44-53)
- Yu H, Franke H, Biran G, Golander A, Nelms T and Bass B Stateful hardware decompression in networking environment Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems, (141-150)
- Ericsson M (2007). The Effects of XML Compression on SOAP Performance, World Wide Web, 10:3, (279-307), Online publication date: 1-Sep-2007.
- Newton A (2006). Replacing the Whois Protocol, IEEE Internet Computing, 10:4, (79-84), Online publication date: 1-Jul-2006.
- Mathiak B, Kupfer A, Münch R, Täubner C and Eckstein S Improving literature preselection by searching for images Proceedings of the 2006 international conference on Knowledge Discovery in Life Science Literature, (18-28)
- Paunov S and Schmidt D RepoMan Proceedings of the 44th annual ACM Southeast Conference, (714-419)
- Manabe D, Kimura S and Ebihara Y A compression method designed for SMTP over TLS Proceedings of the 2006 international conference on Information Networking: advances in Data Communications and Wireless Networks, (803-812)
- Kriegel H, Kunath P, Pfeifle M and Renz M Distributed intersection join of complex interval sequences Proceedings of the 10th international conference on Database Systems for Advanced Applications, (748-760)
- Kohno T Attacking and repairing the winZip encryption scheme Proceedings of the 11th ACM conference on Computer and communications security, (72-81)
- Nakar D and Weiss S Selective main memory compression by identifying program phase changes Proceedings of the 3rd workshop on Memory performance issues: in conjunction with the 31st international symposium on computer architecture, (96-101)
- Phelps T and Wilensky R Two diet plans for fat PDF Proceedings of the 2003 ACM symposium on Document engineering, (175-184)
- Danielsson M and Müller R A time-evolving data structure scalable between discrete and continuous attribute modifications Computer Science in Perspective, (98-114)
- Korn D and Vo K Engineering a Differencing and Compression Data Format Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference, (219-228)
- Sundaresan N and Moussa R Algorithms and programming models for efficient representation of XML for Internet applications Proceedings of the 10th international conference on World Wide Web, (366-375)
- Hovemeyer D and Pugh W More efficient network class loading through bundling Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium - Volume 1, (17-17)
- Factor M, Sheinwald D and Yassour B Software Compression in the Client/Server Environment Proceedings of the Data Compression Conference
- Chi C, Deng J and Lim Y Compression proxy server Proceedings of the 2nd conference on USENIX Symposium on Internet Technologies and Systems - Volume 2, (10-10)
- Santos J and Wetherall D Increasing effective link bandwidth by suppressing replicated data Proceedings of the annual conference on USENIX Annual Technical Conference, (18-18)
- Nielsen H, Gettys J, Baird-Smith A, Prud'hommeaux E, Lie H and Lilley C (1997). Network performance effects of HTTP/1.1, CSS1, and PNG, ACM SIGCOMM Computer Communication Review, 27:4, (155-166), Online publication date: 1-Oct-1997.
- Nielsen H, Gettys J, Baird-Smith A, Prud'hommeaux E, Lie H and Lilley C Network performance effects of HTTP/1.1, CSS1, and PNG Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, (155-166)
Please enable JavaScript to view thecomments powered by Disqus.
Recommendations
A universal compressed data format for foreign file systems
DCC '95: Proceedings of the Conference on Data CompressionThe authors have proposed a compressed data format that can be used with any foreign file system and that allows users to access data randomly in a compressed file without entirely decompressing it. Since the compressed file includes all information ...