The Complete Encyclopedia of File Compression: How ZIP, 7z, RAR, gzip and Zstandard Work

Every day you use file compression without thinking about it: the ZIP file in an email, the app update on your phone, the web page that loads in a second, the Word document you save. Compression makes data smaller so it takes less space and moves faster. Decompression (sometimes called uncompression or extraction) turns it back into the original.

This encyclopedia covers the whole subject: how compression works, the key techniques, the history of the famous formats, what each format is used for, open-source and closed-source archivers, security pitfalls, and a real benchmark. You can try the basics with our free in-browser ZIP Compressor and UnZIP Files tools.

Compression vs. archiving

Two different jobs are often mixed up:

• Archiving bundles many files and folders into one file, keeping names, folder structure and dates. The Unix tar format does only this; it does not make anything smaller.
• Compression makes data smaller by removing redundancy. gzip, for example, compresses a single stream of data and has no idea about folders.
• Many formats do both. ZIP, 7z and RAR bundle files and compress them. On Linux the two jobs are often chained: tar bundles, then gzip or xz compresses, giving .tar.gz or .tar.xz.

Lossless vs. lossy

Lossless compression rebuilds the original exactly, bit for bit. It is the only kind used for archives, documents, software and data, because changing a single byte of a program or a spreadsheet could break it. Every format in this encyclopedia is lossless.

Lossy compression throws away detail people are unlikely to notice: JPEG and WebP for photos, MP3 and AAC for audio, H.264 and AV1 for video. It achieves far smaller files, but the original can never be recovered. Files that are already lossy-compressed barely shrink further when you put them in a ZIP. Our guide to compressing PDFs shows both kinds in action.

How lossless compression works

In 1948 Claude Shannon founded information theory and defined entropy, a measure of how unpredictable data is. Compression works by finding predictability and describing it in fewer bits. Text, code and spreadsheets are full of repetition, so they compress well. Random data has no pattern to exploit, so it cannot be compressed at all. Simple counting shows why: there are fewer short files than long ones, so no method can shrink every possible input.

Almost every practical compressor combines a few building blocks:

Run-length encoding (RLE)

The simplest idea: replace a run of repeated values with the value and a count, so AAAAAAB becomes "6 × A, 1 × B". It works well for simple graphics with large flat areas and was used in early image formats and fax machines. On ordinary text it does little.

Huffman coding

Invented by David Huffman in 1952 as an MIT student, Huffman coding gives common symbols short bit patterns and rare symbols long ones, in a way that can always be decoded unambiguously. It is still a component of ZIP, gzip, PNG, JPEG, Brotli and Zstandard.

Arithmetic coding and ANS

Arithmetic coding, developed in the 1970s, squeezes closer to the theoretical limit than Huffman by encoding a whole message as a single number. Its newer relative, asymmetric numeral systems (ANS), introduced by Jarek Duda in the late 2000s, offers similar efficiency at much higher speed. Zstandard uses an ANS-based coder.

Dictionary methods: LZ77, LZ78 and LZW

In 1977 and 1978 Abraham Lempel and Jacob Ziv published LZ77 and LZ78, the basis of most compressors in use today. LZ77 replaces a repeated piece of data with a short pointer back to where it appeared before ("copy 12 bytes from 300 bytes ago"). In 1984 Terry Welch published LZW, a variant of LZ78 used by the Unix compress command and the GIF image format.

Burrows–Wheeler transform (BWT)

Published by Michael Burrows and David Wheeler in 1994, the BWT rearranges data so that similar characters end up next to each other, which simpler coders can then compress very well. It is the heart of bzip2.

Context modelling (PPM)

Prediction by partial matching (PPM) predicts each next symbol from the symbols before it, then encodes how surprising it was. It gives excellent results on text and is offered by RAR and 7-Zip as the PPMd method.

How decompression works

Decompression replays the compressor's description of the data. It reads each pointer, count or code and writes out the original bytes. Because all the hard work of finding patterns happened during compression, decompression is usually much faster than compression. That is why a file compressed slowly at the highest level can still be opened quickly.

A short history of compression formats

• 1979: tar. The tape archive format of Unix (Version 7) bundles files, but does not compress them.
• 1985: ARC. System Enhancement Associates' ARC became popular on PCs and bulletin-board systems. After a 1988 legal dispute with SEA, Phil Katz of PKWARE created a new format.
• 1989: ZIP. Katz released PKZIP and published the ZIP file specification openly, so anyone could support it. That openness made ZIP the universal archive format.
• 1992: gzip. Jean-loup Gailly and Mark Adler wrote gzip for the GNU project as a free replacement for Unix compress, whose LZW algorithm was covered by patents.
• 1993: RAR. Eugene Roshal created the RAR format; WinRAR followed in 1995.
• 1993: DEFLATE. PKZIP 2 introduced DEFLATE (LZ77 plus Huffman coding), later documented as an internet standard (RFC 1951). It is still the most widely used compression method in the world.
• 1996: bzip2 and PNG. Julian Seward released bzip2. The PNG image format, built on DEFLATE, was created partly because of the LZW patent dispute over GIF.
• 1999: 7-Zip. Igor Pavlov released 7-Zip and its 7z format with the powerful LZMA algorithm, as open source.
• 2009: xz. XZ Utils brought LZMA2 to Linux; many distributions adopted .tar.xz.
• 2011: LZ4 and Snappy. Speed-first compressors arrived, from Yann Collet (LZ4) and Google (Snappy).
• 2015–2016: Brotli and Zstandard. Google released Brotli for the web, and Facebook released Yann Collet's Zstandard, which balances speed and ratio better than anything before it.
• 2023: Windows catches up. Windows has opened ZIP files natively since the Windows XP era. In 2023 Windows 11 added built-in support for opening RAR, 7z, tar and other formats.

Every major format, and what it is used for

ZIP

Uses: sharing files by email or download, sending folders to clients, backups, and the built-in "send to compressed folder" in Windows and macOS. ZIP is also hidden inside many everyday formats: Word, Excel and PowerPoint files (.docx, .xlsx, .pptx), OpenDocument files, EPUB e-books, Java JAR files and Android APK apps are all ZIP archives with a particular structure. Strengths: opens everywhere, and each file is compressed separately, so one file can be extracted quickly. Weaknesses: modest compression ratio, and the old "ZipCrypto" password protection is weak (see Security).

gzip (.gz) and .tar.gz

Uses: compressing single files and logs on servers, distributing source code as .tar.gz, and compressing web pages on the fly (HTTP Content-Encoding: gzip). The same DEFLATE stream (zlib) sits inside PNG images and countless programs. Strengths: fast, universal, extremely well tested. Weaknesses: gzip handles one stream, so it needs tar for folders, and newer methods compress better.

tar

Uses: bundling directories on Linux and macOS with permissions and ownership preserved, for software releases and server backups. Strengths: simple, streamable, and keeps Unix file metadata. Weaknesses: no compression of its own, and no index, so finding one file means reading through the archive.

bzip2 (.bz2)

Uses: older Linux packages and source releases, and scientific and text datasets. Strengths: better ratio than gzip on text. Weaknesses: slower than gzip in both directions, and now largely replaced by xz and Zstandard.

7z

Uses: getting the smallest possible archive for storage or download, and strongly encrypted archives. Strengths: high ratio with LZMA2, "solid" compression (treating many files as one stream so similarities between files are found), AES-256 encryption that can hide file names too, and a free open-source tool. Weaknesses: not every system opens it without extra software (though Windows 11 now can), and compression is slow at high settings.

RAR

Uses: large downloads split into parts, and archives that need recovery records, extra data that can repair some damage. Strengths: good ratio, recovery records, AES-256 encryption in RAR5. Weaknesses: proprietary; creating RAR files requires WinRAR or other licensed software from its developer, though free tools can extract them.

xz (.xz, .tar.xz)

Uses: Linux distribution packages and kernel and source releases, where a small download matters more than compression time. Strengths: high ratio, fast enough to decompress. Weaknesses: slow to compress at high levels. In 2024 a deliberately planted backdoor was discovered in XZ Utils versions 5.6.0 and 5.6.1 and caught before it reached most stable systems. It is a famous reminder that open-source supply chains need careful review.

Zstandard (.zst)

Uses: anywhere speed and size both matter: databases, file systems, backups, the Linux kernel, package managers (Arch Linux and Fedora use it for packages), and web pages in modern browsers. Strengths: an enormous range of levels, from faster than gzip with a better ratio to near-xz ratios, and very fast decompression at every level. Weaknesses: newer, so older systems may lack it.

Brotli (.br)

Uses: compressing web pages, scripts and stylesheets sent by servers (Content-Encoding: br), and WOFF2 web fonts. Strengths: excellent ratio on web text, helped by a built-in dictionary of common web words, plus fast decompression in every browser. Weaknesses: very slow at its highest quality, so sites usually pre-compress static files.

LZ4 and Snappy

Uses: real-time compression where speed beats size: file systems such as ZFS, databases, game assets, and network and in-memory data. Strengths: extremely fast in both directions. Weaknesses: a modest ratio.

CAB

Uses: Microsoft's Cabinet format packages Windows installers, drivers and system updates. Strengths: built into Windows. Weaknesses: rarely used outside Microsoft software.

Our benchmark: the same files, seven methods

We compressed five real files with seven common methods to show the trade-offs. The figures show compressed size as a percentage of the original, so smaller is better. The test files: 911 KB of English text (Markdown), 2.2 MB of TypeScript source code, 515 KB of JSON data, a 330 KB file that was already gzip-compressed, and 1 MB of random bytes.

What the numbers show:

• Text and code compress very well: code shrank to under a fifth of its size with the strongest methods.
• Already-compressed and random data do not shrink at all. bzip2 even made them slightly bigger. Zipping JPEGs, MP4s or existing ZIP files mainly bundles them; it does not save space.
• The trade-off is speed. Zstandard level 3 was the fastest method here and still beat gzip's ratio, while Brotli at maximum quality gave the smallest results for code and JSON but was the slowest by far. On plain text, bzip2 came out slightly smaller.

Open-source and closed-source archivers

An archive format and the software that handles it are two different things. A format can be openly documented even if some programs that use it are commercial.

Open-source tools and libraries

• 7-Zip: free and open source (LGPL) for Windows, with its own 7z format. It also opens ZIP, RAR, tar, gzip, xz and many more.
• PeaZip: a free open-source graphical archiver for Windows and Linux.
• Info-ZIP (zip and unzip), gzip, bzip2 and XZ Utils: the classic command-line tools found on Linux and macOS.
• zlib: the DEFLATE library inside an enormous range of software, from web browsers to PNG viewers.
• libarchive: a library that reads and writes many formats; Windows 11 uses it for its new archive support.
• Zstandard, Brotli and LZ4: open-source libraries with permissive licences, built into browsers, servers and operating systems.

Closed-source and commercial tools

• WinRAR: the RAR format's own tool, distributed as a trial and sold as a licence. The RAR compression method is proprietary; a free "unRAR" component may be used to extract RAR files, but its licence forbids using it to recreate the RAR compressor.
• WinZip: a long-standing commercial ZIP tool, which introduced the AES encryption extension widely used in ZIP files.
• PKZIP: PKWARE's commercial tools, from the company that created the ZIP format.
• Built-in operating-system tools: Windows File Explorer and macOS's Archive Utility handle common formats out of the box.

For most people, open formats are the safer long-term choice: ZIP for compatibility, 7z for small size. An archive you can still open in twenty years is worth more than one that saved a few extra percent.

ArchiveUtility by IttisalXone

If you work with archives on Windows every day, take a look at ArchiveUtility, a Windows desktop application from IttisalXone. IttisalXone also offers an ArchiveUtility module within its platform, for teams that want archive handling alongside their other business tools. Visit ittisalx.com for supported formats, features and availability.

Security: what can go wrong with archives

• Weak ZIP passwords. The original "ZipCrypto" encryption in ZIP can be broken with known-plaintext attacks, published as early as 1994. Use AES-256 encryption (supported by 7-Zip, WinZip, WinRAR and others) with a strong password.
• Visible file names. Standard ZIP encryption protects file contents but not file names. 7z and RAR can encrypt the names as well.
• Zip bombs. A tiny archive can expand to an enormous size. The famous "42.zip" is about 42 KB but unpacks to petabytes through nested layers, aiming to fill a disk or crash software. Good extractors cap how much they will unpack: our UnZIP Files and UnTAR Files tools stop at 750 MB of extracted data or 20,000 entries and say so.
• Path traversal ("Zip Slip"). A malicious archive can contain names like ../../file that try to write outside the folder you extract to. This class of bug was widely publicised in 2018. Our extractors flag and block any entry with .., an absolute path or another unsafe path.
• Malware in archives. Archives are a favourite way to slip malicious programs past email filters, so be cautious with unexpected attachments, especially password-protected ones.
• Corrupted downloads. If a publisher gives a checksum, verify the archive with our File Checksum tool before you extract it.

Which format should you use?

Compress and extract files in your browser

Our free tools handle the most common tasks without installing anything, and without uploading your files:

1. ZIP Compressor: add files or a whole folder (the structure is kept) and download a single .zip, compressed with DEFLATE at a balanced level 6.
2. UnZIP Files: open a .zip, browse its folders and download only the files you need, with unsafe paths blocked.
3. UnTAR Files: open .tar, .tar.gz and .tgz archives the same way.
4. File Checksum: confirm a downloaded archive is complete and unchanged.
5. Compress PDF: shrink a PDF itself, rather than zipping it.

The bottom line

Compression finds patterns and describes them in fewer bits. Text, code and data shrink dramatically; photos, videos and existing archives do not. Use ZIP when compatibility matters, 7z or xz when size matters, and Zstandard when speed matters. Protect sensitive archives with AES-256, and treat archives from strangers with care.