Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #3818

Re: Oddities of popular archivers

Show all headers | View raw

Elhana <tanarriscourge@yahoo.com> writes:
> I used some popular archivers to compress a text file, and results
> surprised me quite much.
> 
> The worst contender turned out to be gzip, with average 60%
> reduction. Interestingly, the UTF-8 version is compressed worse
> than ISO one, with about 15% overhead. I was under impression
> that both files contain the same amount of information, so they
> should compress to a comparable amount.

What do you mean by "ISO"?  I'm going to guess that you're referring to
something like UTF-16 or UCS-2, commonly used on Windows.

> The next result belongs to PKZIP. It managed to compress each
> file about 40 bytes better than gzip. (the gzip header was 25
> bytes long).
> 
> The next result belongs to xzip. It managed with the UTF-8 text
> much better, giving only 8% overhead (which is still too much in
> my opinion). Average compression was 70%.
> 
> Next comes 7-zip, with default settings, which failed spectacularly
> on UTF-8 file, which turned out 8k more than xzip one. The other
> files compressed about 400 bytes better.
> 
> The silver prize went to bzip2, with its impressive 72%
> compression. Surprisingly, it processed the UTF-8 file even better,
> with only 5% overhead.
> 
> And the undisputed champion was WinRAR, with 81% compression.
> 
> The following questions arose:
> 
> * Why does xzip suck?
> * Why UTF-8 is not supported by mainstream compression software?
> * And why proprietary compression software so easily outperforms the
>   'free' one?

A UTF-8 version of a given chunk of text is very likely to be smaller
than a UTF-16 or UCS-2 version of the same text.  UCS-2 represents each
character as 16 bits.  UTF-8 represents each character with an encoding
in the range 0..127 in 8 bits (an in typical text, that's likely to be
most characters).

For a large chunk of ASCII text, it's likely that a UTF-8 representation
will be about half the size of a UCS-2 or UTF-16 representation.
Compressed versions of both are likely to be roughly the same size,
since the files contain about the same amount of information.  This
could vary if the input contains a not of non-ASCII characters.

(I don't have answers for your other questions.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

Back to comp.compression | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Oddities of popular archivers Elhana <tanarriscourge@yahoo.com> - 2019-07-10 09:21 -0700
  Re: Oddities of popular archivers Keith Thompson <kst-u@mib.org> - 2019-07-10 14:51 -0700
    Re: Oddities of popular archivers Elhana <tanarriscourge@yahoo.com> - 2019-07-10 19:59 -0700
      Re: Oddities of popular archivers Keith Thompson <kst-u@mib.org> - 2019-07-11 13:22 -0700
        Re: Oddities of popular archivers Elhana <tanarriscourge@yahoo.com> - 2019-07-14 08:35 -0700
          Re: Oddities of popular archivers Keith Thompson <kst-u@mib.org> - 2019-07-14 18:38 -0700

csiph-web