Re: Words as well as characters as encoding units

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!eternal-september.org!feeder.eternal-september.org!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From	Niels Fröhling <spamtrap@adsignum.com>
Newsgroups	comp.compression
Subject	Re: Words as well as characters as encoding units
Date	Wed, 09 May 2012 15:14:31 -0600
Lines	19
Message-ID	<a10539FfasU1@mid.individual.net> (permalink)
References	<jo2o6f$nd9$1@news.albasani.net> <357e1db3-17e2-4911-9423-1cf394d894d5@ns1g2000pbc.googlegroups.com> <jo38jj$p8u$1@news.albasani.net> <jo3fj1$65m$1@news.belwue.de> <jo3oq1$qhu$1@news.albasani.net> <jo3rpm$lv1$1@speranza.aioe.org> <jo3ulr$79j$1@news.albasani.net> <jo41tc$6le$1@speranza.aioe.org> <jo430j$fod$1@news.albasani.net> <jo463v$hdq$1@speranza.aioe.org> <jo47ev$o9a$1@news.albasani.net> <a0rtihFu2kU1@mid.individual.net> <joakv1$99n$3@speranza.aioe.org>
Mime-Version	1.0
Content-Type	text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding	7bit
X-Trace	individual.net gn3jFlSHw5CYtq8TWaVhJA1tyWQkBsxOB5/dWxMWCrISbsZI3Eimi/1O/yG9L+O0cS
Cancel-Lock	sha1:qfOVF3tEA+tb4Yb15Fgl5ZXQyjo=
User-Agent	Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.7.5) Gecko/20041206 Thunderbird/1.0 Mnenhy/0.7.1
In-Reply-To	<joakv1$99n$3@speranza.aioe.org>
Xref	csiph.com comp.compression:1300

Show key headers only | View raw

> The question is what might be in a file that one wants to compress,
> and that might not be in a language. For example, one might be
> working with GPS systems, and have files full of latitude and
> longitude values. You can consider those "words", but might not
> have any correlation.

A format-type correlation is often enough for exploitation. The file "geo" of 
the Canterbury Corpus is the shining example, it's an array of floats, 
compressing it as bit-planes (knowing how the values correlate) yields a few 
tenths of percent of compression gain - because all the floats are in a specific 
correlating range AFAIR 0.0 to 1.0.

There are no uncorrelated data-sets besides total random data. Sometimes there 
may not be strong enough a correlation, and then any high-order compressors 
compress bad/worst, but exploiting correlations is their business, not plain 
static stationary order-0.

Ciao
   Niels

Thread

Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 10:26 +0200
  Re: Words as well as characters as encoding units James Dow Allen <jdallen2000@yahoo.com> - 2012-05-05 02:41 -0700
    Re: Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 15:06 +0200
      Re: Words as well as characters as encoding units Thomas Richter <thor@math.tu-berlin.de> - 2012-05-05 17:05 +0200
        Re: Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 19:42 +0200
          Re: Words as well as characters as encoding units glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-05-05 18:33 +0000
            Re: Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 21:22 +0200
              Re: Words as well as characters as encoding units Thomas Richter <thor@math.tu-berlin.de> - 2012-05-05 21:31 +0200
                Re: Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 22:19 +0200
              Re: Words as well as characters as encoding units glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-05-05 20:17 +0000
                Re: Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 22:36 +0200
                Re: Words as well as characters as encoding units glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-05-05 21:29 +0000
                Re: Words as well as characters as encoding units Mok-Kong Shen <mok-kong.shen@t-online.de> - 2012-05-05 23:52 +0200
                Re: Words as well as characters as encoding units Niels Fröhling <spamtrap@adsignum.com> - 2012-05-08 00:42 -0600
                Re: Words as well as characters as encoding units glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-05-08 08:19 +0000
                Re: Words as well as characters as encoding units Niels Fröhling <spamtrap@adsignum.com> - 2012-05-09 15:14 -0600
                Re: Words as well as characters as encoding units James Dow Allen <jdallen2000@yahoo.com> - 2012-05-09 22:17 -0700

csiph-web