how to combine the probability of disjoint symbol models?

From	moogie <budgetanime@mystarship.com>
Newsgroups	comp.compression
Subject	how to combine the probability of disjoint symbol models?
Date	2012-03-22 05:35 -0700
Organization	http://groups.google.com
Message-ID	<4357147.716.1332419723101.JavaMail.geo-discussion-forums@pbcvs5> (permalink)

Show all headers | View raw

As a thought exercise I am attempting to see whether there is some gains to be made via a statistical compression of an already compressed file (as output from another compression method, e.g. LZMA )

My current thought process is as follows:

1. The output of the other compression method produces a file attempting to minimise the entropy, probably at the 8 bit symbol size.

2. As the compressed file is of finite size, there exists a symbol length size (lets call it Smin) (in bits) where there is no duplicate symbols that exist in the compressed file.

3. If we read in this compressed file symbol by symbol (Smin size) then the number of bits necessary to encode each symbol can be derived by the log of the number of unseen symbols in the alphabet divided by log of 2.

4. Arithmetically encode the symbol using the derived probabilities.

5. if we reach the end of file before reading the next symbol fully then we just record those bits as-is

Compression can occur when the symbol size (Smin) is less than the number of bits in the original compressed file.

Before I go too far into my investigation of this idea, are the points above valid? Have I made a logical fallacy?

Back to comp.compression | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-22 05:35 -0700
  Re: how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-22 05:36 -0700
  Re: how to combine the probability of disjoint symbol models? biject <biject.bwts@gmail.com> - 2012-03-22 07:36 -0700
    Re: how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-22 17:48 -0700
  Re: how to combine the probability of disjoint symbol models? Thomas Richter <thor@math.tu-berlin.de> - 2012-03-22 17:04 +0100
    Re: how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-22 20:02 -0700
      Re: how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-22 21:02 -0700
        Re: how to combine the probability of disjoint symbol models? biject <biject.bwts@gmail.com> - 2012-03-22 21:29 -0700
          Re: how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-22 22:38 -0700
  Re: how to combine the probability of disjoint symbol models? moogie <budgetanime@mystarship.com> - 2012-03-23 05:42 -0700

csiph-web