Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #1161

Re: Entropy Definition : Request for Comments

Show key headers only | View raw

On Thursday, March 15, 2012 11:10:51 AM UTC, Thomas Richter wrote:
> On 15.03.2012 07:29, lawcounsels@gmail.com wrote:
> 
> >> So this is not exactly "interesting". The answer is immediate, and
> >> exactly what is expected.
> >>
> >> More interesting things happen if you map this to a smaller set, say,
> >> you consider for example sums of random variables. Then, under mild
> >> constraints of the source, you get something called the "central limit
> >> theorem", i.e. your distribution approaches Gaussians (or stable
> >> distributions under less constrained conditions).
> >>
> >> Greetings,
> >> 	Thomas
> >
> > we are both on the right wavelengths here
> >
> > myself would 100 times out of 100 immediate says the exact same you did here without hesitations ... EXCEPT the INVARIABLE REVERSIBLE resultant transformed file of same N bits long ALWAYS GIVES sizeable portions with unmistakable particularly strong patterns / restrictions ( UNMISTAKABLY UNAMBIGUOUS LOW ENTROPY bits sequences at unmistakable unambiguous ascertainable 'start' / 'end'  bits positions .... mathematics proven instantiated with the described Entropy Definition formula on ALL real transformed resultant files )
> 
> Look, what people often confuse is "has many obvious patterns" = "has 
> low entropy". This is simply wrong. A specific outcome of a random 
> source may very well have "obvious patterns", but might still have a low 
> probability.
> 
> For example, let Y be a binary random iid source with p(0)=7/8 and 
> p(1)=1/8. Let z_k be the n-th digit in the binary expansion of Pi, or to 
> be precise, let z_k = floor(\pi * 2^k) mod 2. Then define x_k = y^k xor 
> z_k. Then, the outcome (realization) of z_k being 0000...000000 is 
> surely very unlikely, yet has an obvious pattern, and the entropy of Y 
> and the entropy of X are identical. (And the entropy of Z is zero 
> because it is not a random process...).
> 
> This "paradox" stems from the fact that sources humans like you or me 
> care about only care about reduntant sources, i.e. the random sources we 
> look at (newspapers, photos, audio recordings) are highly redundant 
> *and* contain many patterns. That is, for the sources we care about, we 
> have "low entropy" = "many patterns". But this is simply false in 
> general. Or to be even more concrete "having a pattern" is not a 
> well-defined term, nor does it have any relation to entropy. It *only* 
> (or should I write "only"?) has a relation to entropy for those sources 
> we as human observers care about. Sources with patterns allow "simple 
> models", but just because the models are simpler that allow you to 
> consider them as low-entropy random sources does not mean that there are 
> low-entropy sources that do not show obvious patterns. In fact, in a 
> precise way, *most* low-order entropy sources are like that (i.e. do not 
> show obvious patterns). Unfortunately, even these "most low order 
> entropy sources" (with non-obvious patterns) are still a very rare 
> species in the set of all sources, rare enough to disallow any escape 
> from the counting argument.
> 
> So long,
> 	Thomas

these are different,  the strong patterns here  mathematics proven instantiated with the described Entropy Definition formula on ALL real transformed resultant files )

Back to comp.compression | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Entropy Definition : Request for Comments LawCounsels <LawCounsels@aol.com> - 2012-03-13 01:40 -0700
  Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-13 10:13 +0100
    Re: Entropy Definition : Request for Comments LawCounsels <lawcounsels@gmail.com> - 2012-03-13 10:03 -0700
      Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-13 19:25 +0100
        Re: Entropy Definition : Request for Comments LawCounsels <lawcounsels@gmail.com> - 2012-03-13 11:56 -0700
          Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-13 20:38 +0100
            Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 01:43 -0700
              Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 01:46 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-14 12:09 +0100
                Re: Entropy Definition : Request for Comments LawCounsels <LawCounsels@aol.com> - 2012-03-14 08:52 -0700
                Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 11:34 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-14 22:16 +0100
                Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 23:29 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-15 12:10 +0100
                Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-15 06:23 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-15 16:45 +0100
                Re: Entropy Definition : Request for Comments LawCounsels <LawCounsels@aol.com> - 2012-03-20 18:08 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-14 22:10 +0100
  Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 16:27 -0700
  Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 16:40 -0700
    Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 17:14 -0700
      Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 17:24 -0700
        Re: Entropy Definition : Request for Comments LawCounsels@aol.com - 2012-03-31 10:30 -0700

csiph-web