Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #1156

Re: Entropy Definition : Request for Comments

Show all headers | View raw

On Wednesday, March 14, 2012 3:52:29 PM UTC, LawCounsels wrote:
> On Mar 14, 11:09 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
> > On 14.03.2012 09:46, lawcouns...@gmail.com wrote:
> >
> >
> >
> > >> yes ... its now corrected says Model
> >
> > > A useful Entropy Definition for variable length N bits produced by
> > > Model for which the probabilities of length N symbols produced&
> > > immediate STOP was defined  :
> >
> > > SUMS [  entropy of N bits * % probability of Model producing N
> > > bits ]  where N = 1 to arbitrary large # of bits produced by Model
> >
> > And once again, the answer is "no" because it depends on your model.
> > Just saying "variable length N bits" does not specify precisely enough
> > what are you looking for because "variable length string" is nothing
> > that is already well-defined.
> >
> > Just to give you an example how one can model "variable length" strings:
> >
> > Model 1) A variable length string on an alphabet \Omega is a string
> > consisting of an alphabet \Omega'=\Omega \cup {EOF} (EOF = EOF-Symbol)
> > and an equivalence relation ~ such that string s_1 from \Omega' and s_2
> > from \Omega' are equivalent under the conditions that
> >
> > s_1(i) = s_2(i) for all i<=i_n where i_n is the first position in s such
> > that s_1(i_n) = s_2(i_n) = EOF.
> >
> > In other words, you consider strings up to an additional EOF symbol and
> > do not care about the strings beyond this. This is certainly one model
> > how to think about variable length strings.
> >
> > Model 2) A variable length string is an element of \Omega^N, where N is
> > a random variable, and \Omega is an alphabet. A probability distribution
> > on random length strings is given by probability distributions
> > p_1,p_2,...,p_n where p_i is defined on \Omega^i. The probability of
> > getting a string s of length N consisting of symbols a_1,...a_n is given
> > by \sum_n q(n) p_n(a_1....a_n) where q is the probability distribution
> > on the string lengths.
> >
> > Here, in this model you first roll a dice to define the length of the
> > string, and then roll dices to fill in the method.
> >
> > It is not ad-hoc clear which model you are talking about (or possibly
> > about a third model, or whether these models are probably identical), so
> > what the probability distributions are. Hence, it is completely unclear
> > what "entropy" should be in this case unless you are more specific how
> > you arrive at the length and at the symbols.
> >
> > IOW, to repeat this again: Your question doesn't make sense. Please be
> > more specific which type of model you have in mind to generate "variable
> > length strings", and depending on that, an entropy definition might be
> > given. Without further information, all is up to speculation.
> 
> In the 1st instance the formula should be generally applicable to any
> models without restrictions
> whatsoever , including ALL those presently ample described by
> Shannon's fixed N-Block ...
> also the Model 1) & Model 2) described  OR any other Models anybody
> could conceiveable
> comes up with  ( hopefully which also can be 'economic'  interesting )
> 
> Otherwise the formula is not a valid general formula in the 1st place

An interesting related phenoma would be to ask :

if complete random binary file of length N bits were to be REVERSIBLE 'transformed' / 'randomised' / 'jumbled' by any methods ,
will the resultant 'jumbled' of exact same N bits EXHIBIT any particular
patterns / restrictions  on any portion of the resultant file BEYOND those normally associated with 'random' file ( like the 1Million Random Digits file )

Back to comp.compression | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Entropy Definition : Request for Comments LawCounsels <LawCounsels@aol.com> - 2012-03-13 01:40 -0700
  Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-13 10:13 +0100
    Re: Entropy Definition : Request for Comments LawCounsels <lawcounsels@gmail.com> - 2012-03-13 10:03 -0700
      Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-13 19:25 +0100
        Re: Entropy Definition : Request for Comments LawCounsels <lawcounsels@gmail.com> - 2012-03-13 11:56 -0700
          Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-13 20:38 +0100
            Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 01:43 -0700
              Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 01:46 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-14 12:09 +0100
                Re: Entropy Definition : Request for Comments LawCounsels <LawCounsels@aol.com> - 2012-03-14 08:52 -0700
                Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 11:34 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-14 22:16 +0100
                Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-14 23:29 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-15 12:10 +0100
                Re: Entropy Definition : Request for Comments lawcounsels@gmail.com - 2012-03-15 06:23 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-15 16:45 +0100
                Re: Entropy Definition : Request for Comments LawCounsels <LawCounsels@aol.com> - 2012-03-20 18:08 -0700
                Re: Entropy Definition : Request for Comments Thomas Richter <thor@math.tu-berlin.de> - 2012-03-14 22:10 +0100
  Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 16:27 -0700
  Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 16:40 -0700
    Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 17:14 -0700
      Re: Entropy Definition : Request for Comments jacko <jackokring@gmail.com> - 2012-03-15 17:24 -0700
        Re: Entropy Definition : Request for Comments LawCounsels@aol.com - 2012-03-31 10:30 -0700

csiph-web