Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #1201 > unrolled thread

Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles

Back to article view | Back to comp.compression

Contents

  Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-03-31 10:32 -0700
    Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-03 11:32 +0200
      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-03 09:07 -0700
        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-06 01:24 -0700
          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-08 00:41 -0700
            Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-08 04:25 -0700
              Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-08 22:58 -0700
                Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-09 04:19 -0700
                  Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles biject <biject.bwts@gmail.com> - 2012-04-09 09:03 -0700
                    Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-10 02:29 -0700
                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-10 05:43 -0700
                        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-10 06:39 -0700
                          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-10 06:51 -0700
                            Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-10 07:29 -0700
                              I knew I was going to dislike New Google Groups! James Dow Allen <jdallen2000@yahoo.com> - 2012-04-10 09:03 -0700
                                Re: I knew I was going to dislike New Google Groups! LawCounsels <lawcounsels@gmail.com> - 2012-04-10 09:56 -0700
                                  Re: I knew I was going to dislike New Google Groups! James Dow Allen <jdallen2000@yahoo.com> - 2012-04-10 10:34 -0700
                                    Re: I knew I was going to dislike New Google Groups! lawcounsels@gmail.com - 2012-04-10 10:54 -0700
                                      Re: I knew I was going to dislike New Google Groups! James Dow Allen <jdallen2000@yahoo.com> - 2012-04-11 13:16 -0700
                                        Re: I knew I was going to dislike New Google Groups! LawCounsels <lawcounsels@gmail.com> - 2012-04-12 03:10 -0700
                                          Re: I knew I was going to dislike New Google Groups! LawCounsels <lawcounsels@gmail.com> - 2012-04-12 03:57 -0700
                                            Re: I knew I was going to dislike New Google Groups! biject <biject.bwts@gmail.com> - 2012-04-12 08:38 -0700
                                              Re: I knew I was going to dislike New Google Groups! LawCounsels <lawcounsels@gmail.com> - 2012-04-13 02:34 -0700
                                  Re: I knew I was going to dislike New Google Groups! Fibonacci Code <anglikai@gmail.com> - 2012-04-30 07:36 -0700
                                    Re: I knew I was going to dislike New Google Groups! lawcounsels@gmail.com - 2012-04-30 08:34 -0700
                                      Re: I knew I was going to dislike New Google Groups! Thomas Richter <thor@math.tu-berlin.de> - 2012-04-30 18:53 +0200
                                      Re: I knew I was going to dislike New Google Groups! Fibonacci Code <anglikai@gmail.com> - 2012-04-30 09:58 -0700
                                        Re: I knew I was going to dislike New Google Groups! lawcounsels@gmail.com - 2012-04-30 10:18 -0700
                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-10 05:58 -0700
                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles biject <biject.bwts@gmail.com> - 2012-04-10 07:26 -0700
                        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-10 08:10 -0700
                          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles biject <biject.bwts@gmail.com> - 2012-04-10 11:02 -0700
                            Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-11 02:43 -0700
                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-11 02:47 +0200
                        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-11 02:56 -0700
                          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-11 16:26 +0200
                            Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-11 07:36 -0700
                              Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-11 10:29 -0700
                              Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-12 02:27 +0200
                                Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-12 03:08 -0700
                                  Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-12 03:34 -0700
                                    Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-13 13:48 +0200
                                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-13 05:15 -0700
                                        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-13 06:57 -0700
                                          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-13 07:03 -0700
                                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-13 06:52 -0700
                                        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-14 00:54 +0200
                                          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-16 02:48 -0700
                                            Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-16 13:08 +0200
                                              Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-16 04:40 -0700
                                              Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-16 08:11 -0700
                                                Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Thomas Richter <thor@math.tu-berlin.de> - 2012-04-16 19:31 +0200
                                                  Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles James Dow Allen <jdallen2000@yahoo.com> - 2012-04-16 14:27 -0700
                                                    Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-24 03:46 -0700
                                                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Jim Leonard <mobygamer@gmail.com> - 2012-04-24 07:27 -0700
                                                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Fibonacci Code <anglikai@gmail.com> - 2012-04-28 09:55 -0700
                      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels <lawcounsels@gmail.com> - 2012-04-16 02:34 -0700
      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-04 00:35 -0700
        Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-04 00:38 -0700
          Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-04 04:06 -0700
    Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles Ernst <Ernst_Berg@sbcglobal.net> - 2012-04-06 19:48 -0700
    Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles LawCounsels@aol.com - 2012-04-07 02:28 -0700
      Re: Mankind's centuries old Kraft's Inequality / Pigeonholes hurdles lawcounsels@gmail.com - 2012-04-07 03:22 -0700

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#1243

On Apr 12, 11:08 am, LawCounsels <lawcouns...@gmail.com> wrote:
> On Apr 12, 1:27 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
>
>
>
> > On 11.04.2012 16:36, James Dow Allen wrote:
>
> > > On Apr 11, 9:26 pm, Thomas Richter<t...@math.tu-berlin.de>  wrote:
> > >> That you cannot compress below 1.5bits/sample is the Shannon
> > >> result of lossless channel coding.
>
> > > Yes, he already knows this much.  What you seem to overlook is
> > > that he's refuted, both experimentally and theoretically,
> > > Shannon's theory, the pigeonhole principle, and even the
> > > Kraft's Inequality.  Naturally he's keeping details of his
> > > method secret; wouldn't you?
>
> > I wouldn't claim nonsense in first place, actually. (-: Initially, when
> > I saw the problem my reaction was that there is potentially a chance for
> > a very small improvement because the number of possible combinations for
> > a string of given size is a tiny bit smaller than the number of
> > combinations of all files. There are for example less than 3^3 = 27
> > possible three letter strings with the given constraint, so you can
> > compress them better because not all combinations are possible. The
> > string "CCC" is, for example, not possible.
>
> YOU'RE ON RIGHT TRACK HERE !
>
> OBSERVANT
>
> > However, the way the problem is stated right now, namely compress a
> > large number of strings of the same type simultaneously into one common
> > file kills this advantage completely because then all you know is just
> > where to separate strings again, and the advantage is going to zero for
> > the total string size going to infinity.
>
> YES THE PROBLEM NEEDED TO & WAS  'SIMPLIFIED' ... now that some
> understandings in place alreasdy I can now reveal the 'final' complete
> details :
>
> . each sequence's  distance to the start position of the next sequence
> ( ie from start of
> present sequence TO start of the next sequence ) IS ALWAYS FIXED [ ie
> ascertainable
> # of bits ! ] .... FOR SIMPLICITY AT THIS STAGE can just assume this
> to be constant fixed
> 513 bits throughout !!!
>
> HINT : HOWEVER BUT STILL YES  , the present exact 1,000 sequence CAN
> INDEED BE
> COMPRESSED SMALLER
>
> ( not withstanding this said to be 'proven' to be 1,5 bits / symbol
> entropy !!!  BUT as James
> said I could not be at liberty to publicise this complete 'new' method
> at this time ... INDEED they
> were experimentally tested proven & mathematics proved )
>
> > So yes, there remains a possibility for an incredibly small improvement
> > that tends to zero as N->infinity. How to take any advantage of it I do
> > not see right now, i.e. without requiring infinite precision in an encoder.
>
> > Greetings,
> >         Thomas
>
> LawCounsels

YES THE PROBLEM NEEDED TO & WAS  'SIMPLIFIED' ... now that some
understandings in place alreasdy I can now reveal the 'final' complete
details :

. each sequence's  distance to the start position of the next sequence
( ie from start of present sequence TO start of the next sequence ) IS
ALWAYS FIXED
[ ie ascertainable # of bits ! ] .... FOR SIMPLICITY AT THIS STAGE can
just assume this
to be constant fixed 513 bits throughout !!!

[ should the present sequence be just 2 symbols " C C "  ( 4 bits
long ) THEN will be followed
  by 513 -4 = 509 bits of filler 'dont-care'  bits  ]

[toc] | [prev] | [next] | [standalone]

#1247

> YES THE PROBLEM NEEDED TO&  WAS  'SIMPLIFIED' ... now that some
> understandings in place alreasdy I can now reveal the 'final' complete
> details :

So, in other words, you keep changing the problem?

> . each sequence's  distance to the start position of the next sequence
> ( ie from start of present sequence TO start of the next sequence ) IS
> ALWAYS FIXED
> [ ie ascertainable # of bits ! ] .... FOR SIMPLICITY AT THIS STAGE can
> just assume this
> to be constant fixed 513 bits throughout !!!

Still, makes no sense. Do you mean 513 *symbols*? Or do you mean that a 
sequence has to terminate after 513 output bits have been produced? Or 
"N" bits? Or "N" symbols?

Look, your problem description is lousy, at least.

> [ should the present sequence be just 2 symbols " C C "  ( 4 bits
> long ) THEN will be followed
>    by 513 -4 = 509 bits of filler 'dont-care'  bits  ]

Huh? I don't need "filler bits" if I know that the sequence terminates. 
There are surely better encodings for a finitely sized sequence provided 
I know that it must terminate after N symbols - or bits?

Sorry, no go.

[toc] | [prev] | [next] | [standalone]

#1248

On Apr 13, 6:48 pm, Thomas Richter <t...@math.tu-berlin.de> wrote:
> So, in other words, you keep changing the problem?

And this was AFTER he posted his FINAL & COMPLETE SPECIFICATIONS.
In capital letters, no less.

I think OP needs to hire someone at journeyman's wages to
figure out what is problem is, but I don't think OP can
afford 100's of dollars.  Unless someone can make change
for a Million-dollar bill.

BTW, has OP acknowledged yet that he started this thread
on the day called Poisson d'Avril?

James

[toc] | [prev] | [next] | [standalone]

#1250

On Friday, April 13, 2012 1:15:40 PM UTC+1, James Dow Allen wrote:
> On Apr 13, 6:48 pm, Thomas Richter <t...@math.tu-berlin.de> wrote:
> > So, in other words, you keep changing the problem?
> 
> And this was AFTER he posted his FINAL & COMPLETE SPECIFICATIONS.
> In capital letters, no less.

was also then 'explicit' stated to be 'SIMPLIFIED"
 
> I think OP needs to hire someone at journeyman's wages to
> figure out what is problem is, but I don't think OP can
> afford 100's of dollars.  Unless someone can make change
> for a Million-dollar bill.

well this takes 2 stage to reach , but I think more effective did the job easing in audiences 1st with main "simplified" problem description
 
> BTW, has OP acknowledged yet that he started this thread
> on the day called Poisson d'Avril?

NOT if someone comes up fine tunes improved bits savings/ per sequence
to 0.08 from present 0.063 attained 

[toc] | [prev] | [next] | [standalone]

#1251

On Friday, April 13, 2012 2:57:17 PM UTC+1, lawco...@gmail.com wrote:
> On Friday, April 13, 2012 1:15:40 PM UTC+1, James Dow Allen wrote:
> > On Apr 13, 6:48 pm, Thomas Richter <t...@math.tu-berlin.de> wrote:
> > > So, in other words, you keep changing the problem?
> > 
> > And this was AFTER he posted his FINAL & COMPLETE SPECIFICATIONS.
> > In capital letters, no less.
> 
> was also then 'explicit' stated to be 'SIMPLIFIED"

IF SOMEONE COMES UP WITH 0.08 BITS SAVINGS PER SEQUENCE ON THIS POSTED
"SIMOPLIFIED" FINAL & COMPLETE SPECIFICATIONS  ALL THE SAME ( without any of the subsequent fine filled-in details ) WINS THE REWARD ..... in this sense it was Final & Complete enough to win the rewards base on 
  
> > I think OP needs to hire someone at journeyman's wages to
> > figure out what is problem is, but I don't think OP can
> > afford 100's of dollars.  Unless someone can make change
> > for a Million-dollar bill.
> 
> well this takes 2 stage to reach , but I think more effective did the job easing in audiences 1st with main "simplified" problem description
>  
> > BTW, has OP acknowledged yet that he started this thread
> > on the day called Poisson d'Avril?
> 
> NOT if someone comes up fine tunes improved bits savings/ per sequence
> to 0.08 from present 0.063 attained

[toc] | [prev] | [next] | [standalone]

#1249

On Friday, April 13, 2012 12:48:38 PM UTC+1, Thomas Richter wrote:
> > YES THE PROBLEM NEEDED TO&  WAS  'SIMPLIFIED' ... now that some
> > understandings in place alreasdy I can now reveal the 'final' complete
> > details :
> 
> So, in other words, you keep changing the problem?

NOT REALLY KEEP CHANGING ... necessarily best starts with the main basic problem description 1st THEN once audience showed overall understandings then to fill in the few fine details left out from 'simplification' ( to help get correct bearings 1st ) ... IN FACT , IT WAS 'EXPLICIT' WRITTEN "...SIMPLIFIED..." , this must means something 

THE MAIN PROBLEM AS "SIMPLIFIED" DESCRIBED was a real problem on its own right with most already thought practical intractable not possible represents with less than 1.5 bits / symbol practically  [ no practical present existing compressor can do this )
 
> > . each sequence's  distance to the start position of the next sequence
> > ( ie from start of present sequence TO start of the next sequence ) IS
> > ALWAYS FIXED
> > [ ie ascertainable # of bits ! ] .... FOR SIMPLICITY AT THIS STAGE can
> > just assume this
> > to be constant fixed 513 bits throughout !!!
> 
> Still, makes no sense. Do you mean 513 *symbols*? Or do you mean that a 
> sequence has to terminate after 513 output bits have been produced? Or 
> "N" bits? Or "N" symbols?

has to terminate after 513 output BITS 
 
> Look, your problem description is lousy, at least.

was already from very beginning 'explicit' stated problem description was 'simplified' to help get basic understandings 1st... the subsequent fine details ( once basic grasp of problem gained ) does not alter things tremendous

... will overload everyone to start with full-monty
 
> > [ should the present sequence be just 2 symbols " C C "  ( 4 bits
> > long ) THEN will be followed
> >    by 513 -4 = 509 bits of filler 'dont-care'  bits  ]
> 
> Huh? I don't need "filler bits" if I know that the sequence terminates. 
> There are surely better encodings for a finitely sized sequence provided 
> I know that it must terminate after N symbols - or bits?
> 
> Sorry, no go.


yes ... you dont need them ... these so-called  "filler bits" really are just other compressed data part which needs not concern us 

[toc] | [prev] | [next] | [standalone]

#1252

On 13.04.2012 15:52, lawcounsels@gmail.com wrote:
> On Friday, April 13, 2012 12:48:38 PM UTC+1, Thomas Richter wrote:
>>> YES THE PROBLEM NEEDED TO&   WAS  'SIMPLIFIED' ... now that some
>>> understandings in place alreasdy I can now reveal the 'final' complete
>>> details :
>>
>> So, in other words, you keep changing the problem?
>
> NOT REALLY KEEP CHANGING ... necessarily best starts with the main basic problem description 1st THEN once audience showed overall understandings then to fill in the few fine details left out from 'simplification' ( to help get correct bearings 1st ) ... IN FACT , IT WAS 'EXPLICIT' WRITTEN "...SIMPLIFIED..." , this must means something

It did change the problem completely. It seems "a minor detail" for you, 
but unless the problem specification is complete, there is no solution. 
Just to give you the difference, compressing an infinite number of 
sequences to a common stream is a *different* problem than compressing a 
set of finitely sized streams. Actually, for the reasons indicated above.

BTW, "upper case" doesn't make you sound more credible.

>> Huh? I don't need "filler bits" if I know that the sequence terminates.
>> There are surely better encodings for a finitely sized sequence provided
>> I know that it must terminate after N symbols - or bits?
>>
>> Sorry, no go.
>
>
> yes ... you dont need them ... these so-called  "filler bits" really are just other compressed data part which needs not concern us

No, again - a problem change! Yes, it does make a change whether the 
stream terminates at this specific point or whether "other data" follows 
and you need to include truncation or termination information of some sorts.

Just to give you an idea why this is important: Consider an arithmetic 
coder to generate the output. By convention, we can truncate the file if 
we are sure that the remaining sequence generated by the AC coder is an 
infinite repetition of zeros (or ones, for that matter). A decoder 
could, if running into an EOF, pull "by convention" just zeros or ones. 
This would allow a clever encoder to truncate the output stream by all 
terminating zero bits (or one-bits, depending on the convention) and 
hence make it shorter.

If this sounds rather academic to you, then I should probably say that 
exactly this type of trick is played in some existing compression 
applications - of course using the "inherent" side information that the 
compressed stream has a finite size, and that this size can be obtained 
from other sources as side information. This trick allows you to 
compress a little bit better. Before you ask, it is prior art and a 
known trick, so no money to collect for this.

So, no, your problem IS NOT SPECIFIED FULLY. (Does upper case help to 
understand this better? I doubt it, but let's try).

What would be necessary to know, for example, whether these "other bits" 
need to be decoded as well, probably by using the same arithmetic 
decoder. If so, a simple "bit counter" to define the size of the decoder 
input (rather than the decoder output) does not make too much sense in 
first place. Or whether the input stream is "self-truncating", i.e. 
there is any other side information I could use to understand that 
decoding has reached an end.

To give you a glimpse of an idea what AC coders do: There are typically 
two lengths one could understand as the "size" of an AC coded stream:

a) the minimal number of input bits required to decode the signal up to 
a point A and then stop ("truncation") - and -

b) the minimal number of input bits required to decode the signal up to 
a point, and then to correctly continuing the decoding for any bits beyond.

Tricks mentioned as in the paragraph above should make clear that in 
general the "length a)" is smaller than the "length b)", in the MQ coder 
typically by two to three *bytes*. This is all due to the internal 
buffering and delay.

Unless you understand such technicalities, or at least are able to 
define the problem really completely, there is absolutely nothing that 
can be said.

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#1257

On Apr 13, 11:54 pm, Thomas Richter <t...@math.tu-berlin.de> wrote:
> On 13.04.2012 15:52, lawcouns...@gmail.com wrote:
>
> > On Friday, April 13, 2012 12:48:38 PM UTC+1, Thomas Richter wrote:
> >>> YES THE PROBLEM NEEDED TO&   WAS  'SIMPLIFIED' ... now that some
> >>> understandings in place alreasdy I can now reveal the 'final' complete
> >>> details :
>
> >> So, in other words, you keep changing the problem?
>
> > NOT REALLY KEEP CHANGING ... necessarily best starts with the main basic problem description 1st THEN once audience showed overall understandings then to fill in the few fine details left out from 'simplification' ( to help get correct bearings 1st ) ... IN FACT , IT WAS 'EXPLICIT' WRITTEN "...SIMPLIFIED..." , this must means something
>
> It did change the problem completely. It seems "a minor detail" for you,
> but unless the problem specification is complete, there is no solution.
> Just to give you the difference, compressing an infinite number of
> sequences to a common stream is a *different* problem than compressing a
> set of finitely sized streams. Actually, for the reasons indicated above.
>
> BTW, "upper case" doesn't make you sound more credible.
>
> >> Huh? I don't need "filler bits" if I know that the sequence terminates.
> >> There are surely better encodings for a finitely sized sequence provided
> >> I know that it must terminate after N symbols - or bits?
>
> >> Sorry, no go.
>
> > yes ... you dont need them ... these so-called  "filler bits" really are just other compressed data part which needs not concern us
>
> No, again - a problem change! Yes, it does make a change whether the
> stream terminates at this specific point or whether "other data" follows
> and you need to include truncation or termination information of some sorts.
>
> Just to give you an idea why this is important: Consider an arithmetic
> coder to generate the output. By convention, we can truncate the file if
> we are sure that the remaining sequence generated by the AC coder is an
> infinite repetition of zeros (or ones, for that matter). A decoder
> could, if running into an EOF, pull "by convention" just zeros or ones.
> This would allow a clever encoder to truncate the output stream by all
> terminating zero bits (or one-bits, depending on the convention) and
> hence make it shorter.
>
> If this sounds rather academic to you, then I should probably say that
> exactly this type of trick is played in some existing compression
> applications - of course using the "inherent" side information that the
> compressed stream has a finite size, and that this size can be obtained
> from other sources as side information. This trick allows you to
> compress a little bit better. Before you ask, it is prior art and a
> known trick, so no money to collect for this.
>
> So, no, your problem IS NOT SPECIFIED FULLY. (Does upper case help to
> understand this better? I doubt it, but let's try).
>
> What would be necessary to know, for example, whether these "other bits"
> need to be decoded as well, probably by using the same arithmetic
> decoder. If so, a simple "bit counter" to define the size of the decoder
> input (rather than the decoder output) does not make too much sense in
> first place. Or whether the input stream is "self-truncating", i.e.
> there is any other side information I could use to understand that
> decoding has reached an end.
>
> To give you a glimpse of an idea what AC coders do: There are typically
> two lengths one could understand as the "size" of an AC coded stream:
>
> a) the minimal number of input bits required to decode the signal up to
> a point A and then stop ("truncation") - and -
>
> b) the minimal number of input bits required to decode the signal up to
> a point, and then to correctly continuing the decoding for any bits beyond.
>
> Tricks mentioned as in the paragraph above should make clear that in
> general the "length a)" is smaller than the "length b)", in the MQ coder
> typically by two to three *bytes*. This is all due to the internal
> buffering and delay.
>
> Unless you understand such technicalities, or at least are able to
> define the problem really completely, there is absolutely nothing that
> can be said.
>
> Greetings,
>         Thomas


YES TRUE in the above MQ case

However here would already be failing badly real 'desperate' if solver
here needs ever resort to
possible but 'far far too small' potential savings from this AC
truncations which in all cases further
diminishes to insignificant when N -> arbitrary large size [ & only if
AC is the solution method
depended on ... ] .... would be better here not to be needless
distracted by non-essential details

[toc] | [prev] | [next] | [standalone]

#1258

On 16.04.2012 11:48, LawCounsels wrote:

> YES TRUE in the above MQ case
>
> However here would already be failing badly real 'desperate' if solver
> here needs ever resort to
> possible but 'far far too small' potential savings from this AC
> truncations which in all cases further
> diminishes to insignificant when N ->  arbitrary large size [&  only if
> AC is the solution method
> depended on ... ] .... would be better here not to be needless
> distracted by non-essential details

No, its savings are of the same magnitude as the savings you get from 
requiring that the sequence is 513 bits long. N is not infinity here, 
unless you changed the problem again. If N goes to infinity, there are 
no savings from the truncation condition, neither from the c = b + 2 
condition. This condition only provides "split points", but these do not 
matter for an infinite number of sequences.

So, now what? Will you provide a better problem desciption - the current 
description is not sufficient.

[toc] | [prev] | [next] | [standalone]

#1259

On Monday, April 16, 2012 12:08:45 PM UTC+1, Thomas Richter wrote:
> On 16.04.2012 11:48, LawCounsels wrote:
> 
> > YES TRUE in the above MQ case
> >
> > However here would already be failing badly real 'desperate' if solver
> > here needs ever resort to
> > possible but 'far far too small' potential savings from this AC
> > truncations which in all cases further
> > diminishes to insignificant when N ->  arbitrary large size [&  only if
> > AC is the solution method
> > depended on ... ] .... would be better here not to be needless
> > distracted by non-essential details
> 
> No, its savings are of the same magnitude as the savings you get from 
> requiring that the sequence is 513 bits long. N is not infinity here, 
> unless you changed the problem again. If N goes to infinity, there are 
> no savings from the truncation condition, neither from the c = b + 2 
> condition. This condition only provides "split points", but these do not 
> matter for an infinite number of sequences.
> 
> So, now what? Will you provide a better problem desciption - the current 
> description is not sufficient.

solutions to the Data Compression REWARDS as already stated ( without any subsequent further fine details needed ) does not require any of the possible but 'far far too small'  potential savings AC truncations savings if further now imposes limits each sequence length to 513 bits  , to attain Net 0.08 bits or more compressions savings

.... the couple 'certain' solutions put forth so far now to be tested developed to confirm has not made use of AC at all

think the Data Compression REWARDS as stated best SIMPLEST be just left as is intact 

[ however , I should also leave to anyone attempting a winning solution to 'as of right' if he so chooses , to assume each sequence can be of at most 513 bits long ... if this helps his solution strategy  ] 

[toc] | [prev] | [next] | [standalone]

#1260

On Apr 16, 6:08 pm, Thomas Richter <t...@math.tu-berlin.de> wrote:
> > depended on ... ] .... would be better here not to be needless
> > distracted by non-essential details
>
> No, its savings are of the same magnitude as the savings you get from
> requiring that the sequence is 513 bits long.

I don't think this is strictly true.

My understanding of the problem may be incomplete(!) but
I think you're comparing a *per-file* truncation cost of,
let's guess, about 5 bits *on average*, with a *per-token*
savings from the 513-bit limit.

Let's guess that savings is one nonillionth of a bit.
(Unless the a-b-c constraints have changed while I wasn't
looking, one nonillionth of a bit may be a severe
overestimate but let's go on....)

While one nonillionth of a bit is small, it's a *per-token*
savings and after a nonillion tokens, will add up to
a bit.  After several *decillion* tokens, these will add to
several thousand bits, while the truncation cost will
remain only about 5 bits, on average.

Does this help?

James

[toc] | [prev] | [next] | [standalone]

#1261

On 16.04.2012 17:11, James Dow Allen wrote:
> On Apr 16, 6:08 pm, Thomas Richter<t...@math.tu-berlin.de>  wrote:
>>> depended on ... ] .... would be better here not to be needless
>>> distracted by non-essential details
>>
>> No, its savings are of the same magnitude as the savings you get from
>> requiring that the sequence is 513 bits long.
>
> I don't think this is strictly true.
>
> My understanding of the problem may be incomplete(!) but
> I think you're comparing a *per-file* truncation cost of,
> let's guess, about 5 bits *on average*, with a *per-token*
> savings from the 513-bit limit.
>
> Let's guess that savings is one nonillionth of a bit.
> (Unless the a-b-c constraints have changed while I wasn't
> looking, one nonillionth of a bit may be a severe
> overestimate but let's go on....)
>
> While one nonillionth of a bit is small, it's a *per-token*
> savings and after a nonillion tokens, will add up to
> a bit.  After several *decillion* tokens, these will add to
> several thousand bits, while the truncation cost will
> remain only about 5 bits, on average.
>
> Does this help?

Not really, and I don't think this is the case. As far as I understand 
the problem now, the problem is that *each* of the individual 
subsections of the file have to end at 513 bits (output bits). Why that 
makes sense I do not know, but let it be as it is: It means that the 
number of necessary truncations is linear in the file size, and so is 
the number of truncations due to the condition of c = b + 2. IOW, I 
believe these two effects are of the same magnitude. All that depends of 
course on how this "513 bits" constraint comes into play, which I do not 
understand fully.

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#1262

On Apr 17, 12:31 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
> As far as I understand
> the problem now, the problem is that *each* of the individual
> subsections of the file have to end at 513 bits (output bits). Why that
> makes sense I do not know, ... All that depends of
> course on how this "513 bits" constraint comes into play, which I do not
> understand fully.

I apologize; it looks like I missed the latest memo.

Do we still get the $1 MILLION REWARD for satisfying the
FINAL and COMPLETE specification, with this latest frill just
an extra-credit problem for the $3 MILLION bonus?

Is there a YouTube link?

Best ever,
James

[toc] | [prev] | [next] | [standalone]

#1264

On Apr 16, 10:27 pm, James Dow Allen <jdallen2...@yahoo.com> wrote:
> On Apr 17, 12:31 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
>
> > As far as I understand
> > the problem now, the problem is that *each* of the individual
> > subsections of the file have to end at 513 bits (output bits). Why that
> > makes sense I do not know, ... All that depends of
> > course on how this "513 bits" constraint comes into play, which I do not
> > understand fully.
>
> I apologize; it looks like I missed the latest memo.
>
> Do we still get the $1 MILLION REWARD for satisfying the
> FINAL and COMPLETE specification, with this latest frill just
> an extra-credit problem for the $3 MILLION bonus?
>
> Is there a YouTube link?
>
> Best ever,
> James

Hi :

initial tests confirmed easy exceeded 8.0 bits savings per sequence
target .... am making this into .exe now

can any of you help lead arrange R&Ds at Intel/ Google / NASA / CERN
the likes ?

BTW : has any of you get close to 8.0 as yet ? pls tell

Warm Regards,
LawCounsels

[toc] | [prev] | [next] | [standalone]

#1265

On Apr 24, 5:46 am, LawCounsels <lawcouns...@gmail.com> wrote:
> initial tests confirmed easy exceeded 8.0 bits savings per sequence
> target .... am making this into .exe now

Congratulations.  Can you reverse the target back into the original
source?

> can any of you help lead arrange R&Ds at Intel/ Google / NASA / CERN
> the likes ?

Sure, as soon as you have a working decompressor.

> BTW : has any of you get close to 8.0 as yet ? pls tell

Depends on the data.  For example, an 8-bit RLE scheme will achieve
253:1.

[toc] | [prev] | [next] | [standalone]

#1271

On Apr 24, 6:46 pm, LawCounsels <lawcouns...@gmail.com> wrote:
> On Apr 16, 10:27 pm, James Dow Allen <jdallen2...@yahoo.com> wrote:
>
>
>
>
>
>
>
>
>
> > On Apr 17, 12:31 am, Thomas Richter <t...@math.tu-berlin.de> wrote:
>
> > > As far as I understand
> > > the problem now, the problem is that *each* of the individual
> > > subsections of the file have to end at 513 bits (output bits). Why that
> > > makes sense I do not know, ... All that depends of
> > > course on how this "513 bits" constraint comes into play, which I do not
> > > understand fully.
>
> > I apologize; it looks like I missed the latest memo.
>
> > Do we still get the $1 MILLION REWARD for satisfying the
> > FINAL and COMPLETE specification, with this latest frill just
> > an extra-credit problem for the $3 MILLION bonus?
>
> > Is there a YouTube link?
>
> > Best ever,
> > James
>
> Hi :
>
> initial tests confirmed easy exceeded 8.0 bits savings per sequence
> target .... am making this into .exe now
>
> can any of you help lead arrange R&Ds at Intel/ Google / NASA / CERN
> the likes ?
>
> BTW : has any of you get close to 8.0 as yet ? pls tell
>
> Warm Regards,
> LawCounsels

Years ago, I had already achieve over 8.0 bits sequence for random
tenary sequences. (Random Generator) with n = 128


Cheers

Raymond.

[toc] | [prev] | [next] | [standalone]

#1256

On Apr 10, 10:29 am, LawCouns...@aol.com wrote:
> On Monday, 9 April 2012 17:03:04 UTC+1, biject  wrote:
> > On Apr 9, 5:19 am, James Dow Allen <jdallen2...@yahoo.com> wrote:
> > > On Apr 9, 12:58 pm, lawcouns...@gmail.com wrote:
>
> > > > a source with probability of producing an 'a' symbol 25% of time
> > > > a 'b' symbol 25% of times a 'c' symbol 50% of times
>
> > > Allow me to recommend the optimal Huffman code:
> > >    c - 0
> > >    a - 10
> > >    b - 11
> > > This can be improved, though only slightly, using details
> > > you've omitted from your summary.
>
> > > This was so trivial, I'll discount it down to, say $950.
>
> > > If this is unsatisfactory, I'll withdraw from the contest.
> > > Even paid at minimum wage I'm afraid it would take significant
> > > funds (payable in advance, please!) just to elicit a
> > > proper problem statement from you.
>
> > > I don't have PayPal.  Contact me for instructions on how
> > > to pay the $950.  :-)
>
> > > James
>
> >   Lets see c is .5 * 1 = .5  b = .25*2 = .5  c = .25*2 = .5
> > see thats .5 + .5 + .5 = 1.5  for the average sequence  while
> > if you encode each with 1.5849625  you save about .0849625 which
> > is more than the .08  It appears your in the money. I have a
> > hunch that there still is something missing in which case I would
> > not count on the money yet.
>
> >  First of all does he want at least .08 bits saved in every case
> > or just the average case.  If its the average case you could be
> > on the right track.  If its every case then since you write only
> > whole numbers of bits the .08 savings gets a little harder. It
> > would be nice if the guy decides you haven't won just what does
> > he want. I have read it several times and yet I do not think its
> > clear enough to tackle without him saying oh I meant this and not
> > that.
>
> > Assuming he doesn't declare you the winner
> > 1) is the savings an average things or does each file have to be less.
> > 2) how do you measure the savings is it .08 from a 1.5849625 per
> > symbol
> > or is it .08 less then 1.5
> > 3) not sure why you say source C = .5 while A and B = .25  the
> > fact is even if the source is A = B = C = 1/3  for short files
> > if you run the sources enough times and created a 100 files each
> > you still could get the same set of 100 files for both cases.
> > So you test set up is not valid. There is nothing magical about
> > your source.  Except if I know its a fixed IID souce from say 2 or
> > 3 different models as you create more files. You can with increasing
> > probability determine which one it most likely is. But you can't be
> > 100% certain which one it is unless you do an ever increasing number
> > of file.
>
> >  David A. Scott
> > --
> >  My Crypto code
> >http://bijective.dogma.net/crypto/scott19u.zip
> >http://www.jim.com/jamesd/Kong/scott19u.zipold version
> > My Compression codehttp://bijective.dogma.net/
> > **TO EMAIL ME drop the roman "five" **
> > Disclaimer:I am in no way responsible for any of the statements
> >  made in the above text. For all I know I might be drugged.
> > As a famous person once said "any cryptograhic
> > system is only as strong as its weakest link"
>
> THE COMPLETE SPECIFICATIONS :
> =============================
>
> 1. generates a number eg 1,000 of such sequences ( each sequence composed of ternary symbols 'a' 'b' 'c' , when # of 'c' = # of 'b' + 2 Then sequence ENDS  & next sequences begins ) using a source producing symbol 'a' 25% of times symbol 'b' 25% of times symbol 'c' 50% of times ) .... call the total # of symbols in these 1,000 sequences N . NOTE : among these eg 1,000 sequences the # of 'a' is invariable near = the # of 'b'  & the # of 'c' is invariable near = 2 * the # of 'b'  THUS the probability model here is 25% : 25% : 50%
>
> 2. compresses these eg 1,000 generated sequences using your .exe , & must decode back to the same 1,000 sequences
>
> 3. IF you compressed file bitslength  =<  1.5 * N   - ( 0.08 * N )  THEN YOU WIN THE REWARDS !   ie if your .exe saves 'on average' 0.08 bit each sequences you WON ( needs not be invariable every time on every conceivable file ! ) , but note the original # of sequences is here taken to be of bitslength N * 1.5 bits long   ( as originally 'explicit' stated to be 1.5 * N bits long , NOT 1.5849625 * N bits long )
>
> 4. there is no restrictions on memory storage requirements , you may even show your .exe works on 'research network supercomputer cluster' , BUT processing must complete within a day- Hide quoted text -
>
> - Show quoted text -

=============================================================
UPDATE  ANNOUNCEMENT :
US$3M DATA COMPRESSION PRIZE :
=============================================================

AM NOW MADE AWARE OF COUPLE OF SOLUTIONS PUT FORTH CERTAIN
TO EASILY FAR EXCEED THE REQUIRED 0.08 BIT COMPRESSION SAVINGS PER
SEQUENCE NEEDED ....

WILL NOW NEEDS TAKE SOME TIME TEST DEVELOP CONFIRM THE SOLUTION

KEEP YOUR SOLUTIONS COMING .....
.

Warm Regards,
LawCounsels

[toc] | [prev] | [next] | [standalone]

#1204

On Tuesday, April 3, 2012 9:32:39 AM UTC, Thomas Richter wrote:
> For all 
> practical matters, pick a MQ coder, and encode the symbols by binary 
> decisions like 0->a, 10->b, 11-> c.

yes this is a good start .... this certainly satisfies 'self-delimiting' criteria 

all sequence starts with 'no symbols' ( nothing ) then progressively accumulates 'a' or 'b' or 'c'  .... the initial ( simplifying ) statstistics is such that among ALL the # of sequences ( one can always takes only exactly eg 1,000 etc sequences ... since the 'start' positions of each sequences all ascertainable) the total# 'a' exact = total # 'b'  and total # 'c' exact = total # 'b'

... most 'startling' most certain youwill soon enough exasperate conclude decide there is simply no possible way for existing known compression techniques to possible do this ! ... somewhat surprsingly given each of these sequences definite 'mathematics' NOT RANDOM  endowed with clear unambiguous structures and distributions bias ! 

which then you may then find this further statistics may or may not help you toward a 'complete' new kind of compressions method :  the length of each sequence ( the total # of symbols within ) follows mathematics derivable from the 1 : 1 : 2 distributions ratio of the 3 symbols 

YOU CAN GUARANTEE THIS WON'T BE SIMPLE !  

Warm Regards,
LawCounsels   

[toc] | [prev] | [next] | [standalone]

#1205

On Wednesday, April 4, 2012 7:35:37 AM UTC, LawCo...@aol.com wrote:

.... the initial ( simplifying ) statstistics is such that among ALL the # of sequences ( one can always takes only exactly eg 1,000 etc sequences ... since the 'start' positions of each sequences all ascertainable) the total# 'a' exact = total # 'b'  and total # 'c' exact = total # 'b'  +2

[toc] | [prev] | [next] | [standalone]

#1206

On Wednesday, April 4, 2012 8:38:07 AM UTC+1, LawCo...@aol.com wrote:
> On Wednesday, April 4, 2012 7:35:37 AM UTC, LawCo...@aol.com wrote:
> 
> .... the initial ( simplifying ) statstistics is such that among ALL the # of sequences ( one can always takes only exactly eg 1,000 etc sequences ... since the 'start' positions of each sequences all ascertainable) the total# 'a' exact = total # 'b'  and total # 'c' exact = total # 'b'  +2

.... the initial ( simplifying ) statstistics is such that among ALL the # of sequences ( one can always takes only exactly eg 1,000 etc sequences ... since the 'start' positions of each sequences all ascertainable) the total# 'a' exact = total # 'b'  and total # 'c' exact = total # 'b'  + 2*eg1000   

[ if takes exact 1,000 sequences , each sequence has 2 extra # of 'c' over # of 'b'  ]  

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.compression

csiph-web