Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.compression > #758 > unrolled thread

I have a small toy program...

Started byjules Gilbert <jules.stocks@gmail.com>
First post2012-01-12 13:50 -0800
Last post2012-01-15 03:32 -0800
Articles 20 on this page of 43 — 10 participants

Back to article view | Back to comp.compression


Contents

  I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-12 13:50 -0800
    Re: I have a small toy program... Earl_Colby_Pottinger <earlcolby.pottinger@sympatico.ca> - 2012-01-12 17:21 -0800
    Re: I have a small toy program... Jim Leonard <mobygamer@gmail.com> - 2012-01-13 08:22 -0800
      Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-15 10:30 -0800
      Re: I have a small toy program... Noob <root@127.0.0.1> - 2012-01-16 10:29 +0100
        Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-16 10:16 -0800
          Re: I have a small toy program... Jim Leonard <mobygamer@gmail.com> - 2012-01-16 11:08 -0800
          Re: I have a small toy program... Sebastian <s.gesemann@gmail.com> - 2012-01-16 15:34 -0800
          Re: I have a small toy program... stan <smoore@exis.net> - 2012-01-17 21:07 -0500
            Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-19 17:27 -0800
              Re: I have a small toy program... Jim Leonard <mobygamer@gmail.com> - 2012-01-20 07:36 -0800
                Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-20 19:05 -0800
                  Re: I have a small toy program... Thomas Richter <thor@math.tu-berlin.de> - 2012-01-21 11:02 +0100
                  Re: I have a small toy program... Sebastian <s.gesemann@gmail.com> - 2012-01-21 01:55 -0800
                    Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-27 21:30 -0800
                      Re: I have a small toy program... Sebastian <s.gesemann@gmail.com> - 2012-01-28 06:59 -0800
                  Re: I have a small toy program... Jim Leonard <mobygamer@gmail.com> - 2012-01-23 11:09 -0800
                    Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-23 19:18 -0800
                      Re: I have a small toy program... pfraser <pete_fraser@comcast.net> - 2012-01-23 19:53 -0800
                        Re: I have a small toy program... glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-01-24 11:33 +0000
                        Re: I have a small toy program... Earl_Colby_Pottinger <earlcolby.pottinger@sympatico.ca> - 2012-01-24 06:30 -0800
                          Re: I have a small toy program... glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-01-24 15:35 +0000
                            Re: I have a small toy program... Earl_Colby_Pottinger <earlcolby.pottinger@sympatico.ca> - 2012-01-24 13:00 -0800
                              Re: I have a small toy program... jacko <jackokring@gmail.com> - 2012-01-24 13:37 -0800
                          Re: I have a small toy program... pfraser <pete_fraser@comcast.net> - 2012-01-24 07:55 -0800
                            Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-28 11:03 -0800
                              Re: I have a small toy program... Jim Leonard <mobygamer@gmail.com> - 2012-01-30 10:01 -0800
                                Re: I have a small toy program... stan <smoore@exis.net> - 2012-01-30 18:59 -0500
                      Re: I have a small toy program... Jim Leonard <mobygamer@gmail.com> - 2012-01-24 07:31 -0800
                Re: I have a small toy program... glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-01-21 14:34 +0000
                  Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-27 21:22 -0800
                    Re: I have a small toy program... pfraser <pete_fraser@comcast.net> - 2012-01-28 07:00 -0800
                      Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-28 11:06 -0800
    Re: I have a small toy program... Thomas Richter <thor@math.tu-berlin.de> - 2012-01-13 20:03 +0100
      Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-14 17:13 -0800
        Re: I have a small toy program... Thomas Richter <thor@math.tu-berlin.de> - 2012-01-15 11:28 +0100
          Re: I have a small toy program... glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-01-15 11:53 +0000
            Re: I have a small toy program... Thomas Richter <thor@math.tu-berlin.de> - 2012-01-15 13:24 +0100
              Re: I have a small toy program... glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-01-15 13:48 +0000
                Re: I have a small toy program... jules Gilbert <jules.stocks@gmail.com> - 2012-01-15 10:43 -0800
                  Re: I have a small toy program... Thomas Richter <thor@math.tu-berlin.de> - 2012-01-15 20:44 +0100
          Re: I have a small toy program... stan <smoore@exis.net> - 2012-01-15 21:31 -0500
        Re: I have a small toy program... Sebastian <s.gesemann@gmail.com> - 2012-01-15 03:32 -0800

Page 1 of 3  [1] 2 3  Next page →


#758 — I have a small toy program...

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-12 13:50 -0800
SubjectI have a small toy program...
Message-ID<1ae3367a-f07c-46bd-b521-65b3df34225f@q17g2000yqh.googlegroups.com>
To those who have a business requirement and can prove their identity
and who promise not to further distribute it, I am happy to forward a
small C program I wrote that builds a RAD buffer and then applies it
to zlib, measuring and reporting the degree of compression, buffer
size, etc...  The purpose for which I wrote this small program was to
see what zlib was good at, what it's weaknesses are and more
importantly, what it's strengths are.

For example, I learned that a RAD buffer must contain at least a
57-58% bias in order for it to compress at all.

And I found other non-linearities, these useful, ways that knowing my
goods, I could maximize their compressive effect.

Please include a one sentence statement, stating your promise not to
further distribute my program.

Please include the name of the company that you work for or are
otherwise associated with.

I don't distribute zlib.  Why?, because every OS I've used in the past
few years has it built in.  Even the Apple I use as a mail appliance.
Under unix-like OS's, you merely need to say "-lz" to have the zlib
library loaded.

zlib is not my program, it's the work of Jean-loup and Adler.

Email me at:

repeatable
underscore
compression
ngista
oohay
tod
moc

And please, put the words TOY PROGRAM in the subject so a small mail
tool I use can send it back.

[toc] | [next] | [standalone]


#759

FromEarl_Colby_Pottinger <earlcolby.pottinger@sympatico.ca>
Date2012-01-12 17:21 -0800
Message-ID<37ddb3c1-f275-4771-b02a-f0eb9a53bb37@z12g2000yqm.googlegroups.com>
In reply to#758
On Jan 12, 4:50 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
> To those who have a business requirement and can prove their identity
> and who promise not to further distribute it, I am happy to forward a
> small C program I wrote that builds a RAD buffer and then applies it
> to zlib, measuring and reporting the degree of compression, buffer
> size, etc...  The purpose for which I wrote this small program was to
> see what zlib was good at, what it's weaknesses are and more
> importantly, what it's strengths are.

Who cares? We already know how to use zlib.  Stop wasting time and
write a de-compressor for your compressor and test it.

[toc] | [prev] | [next] | [standalone]


#760

FromJim Leonard <mobygamer@gmail.com>
Date2012-01-13 08:22 -0800
Message-ID<0a83ccd1-410d-4d7f-a986-221d1d5e278f@k28g2000yqn.googlegroups.com>
In reply to#758
On Jan 12, 3:50 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
> For example, I learned that a RAD buffer must contain at least a
> 57-58% bias in order for it to compress at all.

The fact that you are using an LZ77/LZSS compressor as the processor
for your residuals shows that you either are unwilling or unable to
comprehend the advice you have been given over the last 1.5 decades.
It also shows a lack of understanding of what zlib is best suited
for.  An order-0 statistical back-end will produce better results
(unless your residuals are full of repeating patterns).  Try a huffman
encoder instead.

Better yet, stop dicking around and write a decompressor.  I'm amazed
you haven't actually tested your decompression method *even once*.
How can you make claims if you haven't actually tested your process?

[toc] | [prev] | [next] | [standalone]


#768

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-15 10:30 -0800
Message-ID<f70212ae-651a-4ec8-bbcd-49365ccad52c@o9g2000yqa.googlegroups.com>
In reply to#760
On Jan 13, 11:22 am, Jim Leonard <mobyga...@gmail.com> wrote:
> On Jan 12, 3:50 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
>
> > For example, I learned that a RAD buffer must contain at least a
> > 57-58% bias in order for it to compress at all.
>
> The fact that you are using an LZ77/LZSS compressor as the processor
> for your residuals shows that you either are unwilling or unable to
> comprehend the advice you have been given over the last 1.5 decades.
> It also shows a lack of understanding of what zlib is best suited
> for.  An order-0 statistical back-end will produce better results
> (unless your residuals are full of repeating patterns).  Try a huffman
> encoder instead.
>
> Better yet, stop dicking around and write a decompressor.  I'm amazed
> you haven't actually tested your decompression method *even once*.
> How can you make claims if you haven't actually tested your process?

To Jim Leonard:  actually Jim, I think being able to reconstruct the
client buffer by exercising a PRNG tops your suggestions.  Now my
problem is finding ways to sell the program which keep it out of your
hands (among others,) for years and years.  That's my goal now.

And something else, I labeled this note, "A small toy program."   Yet
you guys pounced as if I were a fraud.

[toc] | [prev] | [next] | [standalone]


#772

FromNoob <root@127.0.0.1>
Date2012-01-16 10:29 +0100
Message-ID<jf0ql3$bs1$2@dont-email.me>
In reply to#760
Jim Leonard wrote:

> I'm amazed you haven't actually tested your decompression method
> *even once*. How can you make claims if you haven't actually tested
> your process?

Lemme take a wild guess here.

http://en.wikipedia.org/wiki/Crank_(person)

[toc] | [prev] | [next] | [standalone]


#774

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-16 10:16 -0800
Message-ID<18f1c133-0f05-4ced-b174-a1714d952452@p42g2000vbt.googlegroups.com>
In reply to#772
On Jan 16, 4:29 am, Noob <r...@127.0.0.1> wrote:
> Jim Leonard wrote:
> > I'm amazed you haven't actually tested your decompression method
> > *even once*. How can you make claims if you haven't actually tested
> > your process?
>
> Lemme take a wild guess here.
>
> http://en.wikipedia.org/wiki/Crank_(person)

I had intended to offer a small (70x byte .C program,) to interested
parties.  The program compresses RAD input, and yes it uses zlib as
the compression engine.  A program that may not be perfect in one or
two ways but you know, when I see large files crunched down to a few
hundred bytes I don't think it matters much.

You and others who post here are used to thinking about compression as
a single step, because you see compression as something that can occur
once, assuming your using a modern, quality tool.  Like zlib.

The difference my methods provide is an escape from this "one-time
only" model.  I have a method for representing a random buffer very
succinctly and losslessly recreating it.  And I use zlib to store the
"keys" necessary to perform the decompression (the recreation.)  How?,
well, I will only say that these guys are right, as their programs are
constructed, it's impossible.  Not that this gives them the right to
do as they have done, but I acknowledge that the problem was difficult
and that as most people look at the problem, it is indeed unsolvable
-- because no one, no one, violates natural laws.

My program is real, it doesn't make use of hidden files, anything
related to secret wireless transfers, file renaming, hacking the disk
size reporting, or anything that would permit or facilitate a fraud.
And when I show my program I use a floppy, a regular 3.5" floppy to
transfer the final intermediate file (the compressed image,) from one
machine to another.

Oh, and I get, sometimes, about 20:1 compression.  And that's from a
single pass.  And worst case, about 15:1.  And yes, it's this program
that I repeatably apply compression to.

The bad news is that my program operates on fixed-sized blocks of
data.  So, assuming that the final final size is one block plus a
fragment, say BLKSIZE/2, then you get (BLKSIZE/15) + (BLKSIZE/2) as
the average final file size.

And before a bunch of you decided to interfere, I was offering a demo
program that does just what I say:  It demonstrates that it's possible
to compress RAD buffers using zlib.  It demo's this by doing it.  In
seven hundred bytes.

An early version, (I wrote this in a few minutes because I needed to
make a choice and to make that choice I needed to know some numbers,)
was defective because it didn't free two buffers at the proper fix-
point (a technical term relating to entry and exit points of blocks of
code in a routine.)  Otherwise no one who has this program has
complained.

[toc] | [prev] | [next] | [standalone]


#775

FromJim Leonard <mobygamer@gmail.com>
Date2012-01-16 11:08 -0800
Message-ID<52e3d076-b886-48f9-b38c-0d61bb988f70@h3g2000yqe.googlegroups.com>
In reply to#774
On Jan 16, 12:16 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
> I had intended to offer a small (70x byte .C program,) to interested
> parties.  The program compresses RAD input, and yes it uses zlib as
> the compression engine.

We don't want your compression program, or any code from you.  What we
want is for you to WRITE and TEST the DEcompression phase of your
method.  Until you do this, you are a crank.

> I have a method for representing a random buffer very
> succinctly and losslessly recreating it.

No, you have *claimed* you have a method for recreating it.  It is now
obvious you have never actually tested your recreation method, which
makes your claim meaningless.

> And before a bunch of you decided to interfere, I was offering a demo
> program that does just what I say:  It demonstrates that it's possible
> to compress RAD buffers using zlib.  It demo's this by doing it.  In
> seven hundred bytes.

Anyone can compress any data.  The real trick is decompressing it.
Come back when you can decompress.

[toc] | [prev] | [next] | [standalone]


#776

FromSebastian <s.gesemann@gmail.com>
Date2012-01-16 15:34 -0800
Message-ID<cbe8d318-cb37-4b09-8519-2d30f0dbbddd@v14g2000vbc.googlegroups.com>
In reply to#774
On 16 Jan., 19:16, jules Gilbert wrote:
> On Jan 16, 4:29 am, Noob  wrote:
> > Lemme take a wild guess here.
> > http://en.wikipedia.org/wiki/Crank_(person)

:-)

> [...]
> You and others who post here are used to thinking about compression as
> a single step, because you see compression as something that can occur
> once [...]

Please elaborate.

> The difference my methods provide is an escape from this "one-time
> only" model.  I have a method for representing a random buffer very
> succinctly and losslessly recreating it. [...]

Representing something with fewer bits IS compression -- no matter how
many steps you take to do this. In fact, most of the state-of-the-art
compression algorithms -- especially those for video and audio -- are
very smart concatenations of carefully designed modules that are each
performing a very specific task. So, how is what you do any different?
The "black box compressor interface" is exatly the same: there is some
input and there is some output. And in the case of lossless
compression, there has to be another black box which takes the output
of the previous box and turns it into the original file without any
hidden communication between the black boxes. If you claim that your
method is worth looking at, it has to have such a interface on the
surface (no matter what happens inside). And if there is a
mathematical proof that applies to all such black boxes, it applies to
your method as well since such proof would not exploit any
implementation details. We call them "black" boxes because we cannot
look inside. There is no escape from such a proof unless you change
the rules. And if you change the rules you're basically cheating.

Cheers!
SG

[toc] | [prev] | [next] | [standalone]


#778

Fromstan <smoore@exis.net>
Date2012-01-17 21:07 -0500
Message-ID<93mhu8-olk.ln1@invalid.net>
In reply to#774
jules Gilbert wrote:
>
> I had intended to offer a small (70x byte .C program,) to interested
> parties.  The program compresses RAD input, and yes it uses zlib as
> the compression engine.  A program that may not be perfect in one or
> two ways but you know, when I see large files crunched down to a few
> hundred bytes I don't think it matters much.

To use the term "compression" you must be able to decompress,
otherwise you are simply shrinking data into a small mess. As long as
you never decompress you can easily shrink any file into 1 bit in a
single pass.

> You and others who post here are used to thinking about compression as
> a single step, because you see compression as something that can occur
> once, assuming your using a modern, quality tool.  Like zlib.

You have repeatedly refused to make any real effort to understand
any position other than your own. The above statement is a point of
evidence. To be very clear, the number of steps in your process is in
no way relevant to the validity of your claims.

> The difference my methods provide is an escape from this "one-time
> only" model.  I have a method for representing a random buffer very
> succinctly and losslessly recreating it.

THIS is your repeated bogus claim, and your idea of preprocessing the
data has been considered and studied thoroughly by some pretty smart
people. 

<snip>

> but I acknowledge that the problem was difficult and that as most
> people look at the problem, it is indeed unsolvable -- because no
> one, no one, violates natural laws.

The problem, your claim, is not "difficult" it is one of those natural
laws. The problem seems to be that while nobody violates these laws
they are apparently beyond the grasp of some people. People who fail
to understand frequently find apparent loopholes to get around the
problem only to discover when the rubber hits the road that their
loophole is in fact a pothole.

> My program is real, 
      
 Your claim is real, there is no evidence the program is real and you
 have yet to actually claim you can reverse the process.

> The bad news is that my program operates on fixed-sized blocks of
> data.

I'm afraid the bad news isn't really that limited.

> And before a bunch of you decided to interfere,

Strange you seem to define refusal as interference. You came and made
the same claims as before and got the same response, to expect
otherwise is at best unwise.

[toc] | [prev] | [next] | [standalone]


#792

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-19 17:27 -0800
Message-ID<9084d0b9-b0ab-41ed-86ff-878bfe098ae9@t30g2000vbx.googlegroups.com>
In reply to#778
On Jan 17, 9:07 pm, stan <smo...@exis.net> wrote:
> jules Gilbert wrote:
>
> > I had intended to offer a small (70x byte .C program,) to interested
> > parties.  The program compresses RAD input, and yes it uses zlib as
> > the compression engine.  A program that may not be perfect in one or
> > two ways but you know, when I see large files crunched down to a few
> > hundred bytes I don't think it matters much.
>
> To use the term "compression" you must be able to decompress,
> otherwise you are simply shrinking data into a small mess. As long as
> you never decompress you can easily shrink any file into 1 bit in a
> single pass.
>
> > You and others who post here are used to thinking about compression as
> > a single step, because you see compression as something that can occur
> > once, assuming your using a modern, quality tool.  Like zlib.
>
> You have repeatedly refused to make any real effort to understand
> any position other than your own. The above statement is a point of
> evidence. To be very clear, the number of steps in your process is in
> no way relevant to the validity of your claims.
>
> > The difference my methods provide is an escape from this "one-time
> > only" model.  I have a method for representing a random buffer very
> > succinctly and losslessly recreating it.
>
> THIS is your repeated bogus claim, and your idea of preprocessing the
> data has been considered and studied thoroughly by some pretty smart
> people.
>
> <snip>
>
> > but I acknowledge that the problem was difficult and that as most
> > people look at the problem, it is indeed unsolvable -- because no
> > one, no one, violates natural laws.
>
> The problem, your claim, is not "difficult" it is one of those natural
> laws. The problem seems to be that while nobody violates these laws
> they are apparently beyond the grasp of some people. People who fail
> to understand frequently find apparent loopholes to get around the
> problem only to discover when the rubber hits the road that their
> loophole is in fact a pothole.
>
> > My program is real,
>
>  Your claim is real, there is no evidence the program is real and you
>  have yet to actually claim you can reverse the process.
>
> > The bad news is that my program operates on fixed-sized blocks of
> > data.
>
> I'm afraid the bad news isn't really that limited.
>
> > And before a bunch of you decided to interfere,
>
> Strange you seem to define refusal as interference. You came and made
> the same claims as before and got the same response, to expect
> otherwise is at best unwise.

Actually, for maybe a decade I did try, and very earnestly, to make
friends here.  And yes, with a few people, maybe five, I did.  But I
have gotten attacked by, what?, perhaps as many as 300 people.  This
estimate is probably wrong -- I have no idea of a more accurate
estimate.  (It's above 200, I'm pretty sure.)

So now we fight on and of course I am determined not to budge -- For
example, I recently responded with a letter to a prospect and I made
certain he understood that he could only use my program in certain
countries.

I do have very different ideas about compression, that's true.

Now to you personally, I have to put some timing support in my
program, maybe you can help me...

I want to time everything, which means four components;   My
compression pre-process and decompression post-process, and also the
zlib compression and decompression steps.  Showing these times info is
probably a good idea.  Now, I've got people who might know a little
bit (I'm not sure,) who have commented here, but what they don't know
is that I've got several types of compressors (three types,) and one
method clearly violate those premises that most people who post here
here hold so dear.  Others, those based on the data transfer model,
obey all usual the traffic laws;  Unfortunately those programs are my
worst performers.

My point:  Pretty much I am content to let these guys get agitated.
It used to bother me, not now.  At one time I was hoping to sell my
program for $100M or so and while that could still happen, I've been
in contact with several players at some pretty big companies and they
say that they are turning down deals that once were considered to be
good business opportunities.  So, I'm looking to sell into countries
like China and elsewhere.

Oh, more about some needed extensions...

I use argtable, so I will probably plug in a rich set of options.
Important, I want to be able to show BOTH the compression times and
the decompression times.  And I will probably soon have to do files as
segments, wherein I process a large block, say 10MB, completely, then
continue for all remaining blocks.

[toc] | [prev] | [next] | [standalone]


#796

FromJim Leonard <mobygamer@gmail.com>
Date2012-01-20 07:36 -0800
Message-ID<eeea866f-1e86-4b61-a806-d715486d1c2c@e8g2000yqd.googlegroups.com>
In reply to#792
On Jan 19, 7:27 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
> I use argtable, so I will probably plug in a rich set of options.
> Important, I want to be able to show BOTH the compression times and
> the decompression times.  And I will probably soon have to do files as
> segments, wherein I process a large block, say 10MB, completely, then
> continue for all remaining blocks.

Why are you concerned about timing your code when you haven't verified
your decompression process works?  Shouldn't you be verifying that
first?

[toc] | [prev] | [next] | [standalone]


#797

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-20 19:05 -0800
Message-ID<bd3ccf6d-67d9-490e-87e6-1f9e98f713a9@do4g2000vbb.googlegroups.com>
In reply to#796
On Jan 20, 10:36 am, Jim Leonard <mobyga...@gmail.com> wrote:
> On Jan 19, 7:27 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
>
> > I use argtable, so I will probably plug in a rich set of options.
> > Important, I want to be able to show BOTH the compression times and
> > the decompression times.  And I will probably soon have to do files as
> > segments, wherein I process a large block, say 10MB, completely, then
> > continue for all remaining blocks.
>
> Why are you concerned about timing your code when you haven't verified
> your decompression process works?  Shouldn't you be verifying that
> first?

Some basic numbers.

A file below about 550 bytes won't compress.  I don't know the actual
numeric lower limit, because a specific number doesn't exist, of
course. The process performance depends on the exact content of
course.  But I have files around that size that, when zlib tries, get
larger.  But 600 byte files do indeed compress, just not a lot.

A file of 16k will drop by about 7%, per pass.

A file of 50k will drop by about 11.5% or so.

And a file of about 60k will drop by about 12% or so.




[toc] | [prev] | [next] | [standalone]


#798

FromThomas Richter <thor@math.tu-berlin.de>
Date2012-01-21 11:02 +0100
Message-ID<jfe2fa$d2g$1@news.belwue.de>
In reply to#797
On 21.01.2012 04:05, jules Gilbert wrote:
> On Jan 20, 10:36 am, Jim Leonard<mobyga...@gmail.com>  wrote:
>> On Jan 19, 7:27 pm, jules Gilbert<jules.sto...@gmail.com>  wrote:
>>
>>> I use argtable, so I will probably plug in a rich set of options.
>>> Important, I want to be able to show BOTH the compression times and
>>> the decompression times.  And I will probably soon have to do files as
>>> segments, wherein I process a large block, say 10MB, completely, then
>>> continue for all remaining blocks.
>>
>> Why are you concerned about timing your code when you haven't verified
>> your decompression process works?  Shouldn't you be verifying that
>> first?
>
> Some basic numbers.

85.6% of all numbers are made up.

> A file below about 550 bytes won't compress.  I don't know the actual
> numeric lower limit, because a specific number doesn't exist, of
> course. The process performance depends on the exact content of
> course.  But I have files around that size that, when zlib tries, get
> larger.  But 600 byte files do indeed compress, just not a lot.
>
> A file of 16k will drop by about 7%, per pass.
>
> A file of 50k will drop by about 11.5% or so.
>
> And a file of about 60k will drop by about 12% or so.

And will it decompress again to its original form? No? Not tested, not 
yet implemented after 20 years of work? Oh, too bad.


[toc] | [prev] | [next] | [standalone]


#799

FromSebastian <s.gesemann@gmail.com>
Date2012-01-21 01:55 -0800
Message-ID<7673e818-810b-499a-bf96-a1f08d7a807b@b18g2000vbz.googlegroups.com>
In reply to#797
On 21 Jan., 04:05, jules Gilbert wrote:
>
> Some basic numbers.
>
> A file below about 550 bytes won't compress.  I don't know the actual
> numeric lower limit, because a specific number doesn't exist, of
> course. The process performance depends on the exact content of
> course.  But I have files around that size that, when zlib tries, get
> larger.  But 600 byte files do indeed compress, just not a lot.
>
> A file of 16k will drop by about 7%, per pass.
>
> A file of 50k will drop by about 11.5% or so.
>
> And a file of about 60k will drop by about 12% or so.

These numbers strongly suggest that you're lying and/or that your
"preprocessing" module maps different inputs to the same output
(possibly due to one or more bugs in your software).

[toc] | [prev] | [next] | [standalone]


#869

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-27 21:30 -0800
Message-ID<a129b98e-8708-4e61-a312-cf8b608fda7f@j14g2000vba.googlegroups.com>
In reply to#799
On Jan 21, 4:55 am, Sebastian <s.gesem...@gmail.com> wrote:
> On 21 Jan., 04:05, jules Gilbert wrote:
>
>
>
> > Some basic numbers.
>
> > A file below about 550 bytes won't compress.  I don't know the actual
> > numeric lower limit, because a specific number doesn't exist, of
> > course. The process performance depends on the exact content of
> > course.  But I have files around that size that, when zlib tries, get
> > larger.  But 600 byte files do indeed compress, just not a lot.
>
> > A file of 16k will drop by about 7%, per pass.
>
> > A file of 50k will drop by about 11.5% or so.
>
> > And a file of about 60k will drop by about 12% or so.
>
> These numbers strongly suggest that you're lying and/or that your
> "preprocessing" module maps different inputs to the same output
> (possibly due to one or more bugs in your software).

No, Sebastian -- I'm not lying.  Not one bit!!, these numbers were
written by me several days ago with me looking at one screen and
typing into my Apple, which I use as my internet appliance.

I have a directory of 9,998 gzipped files, named ordinally.  The files
range in size from about 1024 bytes to about 256k bytes.  And I have a
centralized research and test engine that gets a number from a PRNG
and uses that to "pick" a file to process.  (I have no control over
which files get chosen, nor do I know the type of contents, though
most are not things like MPEG or JPEG files.  And probably I should
have more bzip based files.)

I was looking at a simple report as I wrote those numbers.  reading
off of one screen and writing on my apple.  Okay?

[toc] | [prev] | [next] | [standalone]


#872

FromSebastian <s.gesemann@gmail.com>
Date2012-01-28 06:59 -0800
Message-ID<a42d3fb1-d045-4544-bdfe-01162e3d0458@m5g2000yqk.googlegroups.com>
In reply to#869
On 28 Jan., 06:30, jules Gilbert wrote:
> On Jan 21, 4:55 am, Sebastian wrote:

> > These numbers strongly suggest that you're lying and/or that your
> > "preprocessing" module maps different inputs to the same output
> > (possibly due to one or more bugs in your software).
>
> No, Sebastian -- I'm not lying.  Not one bit!!

I didn't say that you are lying. I said that lying is one possible
explanation for the numbers.

[toc] | [prev] | [next] | [standalone]


#807

FromJim Leonard <mobygamer@gmail.com>
Date2012-01-23 11:09 -0800
Message-ID<bf284868-fa86-4bd8-aae6-1c361b310371@c8g2000yqc.googlegroups.com>
In reply to#797
On Jan 20, 9:05 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
> But 600 byte files do indeed compress, just not a lot.

They decompress too?  You've tested it?

[toc] | [prev] | [next] | [standalone]


#808

Fromjules Gilbert <jules.stocks@gmail.com>
Date2012-01-23 19:18 -0800
Message-ID<1799a8c6-8aaf-4e45-85a0-55494ff1071c@i25g2000vbt.googlegroups.com>
In reply to#807
On Jan 23, 2:09 pm, Jim Leonard <mobyga...@gmail.com> wrote:
> On Jan 20, 9:05 pm, jules Gilbert <jules.sto...@gmail.com> wrote:
>
> > But 600 byte files do indeed compress, just not a lot.
>
> They decompress too?  You've tested it?

As I recall the routine name is "uncompress" with the same arguments
as the compress function.

[toc] | [prev] | [next] | [standalone]


#809

Frompfraser <pete_fraser@comcast.net>
Date2012-01-23 19:53 -0800
Message-ID<jfla0c$ibr$1@dont-email.me>
In reply to#808
jules Gilbert wrote:

> As I recall the routine name is "uncompress" with the same arguments
> as the compress function.

Does it work?
i.e., will it take everything that your compressor generates,
and uncompress it to a file identical to the initial file?

[toc] | [prev] | [next] | [standalone]


#812

Fromglen herrmannsfeldt <gah@ugcs.caltech.edu>
Date2012-01-24 11:33 +0000
Message-ID<jfm4te$iq6$1@speranza.aioe.org>
In reply to#809
pfraser <pete_fraser@comcast.net> wrote:
> jules Gilbert wrote:

>> As I recall the routine name is "uncompress" with the same arguments
>> as the compress function.

> Does it work?
> i.e., will it take everything that your compressor generates,
> and uncompress it to a file identical to the initial file?

Especially the very unusual or rare cases? I wrote previously 
about a problem in my jbig2 code that showed up in about one
in one hundred tries, using real scanned files. 

As compressors get more complicated, there may be many conditions
that show up very rarely. There are systematic ways to track down
such problems, but not as much fun as writing program, and so
it is rare that people do it. 

-- glen

[toc] | [prev] | [next] | [standalone]


Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | comp.compression


csiph-web