Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #89274 > unrolled thread

[sOT] compression 1 GB downto 100 MB

Back to article view | Back to comp.lang.c

Contents

  [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 07:25 -0700
    Re: [sOT] compression 1 GB downto 100 MB mark.bluemel@gmail.com - 2016-06-16 08:08 -0700
      Re: [sOT] compression 1 GB downto 100 MB Lynn McGuire <lmc@winsim.com> - 2016-06-17 12:04 -0500
        Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-17 10:16 -0700
          Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 23:08 -0700
    Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-16 08:09 -0700
      Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:15 -0700
        Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:18 -0700
          Re: [sOT] compression 1 GB downto 100 MB mark.bluemel@gmail.com - 2016-06-17 01:28 -0700
            Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-17 02:03 -0700
              Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-17 10:34 +0100
                Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-17 10:44 +0100
                  Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-17 11:03 +0100
                    Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-17 03:51 -0700
                      Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 02:16 -0700
                        Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 02:29 -0700
                    Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-17 08:19 -0500
                  Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 03:17 -0700
                Re: [sOT] compression 1 GB downto 100 MB Gareth Owen <gwowen@gmail.com> - 2016-06-17 18:32 +0100
        Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-16 08:43 -0700
          Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:48 -0700
        Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-16 10:49 -0500
          Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 09:10 -0700
            Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-16 09:36 -0700
              Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-16 11:18 -0700
            Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-16 13:13 -0500
              Re: [sOT] compression 1 GB downto 100 MB Gareth Owen <gwowen@gmail.com> - 2016-06-16 20:18 +0100
                Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-16 22:07 +0100
                  Re: [sOT] compression 1 GB downto 100 MB Gareth Owen <gwowen@gmail.com> - 2016-06-17 06:04 +0100
                Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-16 23:21 -0500
                  Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-17 07:49 -0500
                    Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-17 12:47 -0500
                      Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-17 14:47 -0500
                        Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-17 23:32 -0500
                          Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 01:33 -0700
              Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-17 14:51 -0700
        Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-16 16:50 +0100
          Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-17 16:20 +0100
      Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:34 -0700
    Re: [sOT] compression 1 GB downto 100 MB Les Cargill <lcargill99@comcast.com> - 2016-06-16 12:55 -0500
    Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-16 11:53 -0700
      Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-16 12:14 -0700
      Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 12:33 -0700
        Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 12:46 -0700
      Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-17 15:04 -0700
        Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 22:55 -0700
          Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 01:38 -0700
            Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-18 10:41 +0100
              Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 02:55 -0700
              Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-18 05:08 -0700
                Re: [sOT] compression 1 GB downto 100 MB "Bill Cunningham" <nospam@nspam.invalid> - 2016-06-18 15:11 -0400
                  Re: [sOT] compression 1 GB downto 100 MB "Bill Cunningham" <nospam@nspam.invalid> - 2016-06-18 15:13 -0400
                  Re: [sOT] compression 1 GB downto 100 MB luser droog <luser.droog@gmail.com> - 2016-06-18 12:43 -0700
                  Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-18 20:43 +0100
                    Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-18 12:53 -0700
                      Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-20 20:32 -0700
                    Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-18 13:22 -0700
                      Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 16:35 -0400
                        Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 21:37 -0400
                      Re: [sOT] compression 1 GB downto 100 MB "James R. Kuyper" <jameskuyper@verizon.net> - 2016-06-18 16:59 -0400
                        Re: [sOT] compression 1 GB downto 100 MB crisdunbar@gmail.com - 2016-06-20 09:55 -0700
                          Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-20 21:22 +0100
                            Re: [sOT] compression 1 GB downto 100 MB cris <crisdunbar@gmail.com> - 2016-06-20 17:13 -0700
                            Re: [sOT] compression 1 GB downto 100 MB Tim Rentsch <txr@alumni.caltech.edu> - 2016-06-21 10:42 -0700
                              Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-21 20:09 +0100
                                Re: [sOT] compression 1 GB downto 100 MB crisdunbar@gmail.com - 2016-06-21 13:39 -0700
                                Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-22 08:34 -0700
                                  Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-22 19:48 +0100
                    Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-19 01:37 +0100
                      Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-19 02:22 +0100
                        Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 21:30 -0400
                          Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-19 02:41 +0100
                            Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 21:55 -0400
                        Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-19 03:20 +0100
                          Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-19 02:53 -0700
                  Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-18 16:05 -0700
                    Re: [sOT] compression 1 GB downto 100 MB "J. Clarke" <j.clarke.873638@gmail.com> - 2016-06-19 06:59 -0400
                      Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-19 05:55 -0700
                      Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-19 06:13 -0700
                        Re: [sOT] compression 1 GB downto 100 MB "J. Clarke" <j.clarke.873638@gmail.com> - 2016-06-26 12:26 -0400
                          Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-26 09:53 -0700
                Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-18 20:41 +0100
                Re: [sOT] compression 1 GB downto 100 MB "J. Clarke" <j.clarke.873638@gmail.com> - 2016-06-19 06:45 -0400
              Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-18 14:42 -0500
              Re: [sOT] compression 1 GB downto 100 MB gordonb.vnsk1@burditt.org (Gordon Burditt) - 2016-06-20 01:13 -0500
                Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-20 12:16 -0700
                  Re: [sOT] compression 1 GB downto 100 MB David Brown <david.brown@hesbynett.no> - 2016-06-20 21:27 +0200
                    Re: [sOT] compression 1 GB downto 100 MB Xaltonon <xaltonon@gmail.com> - 2016-06-20 21:25 -0700

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

#89322

On 6/16/2016 7:25 AM, fir wrote:
 > GOD how great it would be have an algorith that would compress
 > totays data which rae not much compressibl;e (like game ISo
 > files, movies, zip archives) like many times, say ten times

 > wouldnt it be great? do you think that it is
 > possible? (im sure it is not really obtainable but other
 > thing if this is not impossible 'in fact')

 > Imagine how great it would be :U


Fantasy Island:
________________________________
Actually, one can theoretically compress truly random data, however the 
time it takes can be very long. The idea involves encrypting that data 
into a form that happens to contain many duplicates, or patterns that 
can be radically exploited for compression. This duplicate/pattern rich 
encrypted data can be compressed again. Sometimes, the new result will 
be smaller than the original. So, if we can store the key for the 
process that creates the duplicates along with the new compressed result 
in a way that is still smaller than the plaintext, then we have just 
compressed random data!
________________________________


;^D

[toc] | [prev] | [next] | [standalone]

#89325

On 6/16/2016 11:53 AM, Chris M. Thomasson wrote:
> On 6/16/2016 7:25 AM, fir wrote:
>> GOD how great it would be have an algorith that would compress
>> totays data which rae not much compressibl;e (like game ISo
>> files, movies, zip archives) like many times, say ten times
>
>> wouldnt it be great? do you think that it is
>> possible? (im sure it is not really obtainable but other
>> thing if this is not impossible 'in fact')
>
>> Imagine how great it would be :U
>
>
> Fantasy Island:
> ________________________________
[...]


Also, do not forget about the following company:

http://www.piedpiper.com

They have an interesting technology indeed!

:^)

> ________________________________

[toc] | [prev] | [next] | [standalone]

#89445

On 6/16/2016 11:53 AM, Chris M. Thomasson wrote:
> On 6/16/2016 7:25 AM, fir wrote:
>> GOD how great it would be have an algorith that would compress
>> totays data which rae not much compressibl;e (like game ISo
>> files, movies, zip archives) like many times, say ten times
>
>> wouldnt it be great? do you think that it is
>> possible? (im sure it is not really obtainable but other
>> thing if this is not impossible 'in fact')
>
>> Imagine how great it would be :U
>
>
> Fantasy Island:
> ________________________________
> [...]

Ahhhh. Here is a story... Alice wants to encrypt 32 bytes of random data 
plaintext P0. So, she runs it through various fractal formulas in shader 
fragments for hours using a super computer comprised of high end 
graphics cards. All of a sudden, she has a hit where a fractal F is 
found to create a shitload of patterns in the ciphertext C0, that happen 
to be very compressible. Now, Alice compresses this duplicate/pattern 
rich ciphertext again into ciphertext C1, and adds on the short fractal 
formula F in a compressed form (think compressed mathematical formula) 
into a ciphertext C2, where the side of C2 is less than the size of P0. 
Alice then sends C2 to Bob.

Decryption and decompression involves the decoder, Bob, to extract and 
decompress C1 from C2 using F, then decompress C1 into C0, then decrypt 
C0 into P1 that is identical to P0.

Wow!  ;^D

> ________________________________
[...]

[toc] | [prev] | [next] | [standalone]

#89447

On 6/17/2016 12:33 PM, Chris M. Thomasson wrote:
> On 6/16/2016 11:53 AM, Chris M. Thomasson wrote:
[...]
> Fantasy Island:
> ________________________________
> [...]
>
> Ahhhh. Here is a story... Alice wants to encrypt 32 bytes of random data
> plaintext P0. So, she runs it through various fractal formulas in shader
> fragments for hours using a super computer comprised of high end
> graphics cards. All of a sudden, she has a hit where a fractal F is
> found to create a shitload of patterns in the ciphertext C0, that happen
> to be very compressible. Now, Alice compresses this duplicate/pattern
> rich ciphertext again into ciphertext C1, and adds on the short fractal
> formula F in a compressed form (think compressed mathematical formula)
> into a ciphertext C2, where the side of C2 is less than the size of P0.
> Alice then sends C2 to Bob.8

"side" of C2 should be "size".



> Decryption and decompression involves the decoder, Bob, to extract and
> decompress C1 from C2 using F
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

NO... What a massive typo! Bob extracts F and C1 from C2, then uses zip, 
or whatever he likes to decompress C0 from C1, then gets P1 by 
decryption of C0 using F. P0 == P1.

Sorry for any radical confusion!

;^o





>, then decompress C1 into C0, then decrypt
> C0 into P1 that is identical to P0.
>
> Wow!  ;^D
>
>> ________________________________
> [...]
>

[toc] | [prev] | [next] | [standalone]

#89465

W dniu czwartek, 16 czerwca 2016 20:53:26 UTC+2 użytkownik Chris M. Thomasson napisał:
> On 6/16/2016 7:25 AM, fir wrote:
>  > GOD how great it would be have an algorith that would compress
>  > totays data which rae not much compressibl;e (like game ISo
>  > files, movies, zip archives) like many times, say ten times
> 
>  > wouldnt it be great? do you think that it is
>  > possible? (im sure it is not really obtainable but other
>  > thing if this is not impossible 'in fact')
> 
>  > Imagine how great it would be :U
> 
> 
> Fantasy Island:
> ________________________________
> Actually, one can theoretically compress truly random data, however the 
> time it takes can be very long. The idea involves encrypting that data 
> into a form that happens to contain many duplicates, or patterns that 
> can be radically exploited for compression. This duplicate/pattern rich 
> encrypted data can be compressed again. Sometimes, the new result will 
> be smaller than the original. So, if we can store the key for the 
> process that creates the duplicates along with the new compressed result 
> in a way that is still smaller than the plaintext, then we have just 
> compressed random data!

i was not talking on truly random data..
I got usabbility on my mind and i got
ny mind on things people usualy zip and send via net (mostly thinking probably on games/ game installers) and things like that)

didnt found an answer that today compressing ratios are roughly all that could be obtainable (and not radical improvement is 
possible) - but as answers are somewhat unsatisfactionary i changed my mind as to try to discussing it (probably) - probably i will take some lecture on a side in later time, sorry

[toc] | [prev] | [next] | [standalone]

#89503

On 6/17/2016 3:04 PM, fir wrote:
> W dniu czwartek, 16 czerwca 2016 20:53:26 UTC+2 użytkownik Chris M. Thomasson napisał:
>> On 6/16/2016 7:25 AM, fir wrote:
>>  > GOD how great it would be have an algorith that would compress
>>  > totays data which rae not much compressibl;e (like game ISo
>>  > files, movies, zip archives) like many times, say ten times
>>
>>  > wouldnt it be great? do you think that it is
>>  > possible? (im sure it is not really obtainable but other
>>  > thing if this is not impossible 'in fact')
>>
>>  > Imagine how great it would be :U
>>
>>
>> Fantasy Island:
>> ________________________________
>> Actually, one can theoretically compress truly random data, however the
>> time it takes can be very long. The idea involves encrypting that data
>> into a form that happens to contain many duplicates, or patterns that
>> can be radically exploited for compression. This duplicate/pattern rich
>> encrypted data can be compressed again. Sometimes, the new result will
>> be smaller than the original. So, if we can store the key for the
>> process that creates the duplicates along with the new compressed result
>> in a way that is still smaller than the plaintext, then we have just
>> compressed random data!
>
> i was not talking on truly random data..
> I got usabbility on my mind and i got
> ny mind on things people usualy zip and send via net (mostly thinking probably on games/ game installers) and things like that)
>
> didnt found an answer that today compressing ratios are roughly all

> that could be obtainable (and not radical improvement is
> possible) - but as answers are somewhat unsatisfactionary i changed

> my mind as to try to discussing it (probably) - probably i will take

> some lecture on a side in later time, sorry
>


Actually, I was trying to think outside the box wrt compressing truly 
random data. Finding an encryption that creates duplicates would be the 
first step. I think fractal encryption can be useful here because of the 
highly compact nature of the formulas.

[toc] | [prev] | [next] | [standalone]

#89507

W dniu sobota, 18 czerwca 2016 07:55:41 UTC+2 użytkownik Chris M. Thomasson napisał:
> On 6/17/2016 3:04 PM, fir wrote:
> > W dniu czwartek, 16 czerwca 2016 20:53:26 UTC+2 użytkownik Chris M. Thomasson napisał:
> >> On 6/16/2016 7:25 AM, fir wrote:
> >>  > GOD how great it would be have an algorith that would compress
> >>  > totays data which rae not much compressibl;e (like game ISo
> >>  > files, movies, zip archives) like many times, say ten times
> >>
> >>  > wouldnt it be great? do you think that it is
> >>  > possible? (im sure it is not really obtainable but other
> >>  > thing if this is not impossible 'in fact')
> >>
> >>  > Imagine how great it would be :U
> >>
> >>
> >> Fantasy Island:
> >> ________________________________
> >> Actually, one can theoretically compress truly random data, however the
> >> time it takes can be very long. The idea involves encrypting that data
> >> into a form that happens to contain many duplicates, or patterns that
> >> can be radically exploited for compression. This duplicate/pattern rich
> >> encrypted data can be compressed again. Sometimes, the new result will
> >> be smaller than the original. So, if we can store the key for the
> >> process that creates the duplicates along with the new compressed result
> >> in a way that is still smaller than the plaintext, then we have just
> >> compressed random data!
> >
> > i was not talking on truly random data..
> > I got usabbility on my mind and i got
> > ny mind on things people usualy zip and send via net (mostly thinking probably on games/ game installers) and things like that)
> >
> > didnt found an answer that today compressing ratios are roughly all
> 
> > that could be obtainable (and not radical improvement is
> > possible) - but as answers are somewhat unsatisfactionary i changed
> 
> > my mind as to try to discussing it (probably) - probably i will take
> 
> > some lecture on a side in later time, sorry
> >
> 
> 
> Actually, I was trying to think outside the box wrt compressing truly 
> random data. Finding an encryption that creates duplicates would be the 
> first step. I think fractal encryption can be useful here because of the 
> highly compact nature of the formulas.

truly random may be hard (maybe impossible)
to compress i dont know.. in normal date you could imagine some phase space i think this is a space of possible states of date in much larger space of all states, whole myriad of states are unusable (will not appear0, you only need some criterium to distinguish one part from another

[toc] | [prev] | [next] | [standalone]

#89512

On 18/06/2016 09:38, fir wrote:

> truly random may be hard (maybe impossible)
> to compress i dont know..

Truly random data probably /can/ be compressed to 1 bit. (Or even 0 
bits. But it will need a flag saying it is truly random; and a length.)

You just generate another random sequence when you decompress it. Who's 
going to know the difference?

If you need the recovered random data to be identical to the original 
for some reason, then I think this can be done for pseudo-random data. 
Then this reduces to an algorithm and some parameters (start state and 
length).

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#89513

W dniu sobota, 18 czerwca 2016 11:41:30 UTC+2 użytkownik Bart napisał:
> On 18/06/2016 09:38, fir wrote:
> 
> > truly random may be hard (maybe impossible)
> > to compress i dont know..
> 
> Truly random data probably /can/ be compressed to 1 bit. (Or even 0 
> bits. But it will need a flag saying it is truly random; and a length.)
> 
> You just generate another random sequence when you decompress it. Who's 
> going to know the difference?
> 
> If you need the recovered random data to be identical to the original 
> for some reason, then I think this can be done for pseudo-random data. 
> Then this reduces to an algorithm and some parameters (start state and 
> length).
> 
thats tru, even normal compresion of random data seem possible If i think of phase space  of all data if it is random then all unrandom cases can be throw out (but it is possible that those unrandom is logarithmicaly tiny bit so this compression is tiny small - but from the same think it
my appear thet nonrandom data may be compressed (sadly on shallow view many nonrandom data may look like mix of random + nonrandom, i dont know the details))

[toc] | [prev] | [next] | [standalone]

#89523

On Saturday, June 18, 2016 at 10:41:30 AM UTC+1, Bart wrote:
>
> If you need the recovered random data to be identical to the original 
> for some reason, then I think this can be done for pseudo-random data. 
> Then this reduces to an algorithm and some parameters (start state and 
> length).
>
Psuedo random data can be compressed more efficiently than almost
anything else. Pass the seed and N

srand(seed);
for(i=0;i<N;i++)
  data[i] = rand();

truly random data can't be compressed at all, but neither does it mean
anything.

[toc] | [prev] | [next] | [standalone]

#89546

"Malcolm McLean" <malcolm.mclean5@btinternet.com> wrote in message 
news:be950b83-2eab-48aa-9616-ccc2174bf61c@googlegroups.com...
> On Saturday, June 18, 2016 at 10:41:30 AM UTC+1, Bart wrote:
>>
>> If you need the recovered random data to be identical to the original
>> for some reason, then I think this can be done for pseudo-random data.
>> Then this reduces to an algorithm and some parameters (start state and
>> length).
>>
> Psuedo random data can be compressed more efficiently than almost
> anything else. Pass the seed and N
>
> srand(seed);
> for(i=0;i<N;i++)
>  data[i] = rand();
>
> truly random data can't be compressed at all, but neither does it mean
> anything.

    It's great that in this subject you bring up C. But as you say "psudo". 
n a *nix type system I always use /dev/random or /dev/urandom and entropy is 
the key to everything. How much can the OS pull off of key pressing, mouse 
moving, CPU cycles, and so on.

    Truely random can't be compressed? If there's a header indicating the 
algorithm of randomness, or encryption why not?

Bill

[toc] | [prev] | [next] | [standalone]

#89547

    Of course I'm assuming randomness for encryption purpose too.

Bill

[toc] | [prev] | [next] | [standalone]

#89552

On Saturday, June 18, 2016 at 2:11:37 PM UTC-5, Bill Cunningham wrote:

>     Truely random can't be compressed? If there's a header indicating the 
> algorithm of randomness, or encryption why not?

I think it's a matter of definitions. If there's
an algorithm, then we call it Deterministic or
Pseudo-Random. Since "pseudo" comes from the Greek
for "false", the qualifier "true" as in "truly random"
is commonly intended to mean precisely that there
is no (known) algorithm for modelling the phenomena
in question and any measurements taken from it are
considered "random".

[toc] | [prev] | [next] | [standalone]

#89553

On 18/06/16 20:11, Bill Cunningham wrote:
<snip>
>
>      Truely random can't be compressed?

Right.

> If there's a header indicating the
> algorithm of randomness,  or encryption why not?

If you can generate it via an algorithm, it isn't truly random.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#89554

On 6/18/2016 12:43 PM, Richard Heathfield wrote:
> On 18/06/16 20:11, Bill Cunningham wrote:
> <snip>
>>
>>      Truely random can't be compressed?
>
> Right.
>
>> If there's a header indicating the
>> algorithm of randomness,  or encryption why not?
>
> If you can generate it via an algorithm, it isn't truly random.
>


Can a given, lets say, 32 bytes of truly random data exist as a sequence 
within some fractal, somewhere over the rainbow?

;^)

[toc] | [prev] | [next] | [standalone]

#89687

On 6/18/2016 12:53 PM, Chris M. Thomasson wrote:
> On 6/18/2016 12:43 PM, Richard Heathfield wrote:
>> On 18/06/16 20:11, Bill Cunningham wrote:
>> <snip>
>>>
>>>      Truely random can't be compressed?
>>
>> Right.
>>
>>> If there's a header indicating the
>>> algorithm of randomness,  or encryption why not?
>>
>> If you can generate it via an algorithm, it isn't truly random.
>>
>
>
> Can a given, lets say, 32 bytes of truly random data exist as a sequence
> within some fractal, somewhere over the rainbow?

What about random data within a certain range? Say 0-9... So, this 
random data is in a sequence like:

[0, 3, 4, 1, 2, …]

Ahh, the symbols are confined to 0-9, surely some sort of compression 
can most likely be applied because we have many more symbols to work 
with on a per-byte basis? Perhaps converting to another base might be 
useful wrt using up some of those unused symbols.

;^o


Hummm... Although, I think encryption just might be able to be used to 
create an abundance of patterns, that might be compressible...

Again, this is in the context of, over the rainbow type shi%!

Is 100% total non-sense, or perhaps 99.999...%?

;^)

[toc] | [prev] | [next] | [standalone]

#89559

Richard Heathfield <rjh@cpax.org.uk> writes:
> On 18/06/16 20:11, Bill Cunningham wrote:
> <snip>
>>
>>      Truely random can't be compressed?
>
> Right.
>
>> If there's a header indicating the
>> algorithm of randomness,  or encryption why not?
>
> If you can generate it via an algorithm, it isn't truly random.

Right.  If a sequence is truly random, then there is no way, given the
first N-1 bits of the sequence to predict the N'th bit with better than
50% accuracy.

There are pseudo-random sequences that can't *practically* be
compressed, because the compression algorithm will be unable to detect
the algorithm and parameters that went into it.  You can compress the
first million digits of pi in base 10 to the string "the first million
digits of pi in base 10", but only if your algorithm recognizes it (and
at the cost of not being able to use that particular string to represent
any other sequence).

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#89564

On 6/18/2016 4:22 PM, Keith Thompson wrote:
> Richard Heathfield <rjh@cpax.org.uk> writes:
>> On 18/06/16 20:11, Bill Cunningham wrote:
>> <snip>
>>>
>>>      Truely random can't be compressed?
>>
>> Right.
>>
>>> If there's a header indicating the
>>> algorithm of randomness,  or encryption why not?
>>
>> If you can generate it via an algorithm, it isn't truly random.
> 
> Right.  If a sequence is truly random, then there is no way, given the
> first N-1 bits of the sequence to predict the N'th bit with better than
> 50% accuracy.
> 
> There are pseudo-random sequences that can't *practically* be
> compressed, because the compression algorithm will be unable to detect
> the algorithm and parameters that went into it.  You can compress the
> first million digits of pi in base 10 to the string "the first million
> digits of pi in base 10", but only if your algorithm recognizes it (and
> at the cost of not being able to use that particular string to represent
> any other sequence).
> 

That is true - you can't predict the next value in a random sequence.
However, once a sequence is generated, it is possible to create an
algorithm which matches that sequence.

The problem is the algorithm may be longer than the original sequence.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#89591

On 6/18/2016 4:35 PM, Jerry Stuckle wrote:
> On 6/18/2016 4:22 PM, Keith Thompson wrote:
>> Richard Heathfield <rjh@cpax.org.uk> writes:
>>> On 18/06/16 20:11, Bill Cunningham wrote:
>>> <snip>
>>>>
>>>>      Truely random can't be compressed?
>>>
>>> Right.
>>>
>>>> If there's a header indicating the
>>>> algorithm of randomness,  or encryption why not?
>>>
>>> If you can generate it via an algorithm, it isn't truly random.
>>
>> Right.  If a sequence is truly random, then there is no way, given the
>> first N-1 bits of the sequence to predict the N'th bit with better than
>> 50% accuracy.
>>
>> There are pseudo-random sequences that can't *practically* be
>> compressed, because the compression algorithm will be unable to detect
>> the algorithm and parameters that went into it.  You can compress the
>> first million digits of pi in base 10 to the string "the first million
>> digits of pi in base 10", but only if your algorithm recognizes it (and
>> at the cost of not being able to use that particular string to represent
>> any other sequence).
>>
> 
> That is true - you can't predict the next value in a random sequence.
> However, once a sequence is generated, it is possible to create an
> algorithm which matches that sequence.
> 
> The problem is the algorithm may be longer than the original sequence.
> 

A perfect example of this right now is my wife's Windows machine.  She
has a problem on it, and I'm going to have to try to recover it (if not,
I'll have to rebuild from scratch).

I've taken disk images of the three partitions on her drive (boot,
recovery and live).  The recovery partition (which already contains
compressed data) is about 2.8GB.  However, the "compressed" size with
bzip2 -9 (maximum compression) is 12.5GB with bzip2 and 12.6GB with gzip
-9.  "Compressed" file is > 3x the original file size.

Could it be compressed further?  Maybe.  But the amount of time required
to perform that compression would be much longer than would be
worthwhile.  Should it be that much larger?  Probably not, but what
would it take to compress it further?

The bottom line here is - there's plenty of space on the NAS, so the
size isn't as important as the processing time it takes to compress the
image.

-- 
==================
Remove the "x" from my email address
Jerry Stuckle
jstucklex@attglobal.net
==================

[toc] | [prev] | [next] | [standalone]

#89569

On 06/18/2016 04:22 PM, Keith Thompson wrote:
> Richard Heathfield <rjh@cpax.org.uk> writes:
>> On 18/06/16 20:11, Bill Cunningham wrote:
>> <snip>
>>>
>>>      Truely random can't be compressed?
>>
>> Right.

That's true, but only on average, and it's not clear to me whether Bill
understands that fact. He probably doesn't, and probably won't even
after I explain it, but I'll give it a try.

A compression algorithm maps a set of input files to a set of output
files, and if the algorithm is lossless, any randomly selected member of
the first set will most likely be mapped to an output file that has, at
best, the same exact size. The usefulness of a compression algorithm
depends upon the fact that the files that are chosen for compression are
not selected at random from among all possible files - a useful
compression algorithm compresses the files most likely to be selected to
small sizes, at the expense of achieving no compression at all on less
likely input files, and negative amounts of compression on the least
likely input files.

Any source of truly random data must have a finite, (if very small)
chance of generating a file that happens to fall into the set that a
given algorithm compresses very efficiently. However, that happens so
infrequently that, on average, a truly random file will not be
compressed at all.

[toc] | [prev] | [next] | [standalone]

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

Back to top | Article view | comp.lang.c

csiph-web