Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #58101

Re: Algorithm that makes maximum compression of completly diffused data.

References <205bfa4f-29de-43de-be5a-72a12d77d0c9@googlegroups.com>
Date 2013-10-30 13:46 -0600
Subject Re: Algorithm that makes maximum compression of completly diffused data.
From Modulok <modulok@gmail.com>
Newsgroups comp.lang.python
Message-ID <mailman.1835.1383162420.18130.python-list@python.org> (permalink)

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw

On Wed, Oct 30, 2013 at 12:21 PM, <jonas.thornvall@gmail.com> wrote:

> I am searching for the program or algorithm that makes the best possible
> of completly (diffused data/random noise) and wonder what the state of art
> compression is.
>
> I understand this is not the correct forum but since i think i have an
> algorithm that can do this very good, and do not know where to turn for
> such question i was thinking to start here.
>
> It is of course lossless compression i am speaking of.
> --
> https://mail.python.org/mailman/listinfo/python-list



>> I am searching for the program or algorithm that makes the best possible
of
>> completly (diffused data/random noise) and wonder what the state of art
>> compression is.

None. If the data to be compressed is truly homogeneous, random noise as you
describe (for example a 100mb file read from cryptographically secure random
bit generator such as /dev/random on *nix systems), the state-of-the-art
lossless compression is zero and will remain that way for the foreseeable
future.

There is no lossless algorithm that will reduce truly random (high entropy)
data by any significant margin. In classical information theory, such an
algorithm can never be invented. See: Kolmogorov complexity

Real world data is rarely completely random. You would have to test various
algorithms on the data set in question. Small things such as non-obvious
statistical clumping can make a big difference in the compression ratio from
one algorithm to another. Data that might look "random", might not actually
be
random in the entropy sense of the word.

>> I understand this is not the correct forum but since i think i have an
>> algorithm that can do this very good, and do not know where to turn for
such
>> question i was thinking to start here.

Not to sound like a downer, but I would wager that the data you're testing
your
algorithm on is not as truly random as you imply or is not a large enough
body
of test data to draw such conclusions from. It's akin to inventing a
perpetual
motion machine or an inertial propulsion engine or any other classically
impossible solutions. (This only applies to truly random data.)

-Modulok-

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 11:21 -0700
  Re: Algorithm that makes maximum compression of completly diffused data. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-30 18:53 +0000
    Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 12:01 -0700
      Re: Algorithm that makes maximum compression of completly diffused data. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-30 19:18 +0000
        Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 12:22 -0700
          Re: Algorithm that makes maximum compression of completly diffused data. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-30 19:31 +0000
        Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 12:23 -0700
          Re: Algorithm that makes maximum compression of completly diffused data. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-30 19:35 +0000
          Re: Algorithm that makes maximum compression of completly diffused data. Ethan Furman <ethan@stoneleaf.us> - 2013-11-02 21:26 -0700
      Re: Algorithm that makes maximum compression of completly diffused data. Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2013-10-30 20:28 +0100
      Re: Algorithm that makes maximum compression of completly diffused data. Joshua Landau <joshua@landau.ws> - 2013-10-30 21:30 +0000
        Re: Algorithm that makes maximum compression of completly diffused data. rusi <rustompmody@gmail.com> - 2013-10-31 05:54 -0700
      Re: Algorithm that makes maximum compression of completly diffused data. Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-10-30 21:52 +0000
      Re: Algorithm that makes maximum compression of completly diffused data. Tim Chase <python.list@tim.thechases.com> - 2013-10-30 18:01 -0500
      Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-10-31 10:41 +1100
  Re: Algorithm that makes maximum compression of completly diffused data. Dan Stromberg <drsalists@gmail.com> - 2013-10-30 12:29 -0700
  Re: Algorithm that makes maximum compression of completly diffused data. Tim Delaney <timothy.c.delaney@gmail.com> - 2013-10-31 06:35 +1100
    Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 12:47 -0700
  Re: Algorithm that makes maximum compression of completly diffused data. Modulok <modulok@gmail.com> - 2013-10-30 13:46 -0600
    Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 12:47 -0700
      Re: Algorithm that makes maximum compression of completly diffused data. Gene Heskett <gheskett@wdtv.com> - 2013-10-30 16:32 -0400
      Re: Algorithm that makes maximum compression of completly diffused data. Tim Roberts <timr@probo.com> - 2013-11-02 14:31 -0700
        Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-11-02 14:37 -0700
        Re: Algorithm that makes maximum compression of completly diffused data. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-11-03 03:17 +0000
          Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-03 15:10 +1100
          Re: Algorithm that makes maximum compression of completly diffused data. Joshua Landau <joshua@landau.ws> - 2013-11-03 15:34 +0000
          Re: Algorithm that makes maximum compression of completly diffused data. Joshua Landau <joshua@landau.ws> - 2013-11-03 15:51 +0000
          Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-11-03 19:40 -0800
          Re: Algorithm that makes maximum compression of completly diffused data. Tim Chase <python.list@tim.thechases.com> - 2013-11-04 07:08 -0600
        Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-11-04 05:53 -0800
          Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-11-04 06:00 -0800
          Re: Algorithm that makes maximum compression of completly diffused
 data. Dave Angel <davea@davea.name> - 2013-11-04 08:27 -0600
            Re: Algorithm that makes maximum compression of completly diffused data. rusi <rustompmody@gmail.com> - 2013-11-04 06:46 -0800
            Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-11-04 14:34 -0800
              Re: Algorithm that makes maximum compression of completly diffused
 data. Dave Angel <davea@davea.name> - 2013-11-04 19:29 -0600
              Re: Algorithm that makes maximum compression of completly diffused data. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-11-05 04:33 +0000
                Re: Algorithm that makes maximum compression of completly diffused data. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-11-05 04:36 +0000
          Re: Algorithm that makes maximum compression of completly diffused data. Tim Roberts <timr@probo.com> - 2013-11-07 00:05 -0800
            Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-11-07 10:59 -0800
            Re: Algorithm that makes maximum compression of completly diffused data. Tim Roberts <timr@probo.com> - 2013-11-07 11:22 -0800
            Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 09:26 +1100
              Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-11-07 18:05 -0800
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 13:17 +1100
                Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-11-07 18:25 -0800
                Re: Algorithm that makes maximum compression of completly diffused data. rusi <rustompmody@gmail.com> - 2013-11-07 18:36 -0800
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 13:36 +1100
                Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-11-07 18:43 -0800
                Re: Algorithm that makes maximum compression of completly diffused data. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-11-08 04:47 +0000
                Re: Algorithm that makes maximum compression of completly diffused   data. Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2013-11-08 20:09 +1300
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 18:21 +1100
                Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-11-08 07:48 -0800
                Re: Algorithm that makes maximum compression of completly diffused data. rusi <rustompmody@gmail.com> - 2013-11-08 07:57 -0800
                Re: Algorithm that makes maximum compression of completly diffused data. Ian Kelly <ian.g.kelly@gmail.com> - 2013-11-08 11:48 -0700
                Re: Algorithm that makes maximum compression of completly diffused data. "R. Michael Weylandt <michael.weylandt@gmail.com>" <michael.weylandt@gmail.com> - 2013-11-07 21:43 -0500
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 14:05 +1100
                Re: Algorithm that makes maximum compression of completly diffused data. Roy Smith <roy@panix.com> - 2013-11-07 22:08 -0500
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 14:24 +1100
                Re: Algorithm that makes maximum compression of completly diffused data. "R. Michael Weylandt <michael.weylandt@gmail.com>" <michael.weylandt@gmail.com> - 2013-11-07 23:05 -0500
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 15:06 +1100
                Re: Algorithm that makes maximum compression of completly diffused
 data. Dave Angel <davea@davea.name> - 2013-11-07 22:12 -0600
                Re: Algorithm that makes maximum compression of completly diffused data. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-11-08 05:32 +0000
                Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-11-07 18:24 -0800
                Re: Algorithm that makes maximum compression of completly diffused   data. Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2013-11-08 20:16 +1300
                Re: Algorithm that makes maximum compression of completly diffused data. Chris Angelico <rosuav@gmail.com> - 2013-11-08 13:27 +1100
      Re: Algorithm that makes maximum compression of completly diffused data. Ethan Furman <ethan@stoneleaf.us> - 2013-11-02 21:26 -0700
      Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-11-02 23:09 -0700
      Re: Algorithm that makes maximum compression of completly diffused data. Michael Torrie <torriem@gmail.com> - 2013-11-03 08:14 -0700
    Re: Algorithm that makes maximum compression of completly diffused data. jonas.thornvall@gmail.com - 2013-10-30 12:49 -0700
  Re: Algorithm that makes maximum compression of completly diffused data. Grant Edwards <invalid@invalid.invalid> - 2013-10-30 21:18 +0000
  Re: Algorithm that makes maximum compression of completly diffused data. Mark Janssen <dreamingforward@gmail.com> - 2013-10-30 14:26 -0700
  Re: Algorithm that makes maximum compression of completly diffused data. Dave Angel <davea@davea.name> - 2013-10-31 03:22 +0000
  Re: Algorithm that makes maximum compression of completly diffused data. Gene Heskett <gheskett@wdtv.com> - 2013-11-03 04:50 -0500

csiph-web