Groups > comp.lang.python > #58077 > unrolled thread

Algorithm that makes maximum compression of completly diffused data.

Started by	jonas.thornvall@gmail.com
First post	2013-10-30 11:21 -0700
Last post	2013-11-03 04:50 -0500
Articles	12 on this page of 72 — 22 participants

Back to article view | Back to comp.lang.python

Page 4 of 4 — ← Prev page 1 2 3 [4]

#58753

From	Steven D'Aprano <steve+comp.lang.python@pearwood.info>
Date	2013-11-08 05:32 +0000
Message-ID	<527c777e$0$29983$c3e8da3$5496439d@news.astraweb.com>
In reply to	#58733

On Thu, 07 Nov 2013 18:25:18 -0800, jonas.thornvall wrote:

> Please, you are he obnoxious, so fuck off

Pot, meet kettle.

> I am not sure if it is just stupidness or laziness that prevent you from
> seeing that 4^8=65536.

And what is it that prevents you from seeing that 4**8=65536 is 
irrelevant to the compression of random data?

All your talk about "reformulation of problems", "infinite number of 
arithmetical solutions" and "generic arithmetic encoding" is just a smoke 
screen to distract from the fact that you don't, in fact, have a 
compression algorithm that can losslessly compress arbitrary random data 
of any length.

I am as sure as this as I am sure you don't have a perpetual motion 
machine, a method for squaring the circle using only a compass and 
straight-edge, or a fool-proof system for winning the lottery every week 
without any chance of failure.

-- 
Steven

[toc] | [prev] | [next] | [standalone]

#58732

From	Mark Janssen <dreamingforward@gmail.com>
Date	2013-11-07 18:24 -0800
Message-ID	<mailman.2172.1383877492.18130.python-list@python.org>
In reply to	#58727

On Thu, Nov 7, 2013 at 6:17 PM, Chris Angelico <rosuav@gmail.com> wrote:
> On Fri, Nov 8, 2013 at 1:05 PM,  <jonas.thornvall@gmail.com> wrote:
>> I guess what matter is how fast an algorithm can encode and decode a big number, at least if you want to use it for very big sets of random data, or losless video compression?
>
> I don't care how fast. I care about the laws of physics :) You can't
> stuff more data into less space without losing some of it.

Technically, the universe could expand temporarily or reconfigure to
allow it; the question is who or what will have to shift out to allow
it?

-- 
MarkJ
Tacoma, Washington

[toc] | [prev] | [next] | [standalone]

#58757 — Re: Algorithm that makes maximum compression of completly diffused data.

From	Gregory Ewing <greg.ewing@canterbury.ac.nz>
Date	2013-11-08 20:16 +1300
Subject	Re: Algorithm that makes maximum compression of completly diffused data.
Message-ID	<be3hfaFr846U1@mid.individual.net>
In reply to	#58732

Mark Janssen wrote:
> Technically, the universe could expand temporarily or reconfigure to
> allow it;

Oh, no! If Jonas ever succeeds in getting his algorithm to
work, the Void will expand and swallow us all!

http://en.wikipedia.org/wiki/The_Dreaming_Void

-- 
Greg

[toc] | [prev] | [next] | [standalone]

#58735

From	Chris Angelico <rosuav@gmail.com>
Date	2013-11-08 13:27 +1100
Message-ID	<mailman.2174.1383877641.18130.python-list@python.org>
In reply to	#58727

On Fri, Nov 8, 2013 at 1:24 PM, Mark Janssen <dreamingforward@gmail.com> wrote:
> On Thu, Nov 7, 2013 at 6:17 PM, Chris Angelico <rosuav@gmail.com> wrote:
>> On Fri, Nov 8, 2013 at 1:05 PM,  <jonas.thornvall@gmail.com> wrote:
>>> I guess what matter is how fast an algorithm can encode and decode a big number, at least if you want to use it for very big sets of random data, or losless video compression?
>>
>> I don't care how fast. I care about the laws of physics :) You can't
>> stuff more data into less space without losing some of it.
>
> Technically, the universe could expand temporarily or reconfigure to
> allow it; the question is who or what will have to shift out to allow
> it?

... okay, I bow to your superior power. If you can make the very
*UNIVERSE* bow to your compression algorithm, I have to admit defeat.
:D

ChrisA

[toc] | [prev] | [next] | [standalone]

#58363

From	Ethan Furman <ethan@stoneleaf.us>
Date	2013-11-02 21:26 -0700
Message-ID	<mailman.1967.1383453986.18130.python-list@python.org>
In reply to	#58103

On 10/30/2013 01:32 PM, Gene Heskett wrote:
>
> Congratulations Jonas.  My kill file for this list used to have only one
> name, but now has 2.

You have more patience than I!  Jonas just made mine seven.  :)

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#58376

From	Mark Janssen <dreamingforward@gmail.com>
Date	2013-11-02 23:09 -0700
Message-ID	<mailman.1969.1383461687.18130.python-list@python.org>
In reply to	#58103

>> Congratulations Jonas.  My kill file for this list used to have only one
>> name, but now has 2.
>
> You have more patience than I!  Jonas just made mine seven.  :)

Gosh, don't kill the guy.  It's not an obvious thing to hardly anyone
but computer scientists.  It's an easy mistake to make.

-- 
MarkJ
Tacoma, Washington

[toc] | [prev] | [next] | [standalone]

#58398

From	Michael Torrie <torriem@gmail.com>
Date	2013-11-03 08:14 -0700
Message-ID	<mailman.1985.1383491688.18130.python-list@python.org>
In reply to	#58103

On 11/03/2013 12:09 AM, Mark Janssen wrote:
>>> Congratulations Jonas.  My kill file for this list used to have only one
>>> name, but now has 2.
>>
>> You have more patience than I!  Jonas just made mine seven.  :)
> 
> Gosh, don't kill the guy.  It's not an obvious thing to hardly anyone
> but computer scientists.  It's an easy mistake to make.

I don't think he's being plonked for not understanding computational
theory.  He's being plonked for resorting to name calling on his second
post!  If he was a smart computer scientist type, then engaging in a
discussion about the theoretical aspects of his algorithm would have
been welcomed by him, because that's what science is all about.  But he
failed that early on.

Thanks to everyone in this part of the thread for turning this
ridiculous farce into a really educational discussion on the theory of
information compression.  Too bad the OP has tuned out a long time ago.

[toc] | [prev] | [next] | [standalone]

#58104

From	jonas.thornvall@gmail.com
Date	2013-10-30 12:49 -0700
Message-ID	<c0d6a652-6a86-48ce-a48b-79b930cbb58c@googlegroups.com>
In reply to	#58101

Den onsdagen den 30:e oktober 2013 kl. 20:46:57 UTC+1 skrev Modulok:
> On Wed, Oct 30, 2013 at 12:21 PM,  <jonas.t...@gmail.com> wrote:
> 
> 
> 
> I am searching for the program or algorithm that makes the best possible of completly (diffused data/random noise) and wonder what the state of art compression is.
> 
> 
> 
> 
> I understand this is not the correct forum but since i think i have an algorithm that can do this very good, and do not know where to turn for such question i was thinking to start here.
> 
> 
> 
> It is of course lossless compression i am speaking of.
> 
> --
> 
> https://mail.python.org/mailman/listinfo/python-list
> 
> 
>  
> 
> >> I am searching for the program or algorithm that makes the best possible of
> >> completly (diffused data/random noise) and wonder what the state of art
> 
> >> compression is.
> 
> 
> None. If the data to be compressed is truly homogeneous, random noise as you
> describe (for example a 100mb file read from cryptographically secure random
> 
> bit generator such as /dev/random on *nix systems), the state-of-the-art
> lossless compression is zero and will remain that way for the foreseeable
> 
> future.
> 
> 
> There is no lossless algorithm that will reduce truly random (high entropy)
> data by any significant margin. In classical information theory, such an
> 
> algorithm can never be invented. See: Kolmogorov complexity
> 
> 
> Real world data is rarely completely random. You would have to test various
> 
> algorithms on the data set in question. Small things such as non-obvious
> statistical clumping can make a big difference in the compression ratio from
> 
> one algorithm to another. Data that might look "random", might not actually be
> random in the entropy sense of the word.
> 
> 
> 
> >> I understand this is not the correct forum but since i think i have an
> >> algorithm that can do this very good, and do not know where to turn for such
> 
> >> question i was thinking to start here.
> 
> 
> Not to sound like a downer, but I would wager that the data you're testing your
> 
> algorithm on is not as truly random as you imply or is not a large enough body
> of test data to draw such conclusions from. It's akin to inventing a perpetual
> 
> motion machine or an inertial propulsion engine or any other classically
> impossible solutions. (This only applies to truly random data.)
> 
> 
> 
> -Modulok-

My algorithm will compress data from any random data source.

[toc] | [prev] | [next] | [standalone]

#58122

From	Grant Edwards <invalid@invalid.invalid>
Date	2013-10-30 21:18 +0000
Message-ID	<l4rt3c$f0t$1@reader1.panix.com>
In reply to	#58077

On 2013-10-30, jonas.thornvall@gmail.com <jonas.thornvall@gmail.com> wrote:

> I am searching for the program or algorithm that makes the best
> possible of completly (diffused data/random noise) and wonder what
> the state of art compression is.

[...]

> It is of course lossless compression i am speaking of.

For completely random noise, the CAT compression algorithm will
acheive the maximum theoretical result.  It's been available on Unix
systems for decades via the "cat" command.

It's also trivial to implement in your own code if you desire.

-- 
Grant Edwards               grant.b.edwards        Yow! I put aside my copy
                                  at               of "BOWLING WORLD" and
                              gmail.com            think about GUN CONTROL
                                                   legislation...

[toc] | [prev] | [next] | [standalone]

#58125

From	Mark Janssen <dreamingforward@gmail.com>
Date	2013-10-30 14:26 -0700
Message-ID	<mailman.1849.1383168371.18130.python-list@python.org>
In reply to	#58077

On Wed, Oct 30, 2013 at 11:21 AM,  <jonas.thornvall@gmail.com> wrote:
> I am searching for the program or algorithm that makes the best possible of completly (diffused data/random noise) and wonder what the state of art compression is.

Is this an April Fool's Joke?  A key idea of "completely" random is
that you *can't* compress it.
-- 
MarkJ
Tacoma, Washington

[toc] | [prev] | [next] | [standalone]

#58147

From	Dave Angel <davea@davea.name>
Date	2013-10-31 03:22 +0000
Message-ID	<mailman.1865.1383189784.18130.python-list@python.org>
In reply to	#58077

On 30/10/2013 14:21, jonas.thornvall@gmail.com wrote:

> I am searching for the program or algorithm that makes the best possible of completly (diffused data/random noise) and wonder what the state of art compression is.
>
> I understand this is not the correct forum but since i think i have an algorithm that can do this very good, and do not know where to turn for such question i was thinking to start here.
>
> It is of course lossless compression i am speaking of.

See http://gailly.net/05533051.html for a discussion of a patent on a
similarly ludicrous "algorithm."

Maybe you can hoodwink the patent office into granting you one as well.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#58390

From	Gene Heskett <gheskett@wdtv.com>
Date	2013-11-03 04:50 -0500
Message-ID	<mailman.1980.1383477755.18130.python-list@python.org>
In reply to	#58077

On Sunday 03 November 2013 04:40:45 Ethan Furman did opine:

> On 10/30/2013 01:32 PM, Gene Heskett wrote:
> > Congratulations Jonas.  My kill file for this list used to have only
> > one name, but now has 2.
> 
> You have more patience than I!  Jonas just made mine seven.  :)
> 
> --
> ~Ethan~

Yeah, well there are a couple others in the mugwump category here yet.  I 
lurk here to try and learn, and baseless arguments are just noise. To be 
filtered.  And its working!

But it may be that this old dog has learned his last "new" trick in the 
language arena too, too many "irons in the fire", and fine tuning machinery 
to run the GCode I write to carve metal or wood is the primary interest 
ATM.  At 79yo, the short term memory needs help.  I'm smart enough to 
understand that, but it doesn't mean I like it.  Its a right PIMA TBE.

Cheers, Gene
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)

All the evidence concerning the universe has not yet been collected,
so there's still hope.
A pen in the hand of this president is far more
dangerous than 200 million guns in the hands of
         law-abiding citizens.

[toc] | [prev] | [standalone]

Page 4 of 4 — ← Prev page 1 2 3 [4]

csiph-web

Algorithm that makes maximum compression of completly diffused data.

Contents

#58753

#58732

#58757 — Re: Algorithm that makes maximum compression of completly diffused data.

#58735

#58363

#58376

#58398

#58104

#58122

#58125

#58147

#58390