Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #89274 > unrolled thread
| Started by | fir <profesor.fir@gmail.com> |
|---|---|
| First post | 2016-06-16 07:25 -0700 |
| Last post | 2016-06-20 21:25 -0700 |
| Articles | 20 on this page of 88 — 24 participants |
Back to article view | Back to comp.lang.c
[sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 07:25 -0700
Re: [sOT] compression 1 GB downto 100 MB mark.bluemel@gmail.com - 2016-06-16 08:08 -0700
Re: [sOT] compression 1 GB downto 100 MB Lynn McGuire <lmc@winsim.com> - 2016-06-17 12:04 -0500
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-17 10:16 -0700
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 23:08 -0700
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-16 08:09 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:15 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:18 -0700
Re: [sOT] compression 1 GB downto 100 MB mark.bluemel@gmail.com - 2016-06-17 01:28 -0700
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-17 02:03 -0700
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-17 10:34 +0100
Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-17 10:44 +0100
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-17 11:03 +0100
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-17 03:51 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 02:16 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 02:29 -0700
Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-17 08:19 -0500
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 03:17 -0700
Re: [sOT] compression 1 GB downto 100 MB Gareth Owen <gwowen@gmail.com> - 2016-06-17 18:32 +0100
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-16 08:43 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:48 -0700
Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-16 10:49 -0500
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 09:10 -0700
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-16 09:36 -0700
Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-16 11:18 -0700
Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-16 13:13 -0500
Re: [sOT] compression 1 GB downto 100 MB Gareth Owen <gwowen@gmail.com> - 2016-06-16 20:18 +0100
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-16 22:07 +0100
Re: [sOT] compression 1 GB downto 100 MB Gareth Owen <gwowen@gmail.com> - 2016-06-17 06:04 +0100
Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-16 23:21 -0500
Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-17 07:49 -0500
Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-17 12:47 -0500
Re: [sOT] compression 1 GB downto 100 MB "Osmium" <r124c4u102@comcast.net> - 2016-06-17 14:47 -0500
Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-17 23:32 -0500
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 01:33 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-17 14:51 -0700
Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-16 16:50 +0100
Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-17 16:20 +0100
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-16 08:34 -0700
Re: [sOT] compression 1 GB downto 100 MB Les Cargill <lcargill99@comcast.com> - 2016-06-16 12:55 -0500
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-16 11:53 -0700
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-16 12:14 -0700
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 12:33 -0700
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 12:46 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-17 15:04 -0700
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-17 22:55 -0700
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 01:38 -0700
Re: [sOT] compression 1 GB downto 100 MB BartC <bc@freeuk.com> - 2016-06-18 10:41 +0100
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-18 02:55 -0700
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-18 05:08 -0700
Re: [sOT] compression 1 GB downto 100 MB "Bill Cunningham" <nospam@nspam.invalid> - 2016-06-18 15:11 -0400
Re: [sOT] compression 1 GB downto 100 MB "Bill Cunningham" <nospam@nspam.invalid> - 2016-06-18 15:13 -0400
Re: [sOT] compression 1 GB downto 100 MB luser droog <luser.droog@gmail.com> - 2016-06-18 12:43 -0700
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-18 20:43 +0100
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-18 12:53 -0700
Re: [sOT] compression 1 GB downto 100 MB "Chris M. Thomasson" <nospam@nospam.invalid> - 2016-06-20 20:32 -0700
Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-18 13:22 -0700
Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 16:35 -0400
Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 21:37 -0400
Re: [sOT] compression 1 GB downto 100 MB "James R. Kuyper" <jameskuyper@verizon.net> - 2016-06-18 16:59 -0400
Re: [sOT] compression 1 GB downto 100 MB crisdunbar@gmail.com - 2016-06-20 09:55 -0700
Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-20 21:22 +0100
Re: [sOT] compression 1 GB downto 100 MB cris <crisdunbar@gmail.com> - 2016-06-20 17:13 -0700
Re: [sOT] compression 1 GB downto 100 MB Tim Rentsch <txr@alumni.caltech.edu> - 2016-06-21 10:42 -0700
Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-21 20:09 +0100
Re: [sOT] compression 1 GB downto 100 MB crisdunbar@gmail.com - 2016-06-21 13:39 -0700
Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-22 08:34 -0700
Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-22 19:48 +0100
Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-19 01:37 +0100
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-19 02:22 +0100
Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 21:30 -0400
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-19 02:41 +0100
Re: [sOT] compression 1 GB downto 100 MB Jerry Stuckle <jstucklex@attglobal.net> - 2016-06-18 21:55 -0400
Re: [sOT] compression 1 GB downto 100 MB Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-06-19 03:20 +0100
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-19 02:53 -0700
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-18 16:05 -0700
Re: [sOT] compression 1 GB downto 100 MB "J. Clarke" <j.clarke.873638@gmail.com> - 2016-06-19 06:59 -0400
Re: [sOT] compression 1 GB downto 100 MB fir <profesor.fir@gmail.com> - 2016-06-19 05:55 -0700
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-19 06:13 -0700
Re: [sOT] compression 1 GB downto 100 MB "J. Clarke" <j.clarke.873638@gmail.com> - 2016-06-26 12:26 -0400
Re: [sOT] compression 1 GB downto 100 MB Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-06-26 09:53 -0700
Re: [sOT] compression 1 GB downto 100 MB Richard Heathfield <rjh@cpax.org.uk> - 2016-06-18 20:41 +0100
Re: [sOT] compression 1 GB downto 100 MB "J. Clarke" <j.clarke.873638@gmail.com> - 2016-06-19 06:45 -0400
Re: [sOT] compression 1 GB downto 100 MB Robert Wessel <robertwessel2@yahoo.com> - 2016-06-18 14:42 -0500
Re: [sOT] compression 1 GB downto 100 MB gordonb.vnsk1@burditt.org (Gordon Burditt) - 2016-06-20 01:13 -0500
Re: [sOT] compression 1 GB downto 100 MB Keith Thompson <kst-u@mib.org> - 2016-06-20 12:16 -0700
Re: [sOT] compression 1 GB downto 100 MB David Brown <david.brown@hesbynett.no> - 2016-06-20 21:27 +0200
Re: [sOT] compression 1 GB downto 100 MB Xaltonon <xaltonon@gmail.com> - 2016-06-20 21:25 -0700
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
| From | fir <profesor.fir@gmail.com> |
|---|---|
| Date | 2016-06-16 08:48 -0700 |
| Message-ID | <cf3610b5-c593-41b9-be6a-791570773ff3@googlegroups.com> |
| In reply to | #89287 |
W dniu czwartek, 16 czerwca 2016 17:43:23 UTC+2 użytkownik Malcolm McLean napisał: > On Thursday, June 16, 2016 at 4:15:42 PM UTC+1, fir wrote: > > > > I dont think "The only way" is proven - this is probably just false and I wouldnt be maybe > > surprised if there is a way to compres .mpg's 10 times yet > > (but i dont know the details on compression never worked in it) If someonne knows maybe > > what is a source of this impossibility (of compressing .mpgs or .iso 10 times) it would be good > > to discuss it maybe > > > To compress, you have to be able to identify some redundancy. Some way of predicting > what the next byte will be from the previous bytes. English text has plenty of such > redundancy, and so have movies - if an image shows grass, the next pixel is also highly > likely to be green, and the same place in the next frame is also highly likely to also > be grass. But MPEG identifies all that redundancy, and strips it out until quality is only > just noticeably affected. > Your options are a) identify some redundancy which everyone else has failed to spot, > b) reduce quality, or c) uses some sort of look-up system. im not sure if those are all options, reduce dquality is not in topic, as to failed to spot i think they fail to spot a lot
[toc] | [prev] | [next] | [standalone]
| From | "Osmium" <r124c4u102@comcast.net> |
|---|---|
| Date | 2016-06-16 10:49 -0500 |
| Message-ID | <dsg04aFn010U1@mid.individual.net> |
| In reply to | #89282 |
"fir" wrote: W dniu czwartek, 16 czerwca 2016 17:09:24 UTC+2 użytkownik Malcolm McLean napisał: > On Thursday, June 16, 2016 at 3:25:42 PM UTC+1, fir wrote: > > GOD how great it would be have an algorith that would compress totays > > data which rae not much > > compressibl;e (like game ISo files, movies, zip archives) like many > > times, say ten times > > > > wouldnt it be great? do you think that it is > > possible? (im sure it is not really obtainable but other thing if this > > is not impossible 'in fact') > > > > Imagine how great it would be :U > > > Unfortunately there are theoretical limits on how far you can compress > data without prior > information about what that data represents. > Movies, for example, are almost certainly already MPEG compressed. That > strips out > essentially all of the redundancy. The only way of getting better > compression is to have > a database of all movie clips ever produced (that's not necessarily > impractical) and > to replace the movies with a reference to that database. > You can't just feed the MPEG stream to a general purpose compressor and > expect it > to compress it. If it were that easy the general purpose compressor would > have already > been included in the MPEG standard, and in fact it is - MPEG creates a > Huffman-friendly > output stream then uses a fairly standard Huffman compressor to achieve > the final bit > reduction. I dont think "The only way" is proven - this is probably just false and I wouldnt be maybe surprised if there is a way to compres .mpg's 10 times yet (but i dont know the details on compression never worked in it) If someonne knows maybe what is a source of this impossibility (of compressing .mpgs or .iso 10 times) it would be good to discuss it maybe <end google garble> I am not at all sure it is a good idea to respond to one of your scribbling. Nevertheless. Shannon's theory dates to 1948 and it forms a firm theoretical base for the work that has come since. One of your "it" candidates is zip files. No, they can not be compressed further by a factor of 10. Most of the "good" has already been taken out of the data. Movies are more subjective because the compression is loss and therefore how compressible they are is subjective. Surely there is *some* room for improvement. in this area - but you are not going to get ten times on a hockey game, for example. https://en.wikipedia.org/wiki/Information_theory
[toc] | [prev] | [next] | [standalone]
| From | fir <profesor.fir@gmail.com> |
|---|---|
| Date | 2016-06-16 09:10 -0700 |
| Message-ID | <c2beb14d-f710-4018-91a1-d33210a6df03@googlegroups.com> |
| In reply to | #89289 |
W dniu czwartek, 16 czerwca 2016 17:49:42 UTC+2 użytkownik Osmium napisał: > "fir" wrote: > > W dniu czwartek, 16 czerwca 2016 17:09:24 UTC+2 użytkownik Malcolm McLean > napisał: > > On Thursday, June 16, 2016 at 3:25:42 PM UTC+1, fir wrote: > > > GOD how great it would be have an algorith that would compress totays > > > data which rae not much > > > compressibl;e (like game ISo files, movies, zip archives) like many > > > times, say ten times > > > > > > wouldnt it be great? do you think that it is > > > possible? (im sure it is not really obtainable but other thing if this > > > is not impossible 'in fact') > > > > > > Imagine how great it would be :U > > > > > Unfortunately there are theoretical limits on how far you can compress > > data without prior > > information about what that data represents. > > Movies, for example, are almost certainly already MPEG compressed. That > > strips out > > essentially all of the redundancy. The only way of getting better > > compression is to have > > a database of all movie clips ever produced (that's not necessarily > > impractical) and > > to replace the movies with a reference to that database. > > You can't just feed the MPEG stream to a general purpose compressor and > > expect it > > to compress it. If it were that easy the general purpose compressor would > > have already > > been included in the MPEG standard, and in fact it is - MPEG creates a > > Huffman-friendly > > output stream then uses a fairly standard Huffman compressor to achieve > > the final bit > > reduction. > > I dont think "The only way" is proven - this is probably just false and I > wouldnt be maybe surprised if there is a way to compres .mpg's 10 times yet > (but i dont know the details on compression never worked in it) If someonne > knows maybe what is a source of this impossibility (of compressing .mpgs or > .iso 10 times) it would be good to discuss it maybe > > <end google garble> > > I am not at all sure it is a good idea to respond to one of your scribbling. > > Nevertheless. Shannon's theory dates to 1948 and it forms a firm theoretical > base for the work that has come since. One of your "it" candidates is zip > files. No, they can not be compressed further by a factor of 10. Most of > the "good" has already been taken out of the data. > prove it > Movies are more subjective because the compression is loss and therefore how > compressible they are is subjective. Surely there is *some* room for > improvement. in this area - but you are not going to get ten times on a > hockey game, for example. > > https://en.wikipedia.org/wiki/Information_theory
[toc] | [prev] | [next] | [standalone]
| From | Malcolm McLean <malcolm.mclean5@btinternet.com> |
|---|---|
| Date | 2016-06-16 09:36 -0700 |
| Message-ID | <80eeb8c0-fc9e-47e4-bfff-ef31b908a7e3@googlegroups.com> |
| In reply to | #89299 |
On Thursday, June 16, 2016 at 5:10:12 PM UTC+1, fir wrote: > > > Nevertheless. Shannon's theory dates to 1948 and it forms a firm theoretical > > base for the work that has come since. One of your "it" candidates is zip > > files. No, they can not be compressed further by a factor of 10. Most of > > the "good" has already been taken out of the data. > > > > prove it > I'm not quite up to proving Shannon's work on entropy, though I ought to be able to do so - it's important in biology as well as information theory. The basic idea is that diversity is given by the sum of logarithms of a frequency distribution. (I'll give you a copy of my novel if you wish, it's a school story. There's a scene where the children are set the task of determining Shannon's measure of diversity for animals found in a forest). The essential idea is easy enough to prove, however. You can only compress two pieces of data if the first piece gives you some sort of idea what the second piece is likely to be. So if we know the input is letters from English text, and one letter is "q", the next letter must almost certainly be "u", there are only a very few English words where that isn't the case. So out compression scheme "set a bit if it is u, if not, clear a bit and use the next 8 bits for the full byte" will give very nearly 8 fold compression. However, by the act of so doing, we've reduced the possibility for further compression using the same system. We can still compress the data, because almost always the bit will be set. We can think of a sophisticated way to code that more compactly, e.g. we could have a "q followed by u" state, and an "escape state". But we're only saving one bit now. Our returns can only diminish as we add layers of compression. Shannon put that on a proper mathematical basis, but you don't need to be able to reproduce his work to see that the basic claim must be correct.
[toc] | [prev] | [next] | [standalone]
| From | Keith Thompson <kst-u@mib.org> |
|---|---|
| Date | 2016-06-16 11:18 -0700 |
| Message-ID | <ln7fdpvvz2.fsf@kst-u.example.com> |
| In reply to | #89302 |
Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
> On Thursday, June 16, 2016 at 5:10:12 PM UTC+1, fir wrote:
>> > Nevertheless. Shannon's theory dates to 1948 and it forms a firm theoretical
>> > base for the work that has come since. One of your "it" candidates is zip
>> > files. No, they can not be compressed further by a factor of 10. Most of
>> > the "good" has already been taken out of the data.
>> >
>>
>> prove it
>>
> I'm not quite up to proving Shannon's work on entropy, though I ought to be able
> to do so - it's important in biology as well as information theory. The basic idea
> is that diversity is given by the sum of logarithms of a frequency distribution.
There's a comp.compression newsgroup where this discussion would be more
appropriate. It has an FAQ that explains why arbitrarily good
compression is mathematically impossible.
--
Keith Thompson (The_Other_Keith) kst-u@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
[toc] | [prev] | [next] | [standalone]
| From | "Osmium" <r124c4u102@comcast.net> |
|---|---|
| Date | 2016-06-16 13:13 -0500 |
| Message-ID | <dsg961ForroU1@mid.individual.net> |
| In reply to | #89299 |
"fir" wrote: W dniu czwartek, 16 czerwca 2016 17:49:42 UTC+2 użytkownik Osmium napisał: > "fir" wrote: > > W dniu czwartek, 16 czerwca 2016 17:09:24 UTC+2 użytkownik Malcolm McLean > napisał: > > On Thursday, June 16, 2016 at 3:25:42 PM UTC+1, fir wrote: > > > GOD how great it would be have an algorith that would compress totays > > > data which rae not much > > > compressibl;e (like game ISo files, movies, zip archives) like many > > > times, say ten times > > > > > > wouldnt it be great? do you think that it is > > > possible? (im sure it is not really obtainable but other thing if this > > > is not impossible 'in fact') > > > > > > Imagine how great it would be :U > > > > > Unfortunately there are theoretical limits on how far you can compress > > data without prior > > information about what that data represents. > > Movies, for example, are almost certainly already MPEG compressed. That > > strips out > > essentially all of the redundancy. The only way of getting better > > compression is to have > > a database of all movie clips ever produced (that's not necessarily > > impractical) and > > to replace the movies with a reference to that database. > > You can't just feed the MPEG stream to a general purpose compressor and > > expect it > > to compress it. If it were that easy the general purpose compressor > > would > > have already > > been included in the MPEG standard, and in fact it is - MPEG creates a > > Huffman-friendly > > output stream then uses a fairly standard Huffman compressor to achieve > > the final bit > > reduction. > > I dont think "The only way" is proven - this is probably just false and I > wouldnt be maybe surprised if there is a way to compres .mpg's 10 times > yet > (but i dont know the details on compression never worked in it) If > someonne > knows maybe what is a source of this impossibility (of compressing .mpgs > or > .iso 10 times) it would be good to discuss it maybe > > <end google garble> > > I am not at all sure it is a good idea to respond to one of your > scribbling. > > Nevertheless. Shannon's theory dates to 1948 and it forms a firm > theoretical > base for the work that has come since. One of your "it" candidates is zip > files. No, they can not be compressed further by a factor of 10. Most of > the "good" has already been taken out of the data. > prove it > Movies are more subjective because the compression is loss and therefore > how > compressible they are is subjective. Surely there is *some* room for > improvement. in this area - but you are not going to get ten times on a > hockey game, for example. > > https://en.wikipedia.org/wiki/Information_theory The proof is in the link I posted. Anyone who has bothered to understand Huffman encoding vis a vis a text in English (part of zip) realizes that Shannon is on to something. My guess is you pretty much ignored the link, considering the elapsed time between my post and your response. My initial guess was right, you are a waste of time.
[toc] | [prev] | [next] | [standalone]
| From | Gareth Owen <gwowen@gmail.com> |
|---|---|
| Date | 2016-06-16 20:18 +0100 |
| Message-ID | <87porhvt5v.fsf@gmail.com> |
| In reply to | #89317 |
"Osmium" <r124c4u102@comcast.net> writes: > prove it OK. Suppose you have an algorithm that can reversibly compress any file of 1kb to 1023 bytes or fewer. There are 2^1024 files of size 1kb There are 2^1024-1 files of size 1023 bytes or fewer. Therefore you a simple count of sources and destination proves cannot uniquely map the former onto the latter.
[toc] | [prev] | [next] | [standalone]
| From | Richard Heathfield <rjh@cpax.org.uk> |
|---|---|
| Date | 2016-06-16 22:07 +0100 |
| Message-ID | <njv4eq$6tp$1@dont-email.me> |
| In reply to | #89326 |
On 16/06/16 20:18, Gareth Owen wrote: > "Osmium" <r124c4u102@comcast.net> writes: > >> prove it > > OK. Suppose you have an algorithm that can reversibly compress any file > of 1kb to 1023 bytes or fewer. > > There are 2^1024 files of size 1kb True if kb = kilobits, but less true if kb = kilobytes (as is suggested by your earlier sentence). Try this: There are 256^1024 possible files of size 1024 octets. > There are 2^1024-1 files of size 1023 bytes or fewer. There are 256^1024 - 1 files of size 1023 octets or fewer. > Therefore you a simple count of sources and destination proves cannot > uniquely map the former onto the latter. Right. TANSTAAFL. -- Richard Heathfield Email: rjh at cpax dot org dot uk "Usenet is a strange place" - dmr 29 July 1999 Sig line 4 vacant - apply within
[toc] | [prev] | [next] | [standalone]
| From | Gareth Owen <gwowen@gmail.com> |
|---|---|
| Date | 2016-06-17 06:04 +0100 |
| Message-ID | <87a8ik4d9t.fsf@gmail.com> |
| In reply to | #89338 |
Richard Heathfield <rjh@cpax.org.uk> writes: > On 16/06/16 20:18, Gareth Owen wrote: >> "Osmium" <r124c4u102@comcast.net> writes: >> >>> prove it >> >> OK. Suppose you have an algorithm that can reversibly compress any file >> of 1kb to 1023 bytes or fewer. >> >> There are 2^1024 files of size 1kb > > True if kb = kilobits, but less true if kb = kilobytes (as is > suggested by your earlier sentence). Facepalm. Bits, of course. > Try this: There are 256^1024 possible files of size 1024 octets. Of course. *bashes head on table* Still, besides the fact that literally every number was orders of magnitude wrong, the argument was fine. :)
[toc] | [prev] | [next] | [standalone]
| From | Robert Wessel <robertwessel2@yahoo.com> |
|---|---|
| Date | 2016-06-16 23:21 -0500 |
| Message-ID | <seu6mbdenu8031pd2b33lgj1bu67jk41f6@4ax.com> |
| In reply to | #89326 |
On Thu, 16 Jun 2016 20:18:52 +0100, Gareth Owen <gwowen@gmail.com> wrote: >"Osmium" <r124c4u102@comcast.net> writes: > >> prove it > >OK. Suppose you have an algorithm that can reversibly compress any file >of 1kb to 1023 bytes or fewer. > >There are 2^1024 files of size 1kb >There are 2^1024-1 files of size 1023 bytes or fewer. > >Therefore you a simple count of sources and destination proves cannot >uniquely map the former onto the latter. Your confusion between bits and bytes has already been pointed out, but it's customary to consider (file) sizes in bits anyway in information theory. But yes, there are 2**1024 possible files of 1024 bits, and if you can store only 1023 bits, you can only identify half of them. Usually known as the "counting theorem" or the "pigeon hole principle" (you can't have more pigeons than pigeon holes and hope to store all your pigeons, without putting more than one in a hole).
[toc] | [prev] | [next] | [standalone]
| From | "Osmium" <r124c4u102@comcast.net> |
|---|---|
| Date | 2016-06-17 07:49 -0500 |
| Message-ID | <dsi9upF6h15U1@mid.individual.net> |
| In reply to | #89360 |
"Robert Wessel" wrote: > On Thu, 16 Jun 2016 20:18:52 +0100, Gareth Owen <gwowen@gmail.com> > wrote: > >>"Osmium" <r124c4u102@comcast.net> writes: >> >>> prove it >> >>OK. Suppose you have an algorithm that can reversibly compress any file >>of 1kb to 1023 bytes or fewer. >> >>There are 2^1024 files of size 1kb >>There are 2^1024-1 files of size 1023 bytes or fewer. >> >>Therefore you a simple count of sources and destination proves cannot >>uniquely map the former onto the latter. > > > Your confusion between bits and bytes has already been pointed out, > but it's customary to consider (file) sizes in bits anyway in > information theory. > > But yes, there are 2**1024 possible files of 1024 bits, and if you can > store only 1023 bits, you can only identify half of them. Usually > known as the "counting theorem" or the "pigeon hole principle" (you > can't have more pigeons than pigeon holes and hope to store all your > pigeons, without putting more than one in a hole). Please note that I didn't contribute anything at all on that message.
[toc] | [prev] | [next] | [standalone]
| From | Robert Wessel <robertwessel2@yahoo.com> |
|---|---|
| Date | 2016-06-17 12:47 -0500 |
| Message-ID | <rrd8mbth1clkutk1pohc9fm9v7fie56t6m@4ax.com> |
| In reply to | #89403 |
On Fri, 17 Jun 2016 07:49:30 -0500, "Osmium" <r124c4u102@comcast.net> wrote: >"Robert Wessel" wrote: > >> On Thu, 16 Jun 2016 20:18:52 +0100, Gareth Owen <gwowen@gmail.com> >> wrote: >> >>>"Osmium" <r124c4u102@comcast.net> writes: >>> >>>> prove it >>> >>>OK. Suppose you have an algorithm that can reversibly compress any file >>>of 1kb to 1023 bytes or fewer. >>> >>>There are 2^1024 files of size 1kb >>>There are 2^1024-1 files of size 1023 bytes or fewer. >>> >>>Therefore you a simple count of sources and destination proves cannot >>>uniquely map the former onto the latter. >> >> >> Your confusion between bits and bytes has already been pointed out, >> but it's customary to consider (file) sizes in bits anyway in >> information theory. >> >> But yes, there are 2**1024 possible files of 1024 bits, and if you can >> store only 1023 bits, you can only identify half of them. Usually >> known as the "counting theorem" or the "pigeon hole principle" (you >> can't have more pigeons than pigeon holes and hope to store all your >> pigeons, without putting more than one in a hole). > >Please note that I didn't contribute anything at all on that message. The "prove it" is yours, although I should have probably snipped that that (and then the attribution).
[toc] | [prev] | [next] | [standalone]
| From | "Osmium" <r124c4u102@comcast.net> |
|---|---|
| Date | 2016-06-17 14:47 -0500 |
| Message-ID | <dsj2fcFbjiqU1@mid.individual.net> |
| In reply to | #89438 |
"Robert Wessel" wrote: > On Fri, 17 Jun 2016 07:49:30 -0500, "Osmium" <r124c4u102@comcast.net> > wrote: > >>"Robert Wessel" wrote: >> >>> On Thu, 16 Jun 2016 20:18:52 +0100, Gareth Owen <gwowen@gmail.com> >>> wrote: >>> >>>>"Osmium" <r124c4u102@comcast.net> writes: >>>> >>>>> prove it >>>> >>>>OK. Suppose you have an algorithm that can reversibly compress any file >>>>of 1kb to 1023 bytes or fewer. >>>> >>>>There are 2^1024 files of size 1kb >>>>There are 2^1024-1 files of size 1023 bytes or fewer. >>>> >>>>Therefore you a simple count of sources and destination proves cannot >>>>uniquely map the former onto the latter. >>> >>> >>> Your confusion between bits and bytes has already been pointed out, >>> but it's customary to consider (file) sizes in bits anyway in >>> information theory. >>> >>> But yes, there are 2**1024 possible files of 1024 bits, and if you can >>> store only 1023 bits, you can only identify half of them. Usually >>> known as the "counting theorem" or the "pigeon hole principle" (you >>> can't have more pigeons than pigeon holes and hope to store all your >>> pigeons, without putting more than one in a hole). >> >>Please note that I didn't contribute anything at all on that message. > > > The "prove it" is yours, although I should have probably snipped that > that (and then the attribution). I can see why you think that. But fir responds via google which doesn't follow Usenet protocol, as a consequence my newsreader messes up attributions. In my first message (*only*) I warned of "google garble" but in the second one I did not. "Prove it" was actually an instruction from fir to me, not realizing that I had already done all the proving I was about to do.
[toc] | [prev] | [next] | [standalone]
| From | Robert Wessel <robertwessel2@yahoo.com> |
|---|---|
| Date | 2016-06-17 23:32 -0500 |
| Message-ID | <0kj9mblml55dvjcbdap5ch0tbqn51sr6ho@4ax.com> |
| In reply to | #89448 |
On Fri, 17 Jun 2016 14:47:57 -0500, "Osmium" <r124c4u102@comcast.net> wrote: >"Robert Wessel" wrote: > >> On Fri, 17 Jun 2016 07:49:30 -0500, "Osmium" <r124c4u102@comcast.net> >> wrote: >> >>>"Robert Wessel" wrote: >>> >>>> On Thu, 16 Jun 2016 20:18:52 +0100, Gareth Owen <gwowen@gmail.com> >>>> wrote: >>>> >>>>>"Osmium" <r124c4u102@comcast.net> writes: >>>>> >>>>>> prove it >>>>> >>>>>OK. Suppose you have an algorithm that can reversibly compress any file >>>>>of 1kb to 1023 bytes or fewer. >>>>> >>>>>There are 2^1024 files of size 1kb >>>>>There are 2^1024-1 files of size 1023 bytes or fewer. >>>>> >>>>>Therefore you a simple count of sources and destination proves cannot >>>>>uniquely map the former onto the latter. >>>> >>>> >>>> Your confusion between bits and bytes has already been pointed out, >>>> but it's customary to consider (file) sizes in bits anyway in >>>> information theory. >>>> >>>> But yes, there are 2**1024 possible files of 1024 bits, and if you can >>>> store only 1023 bits, you can only identify half of them. Usually >>>> known as the "counting theorem" or the "pigeon hole principle" (you >>>> can't have more pigeons than pigeon holes and hope to store all your >>>> pigeons, without putting more than one in a hole). >>> >>>Please note that I didn't contribute anything at all on that message. >> >> >> The "prove it" is yours, although I should have probably snipped that >> that (and then the attribution). > >I can see why you think that. But fir responds via google which doesn't >follow Usenet protocol, as a consequence my newsreader messes up >attributions. In my first message (*only*) I warned of "google garble" but >in the second one I did not. "Prove it" was actually an instruction from >fir to me, not realizing that I had already done all the proving I was about >to do. I see. My apologies. Another good reason to avoid threads where fir is participating.
[toc] | [prev] | [next] | [standalone]
| From | fir <profesor.fir@gmail.com> |
|---|---|
| Date | 2016-06-18 01:33 -0700 |
| Message-ID | <310faf36-0ffb-4071-bbb4-0296091fee61@googlegroups.com> |
| In reply to | #89499 |
W dniu sobota, 18 czerwca 2016 06:32:51 UTC+2 użytkownik robert...@yahoo.com napisał: > On Fri, 17 Jun 2016 14:47:57 -0500, "Osmium" <r124c4u102@comcast.net> > wrote: > > >"Robert Wessel" wrote: > > > >> On Fri, 17 Jun 2016 07:49:30 -0500, "Osmium" <r124c4u102@comcast.net> > >> wrote: > >> > >>>"Robert Wessel" wrote: > >>> > >>>> On Thu, 16 Jun 2016 20:18:52 +0100, Gareth Owen <gwowen@gmail.com> > >>>> wrote: > >>>> > >>>>>"Osmium" <r124c4u102@comcast.net> writes: > >>>>> > >>>>>> prove it > >>>>> > >>>>>OK. Suppose you have an algorithm that can reversibly compress any file > >>>>>of 1kb to 1023 bytes or fewer. > >>>>> > >>>>>There are 2^1024 files of size 1kb > >>>>>There are 2^1024-1 files of size 1023 bytes or fewer. > >>>>> > >>>>>Therefore you a simple count of sources and destination proves cannot > >>>>>uniquely map the former onto the latter. > >>>> > >>>> > >>>> Your confusion between bits and bytes has already been pointed out, > >>>> but it's customary to consider (file) sizes in bits anyway in > >>>> information theory. > >>>> > >>>> But yes, there are 2**1024 possible files of 1024 bits, and if you can > >>>> store only 1023 bits, you can only identify half of them. Usually > >>>> known as the "counting theorem" or the "pigeon hole principle" (you > >>>> can't have more pigeons than pigeon holes and hope to store all your > >>>> pigeons, without putting more than one in a hole). > >>> > >>>Please note that I didn't contribute anything at all on that message. > >> > >> > >> The "prove it" is yours, although I should have probably snipped that > >> that (and then the attribution). > > > >I can see why you think that. But fir responds via google which doesn't > >follow Usenet protocol, as a consequence my newsreader messes up > >attributions. In my first message (*only*) I warned of "google garble" but > >in the second one I did not. "Prove it" was actually an instruction from > >fir to me, not realizing that I had already done all the proving I was about > >to do. > > > I see. My apologies. Another good reason to avoid threads where fir > is participating. funny i feel the waste of participating in threads you all (or nearly) intelectual impotents and weak-heads are participating : ) you dont even imagine how mind off heads you are and how space of brainlessnes you create - im get used to it though its like "you want discusion you find clowns and bores" so be it
[toc] | [prev] | [next] | [standalone]
| From | fir <profesor.fir@gmail.com> |
|---|---|
| Date | 2016-06-17 14:51 -0700 |
| Message-ID | <984615c0-0727-433e-a5d4-9b31fc976828@googlegroups.com> |
| In reply to | #89317 |
W dniu czwartek, 16 czerwca 2016 20:24:10 UTC+2 użytkownik Osmium napisał: > "fir" wrote: > > W dniu czwartek, 16 czerwca 2016 17:49:42 UTC+2 użytkownik Osmium napisał: > > "fir" wrote: > > > > W dniu czwartek, 16 czerwca 2016 17:09:24 UTC+2 użytkownik Malcolm McLean > > napisał: > > > On Thursday, June 16, 2016 at 3:25:42 PM UTC+1, fir wrote: > > > > GOD how great it would be have an algorith that would compress totays > > > > data which rae not much > > > > compressibl;e (like game ISo files, movies, zip archives) like many > > > > times, say ten times > > > > > > > > wouldnt it be great? do you think that it is > > > > possible? (im sure it is not really obtainable but other thing if this > > > > is not impossible 'in fact') > > > > > > > > Imagine how great it would be :U > > > > > > > Unfortunately there are theoretical limits on how far you can compress > > > data without prior > > > information about what that data represents. > > > Movies, for example, are almost certainly already MPEG compressed. That > > > strips out > > > essentially all of the redundancy. The only way of getting better > > > compression is to have > > > a database of all movie clips ever produced (that's not necessarily > > > impractical) and > > > to replace the movies with a reference to that database. > > > You can't just feed the MPEG stream to a general purpose compressor and > > > expect it > > > to compress it. If it were that easy the general purpose compressor > > > would > > > have already > > > been included in the MPEG standard, and in fact it is - MPEG creates a > > > Huffman-friendly > > > output stream then uses a fairly standard Huffman compressor to achieve > > > the final bit > > > reduction. > > > > I dont think "The only way" is proven - this is probably just false and I > > wouldnt be maybe surprised if there is a way to compres .mpg's 10 times > > yet > > (but i dont know the details on compression never worked in it) If > > someonne > > knows maybe what is a source of this impossibility (of compressing .mpgs > > or > > .iso 10 times) it would be good to discuss it maybe > > > > <end google garble> > > > > I am not at all sure it is a good idea to respond to one of your > > scribbling. > > > > Nevertheless. Shannon's theory dates to 1948 and it forms a firm > > theoretical > > base for the work that has come since. One of your "it" candidates is zip > > files. No, they can not be compressed further by a factor of 10. Most of > > the "good" has already been taken out of the data. > > > > prove it > > > Movies are more subjective because the compression is loss and therefore > > how > > compressible they are is subjective. Surely there is *some* room for > > improvement. in this area - but you are not going to get ten times on a > > hockey game, for example. > > > > https://en.wikipedia.org/wiki/Information_theory > > The proof is in the link I posted. Anyone who has bothered to understand > Huffman encoding vis a vis a text in English (part of zip) realizes that > Shannon is on to something. My guess is you pretty much ignored the link, > considering the elapsed time between my post and your response. My initial > guess was right, you are a waste of time. if you reason in such way full of logical holes (and you do) it means you are hevily stupid user (its plain proof) and your buffonade does not help you here You will not talk with me anymore so dont worry it always shock me how stupid some are but your buffonade is so weak, I get used to it, good bye
[toc] | [prev] | [next] | [standalone]
| From | BartC <bc@freeuk.com> |
|---|---|
| Date | 2016-06-16 16:50 +0100 |
| Message-ID | <njuhru$v4p$1@dont-email.me> |
| In reply to | #89282 |
On 16/06/2016 16:15, fir wrote: > I dont think "The only way" is proven - this is probably just false and I wouldnt be maybe surprised if there is a way to compres .mpg's 10 times yet > (but i dont know the details on compression never worked in it) If someonne knows maybe what is a source of this impossibility (of compressing .mpgs or .iso 10 times) it would be good to discuss it maybe > I dont think "The only way" is proven - this is probably just false > and I wouldnt be maybe surprised if there is a way to compres .mpg's > 10 times yet Weren't you saying recently that compilers ought to work at 10M lines per second? (When I reported a raw tokenising speed at that rate.) I said that 1Mlps might be nearer the upper limit. (That's for consumer PCs, non-threaded; I'll tell you next week how far from that I am.) But with compressing something that has already been compressed: the input will be little different from random data, so that could be difficult. Starting from the original data (say uncompressed raw video) then you could compress a bit more I suppose depending on how lossy or poor quality you want it. > (but i dont know the details on compression never worked in it) If someonne knows maybe what is a source of this impossibility (of compressing .mpgs or .iso 10 times) it would be good to discuss it maybe Raw HD video (1920x1080 50p assuming 24-bits per pixel full chroma) requires a data rate of some 2.5Gbs. The BBC manage to deliver on-line HD content at some 2.9Mbps (although not the best quality), which is 800 times less bandwidth. Good luck getting it to 8000 times less! (I think the BBC HD service started at 16Mbps so they might already be scraping the barrel.) -- Bartc -- Bartc
[toc] | [prev] | [next] | [standalone]
| From | BartC <bc@freeuk.com> |
|---|---|
| Date | 2016-06-17 16:20 +0100 |
| Message-ID | <nk14fc$77u$1@dont-email.me> |
| In reply to | #89290 |
On 16/06/2016 16:50, BartC wrote: > On 16/06/2016 16:15, fir wrote: > >> I dont think "The only way" is proven - this is probably just false >> and I wouldnt be maybe surprised if there is a way to compres .mpg's >> 10 times yet > Weren't you saying recently that compilers ought to work at 10M lines > per second? (When I reported a raw tokenising speed at that rate.) > > I said that 1Mlps might be nearer the upper limit. (That's for consumer > PCs, non-threaded; I'll tell you next week how far from that I am.) (I've only just managed it, although not for a C-class language: http://pastebin.com/xeXFz8ZG gives more info. So my prediction wasn't far off!) -- Bartc
[toc] | [prev] | [next] | [standalone]
| From | fir <profesor.fir@gmail.com> |
|---|---|
| Date | 2016-06-16 08:34 -0700 |
| Message-ID | <eda8336c-9032-4536-8b65-edb77f5694b9@googlegroups.com> |
| In reply to | #89280 |
W dniu czwartek, 16 czerwca 2016 17:09:24 UTC+2 użytkownik Malcolm McLean napisał: > On Thursday, June 16, 2016 at 3:25:42 PM UTC+1, fir wrote: > > GOD how great it would be have an algorith that would compress totays data which rae not much > > compressibl;e (like game ISo files, movies, zip archives) like many times, say ten times > > > > wouldnt it be great? do you think that it is > > possible? (im sure it is not really obtainable but other thing if this is not impossible 'in fact') > > > > Imagine how great it would be :U > > > Unfortunately there are theoretical limits on how far you can compress data without prior > information about what that data represents. > Movies, for example, are almost certainly already MPEG compressed. That strips out > essentially all of the redundancy. The only way of getting better compression is to have > a database of all movie clips ever produced (that's not necessarily impractical) and > to replace the movies with a reference to that database. this database by the way is present say like in pi number, you will maybe only need the adress reference and hope it will be shorter than the movie ;C (some loose idea) Wonder if there is some theorem speaking about the ratio ft the sizes key/content in such pi container or some other Probably sompresing people think on such cases but i dont know much on it > You can't just feed the MPEG stream to a general purpose compressor and expect it > to compress it. If it were that easy the general purpose compressor would have already > been included in the MPEG standard, and in fact it is - MPEG creates a Huffman-friendly > output stream then uses a fairly standard Huffman compressor to achieve the final bit > reduction.
[toc] | [prev] | [next] | [standalone]
| From | Les Cargill <lcargill99@comcast.com> |
|---|---|
| Date | 2016-06-16 12:55 -0500 |
| Message-ID | <njup07$qp1$2@dont-email.me> |
| In reply to | #89274 |
fir wrote: > GOD how great it would be have an algorith that would compress totays data which rae not much compressibl;e (like game ISo files, movies, zip archives) like many times, say ten times > > wouldnt it be great? do you think that it is > possible? (im sure it is not really obtainable but other thing if this is not impossible 'in fact') > > Imagine how great it would be :U > So would perpetual motion. Same basic thing. https://en.wikipedia.org/wiki/Second_law_of_thermodynamics -- Les Cargill
[toc] | [prev] | [next] | [standalone]
Page 2 of 5 — ← Prev page 1 [2] 3 4 5 Next page →
Back to top | Article view | comp.lang.c
csiph-web