Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.dsp > #6597 > unrolled thread

information theory?

Back to article view | Back to comp.dsp

Contents

  information theory? RichD <r_delaney2001@yahoo.com> - 2011-10-31 20:55 -0700
    Re: information theory? eric.jacobsen@ieee.org (Eric Jacobsen) - 2011-11-01 04:57 +0000
      Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-01 05:58 -0700
        Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-01 10:29 -0700
          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 11:13 -0700
            Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 14:39 -0700
          Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-01 16:04 -0400
            OT: information theory? Jerry Avins <jya@ieee.org> - 2011-11-01 18:06 -0400
            Re: information theory? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-11-01 22:46 +0000
          Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-01 22:38 -0500
            Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-11-02 17:06 -0500
              Re: information theory? "steveu" <steveu@n_o_s_p_a_m.coppice.org> - 2011-11-02 19:41 -0500
              Re: information theory? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-11-03 06:58 -0500
                Re: information theory? "GO-HERE .NL" <gdewilde@gmail.com> - 2011-11-03 08:57 -0700
            Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-02 19:44 -0500
              Re: information theory? "Christopher J. Henrich" <chenrich@monmouth.com> - 2011-11-02 22:40 -0400
                Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-03 23:21 -0500
          Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 08:47 +0000
            Re: information theory? (copyright question followup) JohnF <john@please.see.sig.for.email.com> - 2011-11-02 09:48 +0000
              Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 10:02 +0000
                Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 16:46 +0000
                Re: information theory? (copyright question followup) Bernd Jendrissek <bernd.jendrissek@gmail.com> - 2011-11-02 15:04 -0700
              Re: information theory? (copyright question followup) David Eather <eather@tpg.com.au> - 2011-11-03 04:00 +1000
                Re: information theory? (copyright question followup) unruh <unruh@physics.ubc.ca> - 2011-11-02 19:05 +0000
              Re: information theory? (copyright question followup) "J. Clarke" <jclarkeusenet@cox.net> - 2011-11-24 08:42 -0500
                Re: information theory? (copyright question followup) Richard Outerbridge <outer@interlog.com> - 2011-11-25 01:37 -0500
                  Re: information theory? (copyright question followup) Les Cargill <lcargill99@comcast.com> - 2011-11-25 11:37 -0600
            Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:38 -0700
              Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-03 08:51 +0000
          Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-02 08:02 -0700
            Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-02 09:38 -0700
            Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:51 -0700
              Re: information theory? VWWall <vwall@large.invalid> - 2011-11-02 15:08 -0700
                Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-11-02 19:59 -0400
                  Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-04 20:41 +0000
                    Re: information theory? "krw@att.bizzzzzzzzzzzz" <krw@att.bizzzzzzzzzzzz> - 2011-11-04 16:59 -0500
                      Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 15:28 -0700
                      Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 21:09 -0400
                        Re: information theory? ggr@nope.ucsd.edu (Greg Rose) - 2011-11-05 02:11 +0000
                        Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-05 11:53 +0000
                      Re: information theory? "Mike Terry" <news.dead.person.stones@darjeeling.plus.com> - 2011-11-05 17:32 +0000
                  Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 18:41 -0800
                Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-03 21:54 -0700
                  Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 03:42 -0500
                  Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 20:33 +1100
                    Re: information theory? Robert Wessel <robertwessel2@yahoo.com> - 2011-11-04 05:10 -0500
                      Re: information theory? kym@kymhorsell.com - 2011-11-04 10:50 +0000
                        Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 22:14 +1100
                          Re: information theory? "Jesse F. Hughes" <jesse@phiwumbda.org> - 2011-11-04 09:13 -0400
                            Re: information theory? Spehro Pefhany <speffSNIP@interlogDOTyou.knowwhat> - 2011-11-04 10:32 -0400
                              Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 18:02 +0000
                                Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 15:54 -0800
                        Re: information theory? VWWall <vwall@large.invalid> - 2011-11-04 09:50 -0700
                          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 12:41 -0700
                          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 13:40 -0700
                      Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:35 +0000
                    Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-04 10:31 +0000
                      Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 21:51 +1100
                      Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 14:18 -0500
                      Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 15:56 -0400
                      Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-12-21 17:49 -0800
                        Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-12-21 20:07 -0600
                        Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-12-22 03:08 -0500
                          Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-12-22 08:30 +0000
                            Re: information theory? John Devereux <john@devereux.me.uk> - 2011-12-22 10:12 +0000
                            Re: information theory? "Peter Webb" <r.peter.webbbbb@gmail.com> - 2011-12-22 21:14 +1100
                    Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:00 +0000
                  Re: information theory? Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2011-11-04 09:45 +0000
                    Re: information theory? unruh <unruh@physics.ubc.ca> - 2011-11-04 19:05 +0000
                Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 17:11 -0800
                  Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 06:23 -0800
                    Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 11:29 -0800
                    Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-08 04:41 -0800
                      Re: information theory? "Michael A. Terrell" <mike.terrell@earthlink.net> - 2011-11-08 13:46 -0500
              Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-03 03:32 -0500
              Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:39 -0700
              Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:28 -0700

Page 4 of 4 — ← Prev page 1 2 3 [4]

#7947

On Nov 4, Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
> > He estimated music contains 40 bits/second entropy.
> > How close is MP3 to that?
>
> >___________________________________
> >I doubt surprised Shannon said that, and if he did its
> >somewhere between meaningless and wrong.
>
> >CD quality mp3s are roughly equivalent to 178 kbps,
> >over 4,000 times his estimate. But then you can
> >encode a lot of sounds that most people would not
> >consider music. And it stereo, so you can halve it
> > if Shannon was talking about mono.
>
> Have you never seen sheet music? That is what
> Shannon was estimating - the bitrate for describing
> music in the abstract.
> There are a finite number of notes, durations and
> amplitudes in a classical composition.

I suspect that's the case.

> I suspect 40 bits/sec is still far too tight,
> but a midi stream using a high end reconstruction
> codec represents a pretty good
> example of what is possible by way of compression
> for *music* as opposed to voice or a random noise
> stream.
>
> > To get a smaller figure for music, you have to
> > define what subsets of sounds are music. Lots of luck.

Not so hard, if you analyze samples of what we label 'music'.

> I think that may have been his intention although
> I don't recall seeing the 40 bit/s number
> As I said if he did anything I think he was estimating
> the information content of music in the already
> concise form of an orchestral score.

That appears to be the case.

> I reckon at a bare minimum about 7 to the note, 8 to
> amplitude, 6 duration, 5 to the instrument - and it
> is already obvious that you cannot encode more than a
> single note per second at this bitrate.

You fail to account for the essential concept
of correlation.

> Can anyone provide a citation to this alleged paper
> on music bitrate?


I found the reference in John Pierce's book.
It's not attributed to Shannon.  My boner.
He claims a humans can absorb information,
e.g. reading or music, at 40 bits/second.
He also cites a paper where it's claimed that
sheet music contains 2.5 bits entropy per note.
This supports the MIDI model, as opposed to
digitization of a recording.  In that case,
what is the bit rate?  Though the question of
chords arises, with much redundancy there.

It also raises the possibility that recorded
music might be adequately represented by a
MIDI stream.


--
Rich


--
Rich

[toc] | [prev] | [next] | [standalone]

#7949

RichD wrote:
<snip>
>
> It also raises the possibility that recorded
> music might be adequately represented by a
> MIDI stream.
>

The people at the History Channel seem to
believe that too. they're wrong. ":)

Nope. Show me a pinch harmonic in MIDI. I would
bet that simply representing all the ways there
are to hit a snare drum would take up *a lot* of
space. heck, track a peizo on one channel and
a mic on the other - of the same acoustic guitar
part - and there's significant dis-correlation,
or at least the deconvolution is not what you'd
expect.

Audio ala music has a (for lack of a better term)
"fractal" quality at that level.

I take Shannon at his word - he's guessing what
the "Taylorist" minimum motion set for music is,
but that's really not enough. It's kind of fascinating
that this is the case, that there's still so much
left for humans to do there.

>
> --
> Rich
>
>
> --
> Rich
>

--
Les Cargill

[toc] | [prev] | [next] | [standalone]

#7975

In article 
<bd7a5bf2-f8be-4dde-82bd-0b8b7d14655c@v31g2000prg.googlegroups.com>,
 RichD <r_delaney2001@yahoo.com> wrote:

> On Nov 4, Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
> > > He estimated music contains 40 bits/second entropy.
> > > How close is MP3 to that?
> >
> > >___________________________________
> > >I doubt surprised Shannon said that, and if he did its
> > >somewhere between meaningless and wrong.
> >
> > >CD quality mp3s are roughly equivalent to 178 kbps,
> > >over 4,000 times his estimate. But then you can
> > >encode a lot of sounds that most people would not
> > >consider music. And it stereo, so you can halve it
> > > if Shannon was talking about mono.
> >
> > Have you never seen sheet music? That is what
> > Shannon was estimating - the bitrate for describing
> > music in the abstract.
> > There are a finite number of notes, durations and
> > amplitudes in a classical composition.
> 
> I suspect that's the case.
> 
> > I suspect 40 bits/sec is still far too tight,
> > but a midi stream using a high end reconstruction
> > codec represents a pretty good
> > example of what is possible by way of compression
> > for *music* as opposed to voice or a random noise
> > stream.
> >
> > > To get a smaller figure for music, you have to
> > > define what subsets of sounds are music. Lots of luck.
> 
> Not so hard, if you analyze samples of what we label 'music'.
> 
> > I think that may have been his intention although
> > I don't recall seeing the 40 bit/s number
> > As I said if he did anything I think he was estimating
> > the information content of music in the already
> > concise form of an orchestral score.
> 
> That appears to be the case.
> 
> > I reckon at a bare minimum about 7 to the note, 8 to
> > amplitude, 6 duration, 5 to the instrument - and it
> > is already obvious that you cannot encode more than a
> > single note per second at this bitrate.
> 
> You fail to account for the essential concept
> of correlation.
> 
> > Can anyone provide a citation to this alleged paper
> > on music bitrate?
> 
> 
> I found the reference in John Pierce's book.
> It's not attributed to Shannon.  My boner.
> He claims a humans can absorb information,
> e.g. reading or music, at 40 bits/second.
> He also cites a paper where it's claimed that
> sheet music contains 2.5 bits entropy per note.
> This supports the MIDI model, as opposed to
> digitization of a recording.  In that case,
> what is the bit rate?  Though the question of
> chords arises, with much redundancy there.
> 
> It also raises the possibility that recorded
> music might be adequately represented by a
> MIDI stream.

IIRC, it's interesting to note that most modern media
compression codecs rely not so much upon what the raw
data can absolutely do without as they do upon what the
human ear or eye can be fooled by: they leave out what
you "cannot" hear or "cannot" see.  Really, this is
nothing new: all screens of any sort are really just
optical illusions of one sort or another.  The codecs
are just leveraging the limitations of the wiring of
the human brain.

outer

[toc] | [prev] | [next] | [standalone]

#7976

On 22/12/2011 08:08, Richard Outerbridge wrote:
> In article
> <bd7a5bf2-f8be-4dde-82bd-0b8b7d14655c@v31g2000prg.googlegroups.com>,
>   RichD<r_delaney2001@yahoo.com>  wrote:
>
>> On Nov 4, Martin Brown<|||newspam...@nezumi.demon.co.uk>  wrote:
>>>> He estimated music contains 40 bits/second entropy.
>>>> How close is MP3 to that?
>>>
>>>> ___________________________________
>>>> I doubt surprised Shannon said that, and if he did its
>>>> somewhere between meaningless and wrong.
>>>
>>>> CD quality mp3s are roughly equivalent to 178 kbps,
>>>> over 4,000 times his estimate. But then you can
>>>> encode a lot of sounds that most people would not
>>>> consider music. And it stereo, so you can halve it
>>>> if Shannon was talking about mono.
>>>
>>> Have you never seen sheet music? That is what
>>> Shannon was estimating - the bitrate for describing
>>> music in the abstract.
>>> There are a finite number of notes, durations and
>>> amplitudes in a classical composition.
>>
>> I suspect that's the case.
>>
>>> I suspect 40 bits/sec is still far too tight,
>>> but a midi stream using a high end reconstruction
>>> codec represents a pretty good
>>> example of what is possible by way of compression
>>> for *music* as opposed to voice or a random noise
>>> stream.
>>>
>>>> To get a smaller figure for music, you have to
>>>> define what subsets of sounds are music. Lots of luck.
>>
>> Not so hard, if you analyze samples of what we label 'music'.
>>
>>> I think that may have been his intention although
>>> I don't recall seeing the 40 bit/s number
>>> As I said if he did anything I think he was estimating
>>> the information content of music in the already
>>> concise form of an orchestral score.
>>
>> That appears to be the case.
>>
>>> I reckon at a bare minimum about 7 to the note, 8 to
>>> amplitude, 6 duration, 5 to the instrument - and it
>>> is already obvious that you cannot encode more than a
>>> single note per second at this bitrate.
>>
>> You fail to account for the essential concept
>> of correlation.
>>
>>> Can anyone provide a citation to this alleged paper
>>> on music bitrate?
>>
>>
>> I found the reference in John Pierce's book.
>> It's not attributed to Shannon.  My boner.
>> He claims a humans can absorb information,
>> e.g. reading or music, at 40 bits/second.

That sounds way too low. I can flash read this entire page in about 
second and there are a lot more than 40 bits of data in it. And the 
processing to do that much text recognition in real time would tax a 
super computer - that is why Captcha is used to test for humans!

>> He also cites a paper where it's claimed that
>> sheet music contains 2.5 bits entropy per note.

Again sounds far too low even when you only allow for pitch, duration 
and amplitude (plus any changes in amplitude, incidentals, time 
signature or syncopation). Please can you post the citation?

I would only believe this claim iff they can demonstrate a real musical 
score compressed to the equivalent of 2.5bits/note. That is the acid 
test. Hand waving arguments and gut feel do not hack it.

>> This supports the MIDI model, as opposed to
>> digitization of a recording.  In that case,
>> what is the bit rate?  Though the question of
>> chords arises, with much redundancy there.
>>
>> It also raises the possibility that recorded
>> music might be adequately represented by a
>> MIDI stream.
>
> IIRC, it's interesting to note that most modern media
> compression codecs rely not so much upon what the raw
> data can absolutely do without as they do upon what the
> human ear or eye can be fooled by: they leave out what
> you "cannot" hear or "cannot" see.  Really, this is
> nothing new: all screens of any sort are really just
> optical illusions of one sort or another.  The codecs
> are just leveraging the limitations of the wiring of
> the human brain.

Absolutely. There is no point in transmitting detail that the ear cannot 
hear or the eye cannot see. But you do have to be a little bit careful. 
The UK's new DAB audio bitrate is insufficient to support quality live 
broadcast of classical music and is distinctly inferior to either FM or 
off satellite SD or HD digital audio. The problems with DAB are 
sufficiently bad that it is hampering uptake of new DAB radios.

Regards,
Martin Brown

[toc] | [prev] | [next] | [standalone]

#7980

Martin Brown <|||newspam|||@nezumi.demon.co.uk> writes:

> On 22/12/2011 08:08, Richard Outerbridge wrote:
>> In article
>> <bd7a5bf2-f8be-4dde-82bd-0b8b7d14655c@v31g2000prg.googlegroups.com>,
>>   RichD<r_delaney2001@yahoo.com>  wrote:
>>
>>> On Nov 4, Martin Brown<|||newspam...@nezumi.demon.co.uk>  wrote:
>>>>> He estimated music contains 40 bits/second entropy.
>>>>> How close is MP3 to that?
>>>>
>>>>> ___________________________________
>>>>> I doubt surprised Shannon said that, and if he did its
>>>>> somewhere between meaningless and wrong.
>>>>
>>>>> CD quality mp3s are roughly equivalent to 178 kbps,
>>>>> over 4,000 times his estimate. But then you can
>>>>> encode a lot of sounds that most people would not
>>>>> consider music. And it stereo, so you can halve it
>>>>> if Shannon was talking about mono.
>>>>
>>>> Have you never seen sheet music? That is what
>>>> Shannon was estimating - the bitrate for describing
>>>> music in the abstract.
>>>> There are a finite number of notes, durations and
>>>> amplitudes in a classical composition.
>>>
>>> I suspect that's the case.
>>>
>>>> I suspect 40 bits/sec is still far too tight,
>>>> but a midi stream using a high end reconstruction
>>>> codec represents a pretty good
>>>> example of what is possible by way of compression
>>>> for *music* as opposed to voice or a random noise
>>>> stream.
>>>>
>>>>> To get a smaller figure for music, you have to
>>>>> define what subsets of sounds are music. Lots of luck.
>>>
>>> Not so hard, if you analyze samples of what we label 'music'.
>>>
>>>> I think that may have been his intention although
>>>> I don't recall seeing the 40 bit/s number
>>>> As I said if he did anything I think he was estimating
>>>> the information content of music in the already
>>>> concise form of an orchestral score.
>>>
>>> That appears to be the case.
>>>
>>>> I reckon at a bare minimum about 7 to the note, 8 to
>>>> amplitude, 6 duration, 5 to the instrument - and it
>>>> is already obvious that you cannot encode more than a
>>>> single note per second at this bitrate.
>>>
>>> You fail to account for the essential concept
>>> of correlation.
>>>
>>>> Can anyone provide a citation to this alleged paper
>>>> on music bitrate?
>>>
>>>
>>> I found the reference in John Pierce's book.
>>> It's not attributed to Shannon.  My boner.
>>> He claims a humans can absorb information,
>>> e.g. reading or music, at 40 bits/second.
>
> That sounds way too low. I can flash read this entire page in about
> second and there are a lot more than 40 bits of data in it. And the
> processing to do that much text recognition in real time would tax a
> super computer - that is why Captcha is used to test for humans!

But your brain compresses all that data into the memory 

"sed. shannon > 40 bits".

Which with suitable encoding could be less than 40 bits :)


[...]


-- 

John Devereux

[toc] | [prev] | [next] | [standalone]

#7981

"Martin Brown" <|||newspam|||@nezumi.demon.co.uk> wrote in message 
news:jcupqt$l1m$3@speranza.aioe.org...
> On 22/12/2011 08:08, Richard Outerbridge wrote:
>> In article
>> <bd7a5bf2-f8be-4dde-82bd-0b8b7d14655c@v31g2000prg.googlegroups.com>,
>>   RichD<r_delaney2001@yahoo.com>  wrote:
>>
>>> On Nov 4, Martin Brown<|||newspam...@nezumi.demon.co.uk>  wrote:
>>>>> He estimated music contains 40 bits/second entropy.
>>>>> How close is MP3 to that?
>>>>
>>>>> ___________________________________
>>>>> I doubt surprised Shannon said that, and if he did its
>>>>> somewhere between meaningless and wrong.
>>>>
>>>>> CD quality mp3s are roughly equivalent to 178 kbps,
>>>>> over 4,000 times his estimate. But then you can
>>>>> encode a lot of sounds that most people would not
>>>>> consider music. And it stereo, so you can halve it
>>>>> if Shannon was talking about mono.
>>>>
>>>> Have you never seen sheet music? That is what
>>>> Shannon was estimating - the bitrate for describing
>>>> music in the abstract.
>>>> There are a finite number of notes, durations and
>>>> amplitudes in a classical composition.
>>>
>>> I suspect that's the case.
>>>
>>>> I suspect 40 bits/sec is still far too tight,
>>>> but a midi stream using a high end reconstruction
>>>> codec represents a pretty good
>>>> example of what is possible by way of compression
>>>> for *music* as opposed to voice or a random noise
>>>> stream.
>>>>
>>>>> To get a smaller figure for music, you have to
>>>>> define what subsets of sounds are music. Lots of luck.
>>>
>>> Not so hard, if you analyze samples of what we label 'music'.
>>>
>>>> I think that may have been his intention although
>>>> I don't recall seeing the 40 bit/s number
>>>> As I said if he did anything I think he was estimating
>>>> the information content of music in the already
>>>> concise form of an orchestral score.
>>>
>>> That appears to be the case.
>>>
>>>> I reckon at a bare minimum about 7 to the note, 8 to
>>>> amplitude, 6 duration, 5 to the instrument - and it
>>>> is already obvious that you cannot encode more than a
>>>> single note per second at this bitrate.
>>>
>>> You fail to account for the essential concept
>>> of correlation.
>>>
>>>> Can anyone provide a citation to this alleged paper
>>>> on music bitrate?
>>>
>>>
>>> I found the reference in John Pierce's book.
>>> It's not attributed to Shannon.  My boner.
>>> He claims a humans can absorb information,
>>> e.g. reading or music, at 40 bits/second.
>
> That sounds way too low. I can flash read this entire page in about second 
> and there are a lot more than 40 bits of data in it. And the processing to 
> do that much text recognition in real time would tax a super computer - 
> that is why Captcha is used to test for humans!
>
>>> He also cites a paper where it's claimed that
>>> sheet music contains 2.5 bits entropy per note.
>
> Again sounds far too low even when you only allow for pitch, duration and 
> amplitude (plus any changes in amplitude, incidentals, time signature or 
> syncopation). Please can you post the citation?
>
> I would only believe this claim iff they can demonstrate a real musical 
> score compressed to the equivalent of 2.5bits/note. That is the acid test. 
> Hand waving arguments and gut feel do not hack it.
>
>>> This supports the MIDI model, as opposed to
>>> digitization of a recording.  In that case,
>>> what is the bit rate?  Though the question of
>>> chords arises, with much redundancy there.
>>>
>>> It also raises the possibility that recorded
>>> music might be adequately represented by a
>>> MIDI stream.
>>
>> IIRC, it's interesting to note that most modern media
>> compression codecs rely not so much upon what the raw
>> data can absolutely do without as they do upon what the
>> human ear or eye can be fooled by: they leave out what
>> you "cannot" hear or "cannot" see.  Really, this is
>> nothing new: all screens of any sort are really just
>> optical illusions of one sort or another.  The codecs
>> are just leveraging the limitations of the wiring of
>> the human brain.
>
> Absolutely. There is no point in transmitting detail that the ear cannot 
> hear or the eye cannot see. But you do have to be a little bit careful. 
> The UK's new DAB audio bitrate is insufficient to support quality live 
> broadcast of classical music and is distinctly inferior to either FM or 
> off satellite SD or HD digital audio. The problems with DAB are 
> sufficiently bad that it is hampering uptake of new DAB radios.
>
> Regards,
> Martin Brown

And here in Australia. The standard allows for 128 kbps or 2 x 64 kbps, in 
practice most DAB broadcasts are stereo compressed into a single 64 kbps 
channel. This gives a quality less than the FM radio it is replacing. When 
everything else is moving to high definition, I have no idea why the new 
standard for digital sound broadcasts is such low definition. Very 
shortsighted.

[toc] | [prev] | [next] | [standalone]

#6686

"Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> wrote in message 
news:4eb3b1c6$0$13394$afc38c87@news.optusnet.com.au...
|
|
|
| He estimated music contains 40 bits/second entropy.
| How close is MP3 to that?
|
| ___________________________________
| I doubt surprised Shannon said that, and if he did its somewhere between
| meaningless and wrong.
|
| CD quality mp3s are roughly equivalent to 178 kbps, over 4,000 times his
| estimate. But then you can encode a lot of sounds that most people would 
not
| consider music. And it stereo, so you can halve it if Shannon was talking
| about mono.

Stereo is multiplexed. When the left channel is played the right channel is
MISSING, and vice versa. One of the earliest compression algorithms was
run length encoding, whereby a byte or short string of bytes is transmitted
as one byte and a count for the number of times the byte is to be repeated.
When the doctor tells you to say "ahhhhhh....." then that encodes to 'a'
once, 'h' 6 times, '.' 5 times.  "a1h6.5".
Text (word) compression is very easy, construct 16 lists of words
with that number of letters, up to 256 words per list. That gives a
vocabulary of up to 4096 words. Now encode each word by its number
of letters and its entry into its list using 2 bytes, and you still have 4 
bits
left over for punctuation. For example, the word "doctor" is to be found
in the list of 6 letter words and might be the 32nd entry, so it is encoded
as 06:32, two bytes instead of seven, the trailing space can be assumed
unless modified by the punctuation nybble.

[toc] | [prev] | [next] | [standalone]

#6681

RichD <r_delaney2001@yahoo.com> writes:

>He estimated music contains 40 bits/second entropy.
>How close is MP3 to that?

Far away; apparently not a person who paid topdollar for gold plated
audio cables.

Casper
-- 

[toc] | [prev] | [next] | [standalone]

#6704

["Followup-To:" header set to sci.crypt.]
On 2011-11-04, Casper H.S  Dik <Casper.Dik@OrSPaMcle.COM> wrote:
> RichD <r_delaney2001@yahoo.com> writes:
>
>>He estimated music contains 40 bits/second entropy.
>>How close is MP3 to that?
>
> Far away; apparently not a person who paid topdollar for gold plated
> audio cables.

Even had they existed, he wouldn't have. He was an engineer, for whom
looks and cost did not determine function. 

>
> Casper

[toc] | [prev] | [next] | [standalone]

#6737

On Nov 2, 5:08 pm, VWWall <vw...@large.invalid> wrote:
> RichD wrote:
> > On Nov 2, Chris <chris.sant...@gmail.com> wrote:
> >> Be careful going down this path. For me it was
> >> much more interesting in theory than in practice.
>
> >> Me in 2008: "Like, what *is* the nature of information,
> >> man?" (waves hand to clear smoke from the room)
>
> >> Then I signed up for Info Theory, and I spent
> >> a whole semester taking integrals of logs of
> >> probability distributions and proving that one
> >> was bounded by the other.
>
> > It's too bad you got so little out of it.
> > It's not 'just a theory', it's a field
> > where theory and practice coincide.
> > The theorists concoct the recipes, the
> > engineers follow the instructions, and
> > it works, bang on.  Fukengrooven.
> > And it's the root of many a Silly Con
> > Valley fortune.
>
> > I strongly suggest, if you have some
> > free neurons and time and nothing's good on
> > tv, read Shannon's original paper.  It's
> > utterly fucking brilliant.  There's no doubt,
> > in Valhalla, Shannon shares a table with Einstein.
>
> I was at Bell Labs in 1949, and Betty Moore, Claude Shannon's wife,
> worked in the same department.  She complained that their new carpet
> didn't have deep enough nap to hide the solder droppings from the
> computer Claude was building.  It could perform math functions with its
> input/output being in Roman numerals.

  Well, but Bell Labs is why the people who really know how solder
works
  invented Artificial Satellites and Lasers, rather than transistors
anyway,

  And it's also why the people who know how i/o works, invented things
other than Hz.



>
> > It's sort of funny, he called it communication
> > theory, it was his disciples who popularized
> > 'information theory'.
>
> Shannon took a very practical outlook on "information".  I recall how
> he would stop people in the halls and ask them to guess the next letter
> in a sentence he'd show them.  I remember one as: "A motorcycle has no
> reverse; it can not back up."  He used this method to determine the
> redundancy in the English language.
>
> Most of the time, he was riding his unicycle down the hall!
>
> --
> Virg Wall

[toc] | [prev] | [next] | [standalone]

#6744

On Nov 6, 8:11 pm, jim <retnuh2...@gmail.com> wrote:
> On Nov 2, 5:08 pm, VWWall <vw...@large.invalid> wrote:
>
>
>
>
>
>
>
>
>
> > RichD wrote:
> > > On Nov 2, Chris <chris.sant...@gmail.com> wrote:
> > >> Be careful going down this path. For me it was
> > >> much more interesting in theory than in practice.
>
> > >> Me in 2008: "Like, what *is* the nature of information,
> > >> man?" (waves hand to clear smoke from the room)
>
> > >> Then I signed up for Info Theory, and I spent
> > >> a whole semester taking integrals of logs of
> > >> probability distributions and proving that one
> > >> was bounded by the other.
>
> > > It's too bad you got so little out of it.
> > > It's not 'just a theory', it's a field
> > > where theory and practice coincide.
> > > The theorists concoct the recipes, the
> > > engineers follow the instructions, and
> > > it works, bang on.  Fukengrooven.
> > > And it's the root of many a Silly Con
> > > Valley fortune.
>
> > > I strongly suggest, if you have some
> > > free neurons and time and nothing's good on
> > > tv, read Shannon's original paper.  It's
> > > utterly fucking brilliant.  There's no doubt,
> > > in Valhalla, Shannon shares a table with Einstein.
>
> > I was at Bell Labs in 1949, and Betty Moore, Claude Shannon's wife,
> > worked in the same department.  She complained that their new carpet
> > didn't have deep enough nap to hide the solder droppings from the
> > computer Claude was building.  It could perform math functions with its
> > input/output being in Roman numerals.
>
>   Well, but Bell Labs is why the people who really know how solder
> works
>   invented Artificial Satellites and Lasers, rather than transistors
> anyway,
>
>   And it's also why the people who know how i/o works, invented things
> other than Hz.
>
   Which is mostly a way telling philosophers, that nobody ever
claimed
   that simple machines don't work. Just that the people who know how
   infrared really works invented self-replicating machines, since
that's
   the way infrared really works. The people who know how Ultra Violet
really works
   invented Lasers and Holograms, since that's the way UV really
works.
   The people who know how Xrays really work, invented nanotechnology,
   since that the way Xrays really work. The people who know how gamma
rays
   really work invented GPS and Space Shuttles, since that's the way
   gamma rays really work. The people who know how Digital really
works
   invented USB and miniature atomic clocks, since that's the way
digital
   really works. The people who know how radar really works, invented
   optical networks, since that's the way radar really works.dd


>
>
>
>
>
>
>
> > > It's sort of funny, he called it communication
> > > theory, it was his disciples who popularized
> > > 'information theory'.
>
> > Shannon took a very practical outlook on "information".  I recall how
> > he would stop people in the halls and ask them to guess the next letter
> > in a sentence he'd show them.  I remember one as: "A motorcycle has no
> > reverse; it can not back up."  He used this method to determine the
> > redundancy in the English language.
>
> > Most of the time, he was riding his unicycle down the hall!
>
> > --
> > Virg Wall

[toc] | [prev] | [next] | [standalone]

#6758

On Nov 7, 9:23 am, jim <retnuh2...@gmail.com> wrote:
> On Nov 6, 8:11 pm, jim <retnuh2...@gmail.com> wrote:
>
>
>
>
>
>
>
> > On Nov 2, 5:08 pm, VWWall <vw...@large.invalid> wrote:
>
> > > RichD wrote:
> > > > On Nov 2, Chris <chris.sant...@gmail.com> wrote:
> > > >> Be careful going down this path. For me it was
> > > >> much more interesting in theory than in practice.
>
> > > >> Me in 2008: "Like, what *is* the nature of information,
> > > >> man?" (waves hand to clear smoke from the room)
>
> > > >> Then I signed up for Info Theory, and I spent
> > > >> a whole semester taking integrals of logs of
> > > >> probability distributions and proving that one
> > > >> was bounded by the other.
>
> > > > It's too bad you got so little out of it.
> > > > It's not 'just a theory', it's a field
> > > > where theory and practice coincide.
> > > > The theorists concoct the recipes, the
> > > > engineers follow the instructions, and
> > > > it works, bang on.  Fukengrooven.
> > > > And it's the root of many a Silly Con
> > > > Valley fortune.
>
> > > > I strongly suggest, if you have some
> > > > free neurons and time and nothing's good on
> > > > tv, read Shannon's original paper.  It's
> > > > utterly fucking brilliant.  There's no doubt,
> > > > in Valhalla, Shannon shares a table with Einstein.
>
> > > I was at Bell Labs in 1949, and Betty Moore, Claude Shannon's wife,
> > > worked in the same department.  She complained that their new carpet
> > > didn't have deep enough nap to hide the solder droppings from the
> > > computer Claude was building.  It could perform math functions with its
> > > input/output being in Roman numerals.
>
> >   Well, but Bell Labs is why the people who really know how solder
> > works
> >   invented Artificial Satellites and Lasers, rather than transistors
> > anyway,
>
> >   And it's also why the people who know how i/o works, invented things
> > other than Hz.
>
>    Which is mostly a way telling philosophers, that nobody ever
> claimed
>    that simple machines don't work. Just that the people who know how
>    infrared really works invented self-replicating machines, since
> that's
>    the way infrared really works. The people who know how Ultra Violet
> really works
>    invented Lasers and Holograms, since that's the way UV really
> works.
>    The people who know how Xrays really work, invented nanotechnology,
>    since that the way Xrays really work. The people who know how gamma
> rays
>    really work invented GPS and Space Shuttles, since that's the way
>    gamma rays really work. The people who know how Digital really
> works
>    invented USB and miniature atomic clocks, since that's the way
> digital
>    really works. The people who know how radar really works, invented
>    optical networks, since that's the way radar really works.dd
>
    And the people who know how computers really work, invented
    LEDs, Digital Photography, Flash Memory, and 21st Century
Computers.
    Rather than reinventing Vacuum Tubes.

>
>
>
>
>
>
> > > > It's sort of funny, he called it communication
> > > > theory, it was his disciples who popularized
> > > > 'information theory'.
>
> > > Shannon took a very practical outlook on "information".  I recall how
> > > he would stop people in the halls and ask them to guess the next letter
> > > in a sentence he'd show them.  I remember one as: "A motorcycle has no
> > > reverse; it can not back up."  He used this method to determine the
> > > redundancy in the English language.
>
> > > Most of the time, he was riding his unicycle down the hall!
>
> > > --
> > > Virg Wall

[toc] | [prev] | [next] | [standalone]

#6795

On Nov 7, 9:23 am, jim <retnuh2...@gmail.com> wrote:
> On Nov 6, 8:11 pm, jim <retnuh2...@gmail.com> wrote:
>
>
>
>
>
>
>
> > On Nov 2, 5:08 pm, VWWall <vw...@large.invalid> wrote:
>
> > > RichD wrote:
> > > > On Nov 2, Chris <chris.sant...@gmail.com> wrote:
> > > >> Be careful going down this path. For me it was
> > > >> much more interesting in theory than in practice.
>
> > > >> Me in 2008: "Like, what *is* the nature of information,
> > > >> man?" (waves hand to clear smoke from the room)
>
> > > >> Then I signed up for Info Theory, and I spent
> > > >> a whole semester taking integrals of logs of
> > > >> probability distributions and proving that one
> > > >> was bounded by the other.
>
> > > > It's too bad you got so little out of it.
> > > > It's not 'just a theory', it's a field
> > > > where theory and practice coincide.
> > > > The theorists concoct the recipes, the
> > > > engineers follow the instructions, and
> > > > it works, bang on.  Fukengrooven.
> > > > And it's the root of many a Silly Con
> > > > Valley fortune.
>
> > > > I strongly suggest, if you have some
> > > > free neurons and time and nothing's good on
> > > > tv, read Shannon's original paper.  It's
> > > > utterly fucking brilliant.  There's no doubt,
> > > > in Valhalla, Shannon shares a table with Einstein.
>
> > > I was at Bell Labs in 1949, and Betty Moore, Claude Shannon's wife,
> > > worked in the same department.  She complained that their new carpet
> > > didn't have deep enough nap to hide the solder droppings from the
> > > computer Claude was building.  It could perform math functions with its
> > > input/output being in Roman numerals.
>
> >   Well, but Bell Labs is why the people who really know how solder
> > works
> >   invented Artificial Satellites and Lasers, rather than transistors
> > anyway,
>
> >   And it's also why the people who know how i/o works, invented things
> > other than Hz.
>
>    Which is mostly a way telling philosophers, that nobody ever
> claimed
>    that simple machines don't work. Just that the people who know how
>    infrared really works invented self-replicating machines, since
> that's
>    the way infrared really works. The people who know how Ultra Violet
> really works
>    invented Lasers and Holograms, since that's the way UV really
> works.
>    The people who know how Xrays really work, invented nanotechnology,
>    since that the way Xrays really work. The people who know how gamma
> rays
>    really work invented GPS and Space Shuttles, since that's the way
>    gamma rays really work. The people who know how Digital really
> works
>    invented USB and miniature atomic clocks, since that's the way
> digital
>    really works. The people who know how radar really works, invented
>    optical networks, since that's the way radar really works.dd
>

    The people who know how nuclear propulsion works invented nuclear
cruise missiles.
    The people who know how ICBMs work invented 21st Century
Helicopters.
    The people who know how idiot tanks work invented composite
materials, reactive armor, phalanx,
    wireless telecomm,  battery traps, and 21st century calibers.


>
>
>
>
>
>
> > > > It's sort of funny, he called it communication
> > > > theory, it was his disciples who popularized
> > > > 'information theory'.
>
> > > Shannon took a very practical outlook on "information".  I recall how
> > > he would stop people in the halls and ask them to guess the next letter
> > > in a sentence he'd show them.  I remember one as: "A motorcycle has no
> > > reverse; it can not back up."  He used this method to determine the
> > > redundancy in the English language.
>
> > > Most of the time, he was riding his unicycle down the hall!
>
> > > --
> > > Virg Wall

[toc] | [prev] | [next] | [standalone]

#6802

jim wrote:
> 
>     The people who know 


   Plonk you.


-- 
You can't have a sense of humor, if you have no sense.

[toc] | [prev] | [next] | [standalone]

#6655

On 11/2/2011 1:51 PM, RichD wrote:

> I strongly suggest, if you have some
> free neurons and time and nothing's good on
> tv, read Shannon's original paper.  It's
> utterly fucking brilliant.  There's no doubt,
> in Valhalla, Shannon shares a table with Einstein.
>
> It's sort of funny, he called it communication
> theory, it was his disciples who popularized
> 'information theory'.

Not funny at all. He worked for Bell Telephone. Duh! And indeed the 
original work DID have the major thrust of a theory of communication. 
Later the breakthrough nature of his ideas were recognized as having 
much wider general application in MANY at first seemingly unrelated 
fields. Hence the name with a wider swath: Information theory.

And yes Shannon's ideas on information including those on Cryptography 
(some probably still classified) definitely ARE "utterly fucking 
brilliant"!

[toc] | [prev] | [next] | [standalone]

#6663

On Nov 2, 2:51 pm, RichD <r_delaney2...@yahoo.com> wrote:
> On Nov 2, Chris <chris.sant...@gmail.com> wrote:
>
> > Be careful going down this path. For me it was
> > much more interesting in theory than in practice.
>
> > Me in 2008: "Like, what *is* the nature of information,
> > man?" (waves hand to clear smoke from the room)
>
> > Then I signed up for Info Theory, and I spent
> > a whole semester taking integrals of logs of
> > probability distributions and proving that one
> > was bounded by the other.
>
> It's too bad you got so little out of it.
> It's not 'just a theory', it's a field
> where theory and practice coincide.
> The theorists concoct the recipes, the
> engineers follow the instructions, and
> it works, bang on.  Fukengrooven.
> And it's the root of many a Silly Con
> Valley fortune.
>
> I strongly suggest, if you have some
> free neurons and time and nothing's good on
> tv, read Shannon's original paper.  It's
> utterly fucking brilliant.  There's no doubt,
> in Valhalla, Shannon shares a table with Einstein.
>
> It's sort of funny, he called it communication
> theory, it was his disciples who popularized
> 'information theory'.
>
> --
> Rich

One of the coolest ideas I remember learning about is directed
divergence:

http://en.wikipedia.org/wiki/Kullback–Leibler_divergence

and being able to estimate how much it will cost you in bits if your
estimate of the underlying probability distribution of the information
you're trying to code wrong. I'd cite the fact that - even though I'm
not a super expert on all the topics - the fact that I can pick up
these topics again rather quickly as evidence that dipping my toes in
the field was worth it. School's not about becoming an expert - it's
about working the muscles so you can become an expert if you need to.

[toc] | [prev] | [next] | [standalone]

#6664

On Nov 2, 2:51 pm, RichD <r_delaney2...@yahoo.com> wrote:
> On Nov 2, Chris <chris.sant...@gmail.com> wrote:
>
> > Be careful going down this path. For me it was
> > much more interesting in theory than in practice.
>
> > Me in 2008: "Like, what *is* the nature of information,
> > man?" (waves hand to clear smoke from the room)
>
> > Then I signed up for Info Theory, and I spent
> > a whole semester taking integrals of logs of
> > probability distributions and proving that one
> > was bounded by the other.
>
> It's too bad you got so little out of it.
> It's not 'just a theory', it's a field
> where theory and practice coincide.
> The theorists concoct the recipes, the
> engineers follow the instructions, and
> it works, bang on.  Fukengrooven.
> And it's the root of many a Silly Con
> Valley fortune.
>
> I strongly suggest, if you have some
> free neurons and time and nothing's good on
> tv, read Shannon's original paper.  It's
> utterly fucking brilliant.  There's no doubt,
> in Valhalla, Shannon shares a table with Einstein.
>
> It's sort of funny, he called it communication
> theory, it was his disciples who popularized
> 'information theory'.
>
> --
> Rich

Oh, I'm not saying I got little from it. On the contrary. It comes up
all the time. I'm just saying that the nitty gritty details can be
painfully boring to work though when disconnected from cool
applications.

[toc] | [prev] | [standalone]

Page 4 of 4 — ← Prev page 1 2 3 [4]

Back to top | Article view | comp.dsp

csiph-web