Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.dsp > #6597 > unrolled thread

information theory?

Back to article view | Back to comp.dsp

Contents

  information theory? RichD <r_delaney2001@yahoo.com> - 2011-10-31 20:55 -0700
    Re: information theory? eric.jacobsen@ieee.org (Eric Jacobsen) - 2011-11-01 04:57 +0000
      Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-01 05:58 -0700
        Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-01 10:29 -0700
          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 11:13 -0700
            Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 14:39 -0700
          Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-01 16:04 -0400
            OT: information theory? Jerry Avins <jya@ieee.org> - 2011-11-01 18:06 -0400
            Re: information theory? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-11-01 22:46 +0000
          Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-01 22:38 -0500
            Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-11-02 17:06 -0500
              Re: information theory? "steveu" <steveu@n_o_s_p_a_m.coppice.org> - 2011-11-02 19:41 -0500
              Re: information theory? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-11-03 06:58 -0500
                Re: information theory? "GO-HERE .NL" <gdewilde@gmail.com> - 2011-11-03 08:57 -0700
            Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-02 19:44 -0500
              Re: information theory? "Christopher J. Henrich" <chenrich@monmouth.com> - 2011-11-02 22:40 -0400
                Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-03 23:21 -0500
          Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 08:47 +0000
            Re: information theory? (copyright question followup) JohnF <john@please.see.sig.for.email.com> - 2011-11-02 09:48 +0000
              Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 10:02 +0000
                Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 16:46 +0000
                Re: information theory? (copyright question followup) Bernd Jendrissek <bernd.jendrissek@gmail.com> - 2011-11-02 15:04 -0700
              Re: information theory? (copyright question followup) David Eather <eather@tpg.com.au> - 2011-11-03 04:00 +1000
                Re: information theory? (copyright question followup) unruh <unruh@physics.ubc.ca> - 2011-11-02 19:05 +0000
              Re: information theory? (copyright question followup) "J. Clarke" <jclarkeusenet@cox.net> - 2011-11-24 08:42 -0500
                Re: information theory? (copyright question followup) Richard Outerbridge <outer@interlog.com> - 2011-11-25 01:37 -0500
                  Re: information theory? (copyright question followup) Les Cargill <lcargill99@comcast.com> - 2011-11-25 11:37 -0600
            Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:38 -0700
              Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-03 08:51 +0000
          Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-02 08:02 -0700
            Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-02 09:38 -0700
            Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:51 -0700
              Re: information theory? VWWall <vwall@large.invalid> - 2011-11-02 15:08 -0700
                Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-11-02 19:59 -0400
                  Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-04 20:41 +0000
                    Re: information theory? "krw@att.bizzzzzzzzzzzz" <krw@att.bizzzzzzzzzzzz> - 2011-11-04 16:59 -0500
                      Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 15:28 -0700
                      Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 21:09 -0400
                        Re: information theory? ggr@nope.ucsd.edu (Greg Rose) - 2011-11-05 02:11 +0000
                        Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-05 11:53 +0000
                      Re: information theory? "Mike Terry" <news.dead.person.stones@darjeeling.plus.com> - 2011-11-05 17:32 +0000
                  Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 18:41 -0800
                Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-03 21:54 -0700
                  Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 03:42 -0500
                  Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 20:33 +1100
                    Re: information theory? Robert Wessel <robertwessel2@yahoo.com> - 2011-11-04 05:10 -0500
                      Re: information theory? kym@kymhorsell.com - 2011-11-04 10:50 +0000
                        Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 22:14 +1100
                          Re: information theory? "Jesse F. Hughes" <jesse@phiwumbda.org> - 2011-11-04 09:13 -0400
                            Re: information theory? Spehro Pefhany <speffSNIP@interlogDOTyou.knowwhat> - 2011-11-04 10:32 -0400
                              Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 18:02 +0000
                                Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 15:54 -0800
                        Re: information theory? VWWall <vwall@large.invalid> - 2011-11-04 09:50 -0700
                          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 12:41 -0700
                          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 13:40 -0700
                      Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:35 +0000
                    Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-04 10:31 +0000
                      Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 21:51 +1100
                      Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 14:18 -0500
                      Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 15:56 -0400
                      Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-12-21 17:49 -0800
                        Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-12-21 20:07 -0600
                        Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-12-22 03:08 -0500
                          Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-12-22 08:30 +0000
                            Re: information theory? John Devereux <john@devereux.me.uk> - 2011-12-22 10:12 +0000
                            Re: information theory? "Peter Webb" <r.peter.webbbbb@gmail.com> - 2011-12-22 21:14 +1100
                    Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:00 +0000
                  Re: information theory? Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2011-11-04 09:45 +0000
                    Re: information theory? unruh <unruh@physics.ubc.ca> - 2011-11-04 19:05 +0000
                Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 17:11 -0800
                  Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 06:23 -0800
                    Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 11:29 -0800
                    Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-08 04:41 -0800
                      Re: information theory? "Michael A. Terrell" <mike.terrell@earthlink.net> - 2011-11-08 13:46 -0500
              Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-03 03:32 -0500
              Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:39 -0700
              Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:28 -0700

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#6726

<krw@att.bizzzzzzzzzzzz> wrote in message
news:ppn8b7d67lc2f3v7hickc6e63rt1llkgtp@4ax.com...
> On Fri, 04 Nov 2011 20:41:09 +0000, Frederick Williams
> <freddywilliams@btinternet.com> wrote:
>
> >Richard Outerbridge wrote:
> >
> >> The seminal introductory work for me was the Undergraduate text
> >> "Coding and Information Theory", by the late Richard W. Hamming,
> >> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
> >> an ISBN number
> >
> >Hardly, they date from 1970.  Also, it's "ISBN" not "ISBN number".
>
> Tight-assed pedants hate "ATM machines", too, right?

That's because they can't remember their PIN numbers.

[toc] | [prev] | [next] | [standalone]

#6739

On Nov 2, 6:59 pm, Richard Outerbridge <ou...@interlog.com> wrote:
> In article <E8udnR4LarelIizTnZ2dnUVZ_ridn...@earthlink.com>,
>
>  VWWall <vw...@large.invalid> wrote:
>
> [....]
>
>
>
>
>
>
>
>
>
> > I was at Bell Labs in 1949, and Betty Moore, Claude Shannon's wife,
> > worked in the same department.  She complained that their new carpet
> > didn't have deep enough nap to hide the solder droppings from the
> > computer Claude was building.  It could perform math functions with its
> > input/output being in Roman numerals.
>
> > > It's sort of funny, he called it communication
> > > theory, it was his disciples who popularized
> > > 'information theory'.
>
> > Shannon took a very practical outlook on "information".  I recall how
> > he would stop people in the halls and ask them to guess the next letter
> > in a sentence he'd show them.  I remember one as: "A motorcycle has no
> > reverse; it can not back up."  He used this method to determine the
> > redundancy in the English language.
>
> > Most of the time, he was riding his unicycle down the hall!
>
> The seminal introductory work for me was the Undergraduate text
> "Coding and Information Theory", by the late Richard W. Hamming,
> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
> an ISBN number).  It cost me $26.35 (CDN), hardcover, and I
> devoured every morsel of it.
>
> His later work "The Art of Probability" is also worth a read for
> anyone who still believes in randomness.

   Since the only people who don't believe in randomness is
philosophers,
   that's why the theory of evolution was discovered quite epochs ago,
anyway.
   And they can play with their mental masturbator number genators.

   And since the only people who believe QM isn't random is chemists,
   that's also why they're the only people that still work in jerk
ball
   dark energy laboratories.


>
> outer
>
> --
> Random : An infinitesimal, yet omnidimensional, god of science.
> Random often appears in the guise of a trickster named Error.

[toc] | [prev] | [next] | [standalone]

#6676

On Nov 2, VWWall <vw...@large.invalid> wrote:
> > I strongly suggest, if you have some
> > free neurons and time and nothing's good on
> > tv, read Shannon's original paper.  It's
> > utterly fucking brilliant.  There's no doubt,
> > in Valhalla, Shannon shares a table with Einstein.
>
> I was at Bell Labs in 1949, and Betty Moore, Claude
> Shannon's wife, worked in the same department.

Holy Methuselah, Batman!  Did you get to shake
Abe Lincoln's hand, too?

> > It's sort of funny, he called it communication
> > theory, it was his disciples who popularized
> > 'information theory'.
>
> Shannon took a very practical outlook on "information".
> I recall how he would stop people in the halls and ask
> them to guess the next letter in a sentence he'd show
 >them.  I remember one as: "A motorcycle has no
> reverse; it can not back up."  He used this method
> to determine the redundancy in the English language.

He estimated music contains 40 bits/second entropy.
How close is MP3 to that?

--
Rich

[toc] | [prev] | [next] | [standalone]

#6678

On 11/3/2011 11:54 PM, RichD wrote:

> He estimated music contains 40 bits/second entropy.
> How close is MP3 to that?

And .jpg sucks too. Which shows you that in spite of the years since the 
original break-throughs most people haven't a clue as to what is going 
on in this area. That does not mean, however that NOBODY understands it. 
I DO know of people (working in the telecommunications industry as 
Shannon did) who do understand this and have developed compression 
programs that actually work. Eventually I suppose they will filter down 
to us peons.

[toc] | [prev] | [next] | [standalone]

#6680

He estimated music contains 40 bits/second entropy.
How close is MP3 to that?

___________________________________
I doubt surprised Shannon said that, and if he did its somewhere between 
meaningless and wrong.

CD quality mp3s are roughly equivalent to 178 kbps, over 4,000 times his 
estimate. But then you can encode a lot of sounds that most people would not 
consider music. And it stereo, so you can halve it if Shannon was talking 
about mono.

And how do you define music, except as sound? And random sound waveforms 
cannot be compressed on average at all. To get a smaller figure for music, 
you have to define what subsets of sounds are music. Lots of luck.

The real number probably lies somewhere between 178 kbps and 40 bps. The 
actual number is the base 2 logarithm of the number of different 1 second 
sound bites that the ear can distuingish and would consider as music. I 
doubt even Shannon would have known the answer to that equation.

[toc] | [prev] | [next] | [standalone]

#6682

On Fri, 4 Nov 2011 20:33:48 +1100, "Peter Webb"
<webbfamily@optusnetDIESPAMDIE.com.au> wrote:

>
>
>
>He estimated music contains 40 bits/second entropy.
>How close is MP3 to that?
>
>___________________________________
>I doubt surprised Shannon said that, and if he did its somewhere between 
>meaningless and wrong.
>
>CD quality mp3s are roughly equivalent to 178 kbps, over 4,000 times his 
>estimate. But then you can encode a lot of sounds that most people would not 
>consider music. And it stereo, so you can halve it if Shannon was talking 
>about mono.
>
>And how do you define music, except as sound? And random sound waveforms 
>cannot be compressed on average at all. To get a smaller figure for music, 
>you have to define what subsets of sounds are music. Lots of luck.
>
>The real number probably lies somewhere between 178 kbps and 40 bps. The 
>actual number is the base 2 logarithm of the number of different 1 second 
>sound bites that the ear can distuingish and would consider as music. I 
>doubt even Shannon would have known the answer to that equation.


I expect that if Shannon said something like that, he was considering
music to be more along the lines of a MIDI stream than a digitized
sound file.

Assuming a pianist can make 20 keystroke per second (and there are
scores that require that), 40 bits is a bit light, since you need to
refine the timing a bit, and specify the force of the keystroke, as
well as deal with the releases and the pedals (although those should
need a relatively low bit rate).

But all that together would certainly be less than 1000 bits per
second.  That assumes that music is actually random keystrokes at
random times and random forces, at at very high keystroke rate - which
it is, of course, not, no more than English sentences are random
sequences of letters.

Consider that high keystroke rates pretty much require chords, and a
chord is most certainly not three or four random keys, pressed at
different times and forces - rather they're a small subset of the
possible combinations of keys within a hand-span (or two hand-spans in
some cases), hit at the same time, and with the same force (in fact
there are less than about 10,000 chords).  And just like English, you
can't really have random sequences, or you'd just have noise.

Still, 40bps intuitively feels a bit light, but not by that much. I
expect that if Shannon said something like that, he was considering
music to be more along the lines of a MIDI stream than a digitized
sound file.

Assuming a pianist can make 20 keystroke per second (and there are
scores that require that), 40 bits is a bit light, since you need to
refine the timing a bit, and specify the force of the keystroke, as
well as deal with the releases and the pedals (although those should
need a relatively low bit rate).

But all that together would certainly be less than 1000 bits per
second.  That assumes that music is actually random keystrokes at
random times and random forces, at at very high keystroke rate - which
it is, of course, not, no more than English sentences are random
sequences of letters.

Consider that high keystroke rates pretty much require chords, and a
chord is most certainly not three or four random keys, pressed at
different times and forces - rather they're a small subset of the
possible combinations of keys within a hand-span (or two hand-spans in
some cases), hit at the same time, and with the same force (in fact
there are less than about 10,000 chords).  And just like English, you
can't really have random sequences, or you'd just have noise.

Still, 40bps intuitively feels a bit light for some of the busiest
music, but not by that much.  For more typical passages, it seems
adequate.

[toc] | [prev] | [next] | [standalone]

#6684

In sci.math Robert Wessel <robertwessel2@yahoo.com> wrote:
...
> Consider that high keystroke rates pretty much require chords, and a
> chord is most certainly not three or four random keys, pressed at
> different times and forces - rather they're a small subset of the
> possible combinations of keys within a hand-span (or two hand-spans in
> some cases), hit at the same time, and with the same force (in fact
> there are less than about 10,000 chords).  And just like English, you
> can't really have random sequences, or you'd just have noise.
> 
> Still, 40bps intuitively feels a bit light, but not by that much. I
> expect that if Shannon said something like that, he was considering
> music to be more along the lines of a MIDI stream than a digitized
> sound file.
...

Perhaps we could use spoken language as a guide to the auditory bidrate
we've evolved to process. As Shannon estimated, each letter in English
has only a couple bits of entropy, with normal reading speed (that
uses the auditory processing -- the reason you can't count the number
of words in a limerick without using your fingers) around 150 wpm.
Let's call that 25 bps.

40 bps is then about the "square" (in log terms :) of our evolved language
processing speeds. 

Maybe not such a bad estimate.

The same disparity is then seen between normal good voice compression
and the actually bitrate we operate on. I.e. 25 bps "back-end" processing 
speed, yet voice compression being still 1000s of bps.

In some applications  over the years I've used speach-to-text and text-to-speach
over very low speed lines as a slap-dash compression method. With a bit
of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still in
use, but I don't think I've heard of it in preference to straight dsp methods.

-- 
co2 has no climate forcing effect and is not a greenhouse gas and, for that 
matter, neither is water vapour. 
-- BONZO@27-32-240-172 [100s of nyms], 5 Sep 2009

Earth's atmosphere contains natural greenhouse gases (mostly water
vapor, carbon dioxide, and methane) which act to keep the lower layers
of the atmosphere warmer than they otherwise would be without those gases. 
-- Dr Roy W. Spencer, "Global Warming", 2008

This is what the real climate scientist Dr Roy Spencer said.
-- BONZO@27-32-240-172 [daily nymshifter], 3 Mar 2011 16:29 +1100

[toc] | [prev] | [next] | [standalone]

#6687

<kym@kymhorsell.com> wrote in message 
news:4eb3c38a$0$22471$afc38c87@news.optusnet.com.au...
> In sci.math Robert Wessel <robertwessel2@yahoo.com> wrote:
> ...
>> Consider that high keystroke rates pretty much require chords, and a
>> chord is most certainly not three or four random keys, pressed at
>> different times and forces - rather they're a small subset of the
>> possible combinations of keys within a hand-span (or two hand-spans in
>> some cases), hit at the same time, and with the same force (in fact
>> there are less than about 10,000 chords).  And just like English, you
>> can't really have random sequences, or you'd just have noise.
>>
>> Still, 40bps intuitively feels a bit light, but not by that much. I
>> expect that if Shannon said something like that, he was considering
>> music to be more along the lines of a MIDI stream than a digitized
>> sound file.
> ...
>
> Perhaps we could use spoken language as a guide to the auditory bidrate
> we've evolved to process. As Shannon estimated, each letter in English
> has only a couple bits of entropy, with normal reading speed (that
> uses the auditory processing -- the reason you can't count the number
> of words in a limerick without using your fingers) around 150 wpm.
> Let's call that 25 bps.
>
> 40 bps is then about the "square" (in log terms :) of our evolved language
> processing speeds.
>
> Maybe not such a bad estimate.
>
> The same disparity is then seen between normal good voice compression
> and the actually bitrate we operate on. I.e. 25 bps "back-end" processing
> speed, yet voice compression being still 1000s of bps.
>
> In some applications  over the years I've used speach-to-text and 
> text-to-speach
> over very low speed lines as a slap-dash compression method. With a bit
> of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still 
> in
> use, but I don't think I've heard of it in preference to straight dsp 
> methods.
>

I had a play with a TI phoneme generator chip hooked up to a 6502 
microproceesor in 1979 or 80. Couldn't get a decent "fuck" out of it - came 
out as a flat "fut". Times have changed.



> -- 
> co2 has no climate forcing effect and is not a greenhouse gas and, for 
> that
> matter, neither is water vapour.
> -- BONZO@27-32-240-172 [100s of nyms], 5 Sep 2009
>
> Earth's atmosphere contains natural greenhouse gases (mostly water
> vapor, carbon dioxide, and methane) which act to keep the lower layers
> of the atmosphere warmer than they otherwise would be without those gases.
> -- Dr Roy W. Spencer, "Global Warming", 2008
>
> This is what the real climate scientist Dr Roy Spencer said.
> -- BONZO@27-32-240-172 [daily nymshifter], 3 Mar 2011 16:29 +1100 

[toc] | [prev] | [next] | [standalone]

#6690

"Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> writes:

> I had a play with a TI phoneme generator chip hooked up to a 6502 
> microproceesor in 1979 or 80. Couldn't get a decent "fuck" out of it - came 
> out as a flat "fut". Times have changed.

Well, technology has changed, but your choice of test words shows human
nature hasn't.   
-- 
Jesse F. Hughes

"C is for Cookie.  That's good enough for me."  
                                     Cookie Monster

[toc] | [prev] | [next] | [standalone]

#6692

On Fri, 04 Nov 2011 09:13:31 -0400, "Jesse F. Hughes"
<jesse@phiwumbda.org> wrote:

>"Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> writes:
>
>> I had a play with a TI phoneme generator chip hooked up to a 6502 
>> microproceesor in 1979 or 80. Couldn't get a decent "fuck" out of it - came 
>> out as a flat "fut". Times have changed.
>
>Well, technology has changed, but your choice of test words shows human
>nature hasn't.   

Reminds me of this painting...
http://www.barber.org.uk/653.html

Lots of things have changed in ~300 years, but boys have not. 

[toc] | [prev] | [next] | [standalone]

#6703

"Spehro Pefhany" <speffSNIP@interlogDOTyou.knowwhat> wrote in message 
news:6ot7b7d7pkgh2c80oquvrd14chprebkob1@4ax.com...
| On Fri, 04 Nov 2011 09:13:31 -0400, "Jesse F. Hughes"
| <jesse@phiwumbda.org> wrote:
|
| >"Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> writes:
| >
| >> I had a play with a TI phoneme generator chip hooked up to a 6502
| >> microproceesor in 1979 or 80. Couldn't get a decent "fuck" out of it - 
came
| >> out as a flat "fut". Times have changed.
| >
| >Well, technology has changed, but your choice of test words shows human
| >nature hasn't.
|
| Reminds me of this painting...
| http://www.barber.org.uk/653.html
|
| Lots of things have changed in ~300 years, but boys have not.
|
Sex is 3 billion years old and still it fascinates the young who imagine
they invented its variations.

[toc] | [prev] | [next] | [standalone]

#6776

On Nov 4, 1:02 pm, "Androcles" <Headmas...@Hogwarts.physics.October.
2011> wrote:
> "Spehro Pefhany" <speffS...@interlogDOTyou.knowwhat> wrote in message
>
> news:6ot7b7d7pkgh2c80oquvrd14chprebkob1@4ax.com...
> | On Fri, 04 Nov 2011 09:13:31 -0400, "Jesse F. Hughes"| <je...@phiwumbda.org> wrote:
>
> |
> | >"Peter Webb" <webbfam...@optusnetDIESPAMDIE.com.au> writes:
> | >
> | >> I had a play with a TI phoneme generator chip hooked up to a 6502
> | >> microproceesor in 1979 or 80. Couldn't get a decent "fuck" out of it -
> came
> | >> out as a flat "fut". Times have changed.
> | >
> | >Well, technology has changed, but your choice of test words shows human
> | >nature hasn't.
> |
> | Reminds me of this painting...
> |http://www.barber.org.uk/653.html
> |
> | Lots of things have changed in ~300 years, but boys have not.
> |
> Sex is 3 billion years old and still it fascinates the young who imagine
> they invented its variations.

  It even fascinate the idiots in law even more, who believe patent
insurance is sex.

[toc] | [prev] | [next] | [standalone]

#6698

kym@kymhorsell.com wrote:
> In sci.math Robert Wessel <robertwessel2@yahoo.com> wrote:
> ...
>> Consider that high keystroke rates pretty much require chords, and a
>> chord is most certainly not three or four random keys, pressed at
>> different times and forces - rather they're a small subset of the
>> possible combinations of keys within a hand-span (or two hand-spans in
>> some cases), hit at the same time, and with the same force (in fact
>> there are less than about 10,000 chords).  And just like English, you
>> can't really have random sequences, or you'd just have noise.
>>
>> Still, 40bps intuitively feels a bit light, but not by that much. I
>> expect that if Shannon said something like that, he was considering
>> music to be more along the lines of a MIDI stream than a digitized
>> sound file.
> ...
> 
> Perhaps we could use spoken language as a guide to the auditory bidrate
> we've evolved to process. As Shannon estimated, each letter in English
> has only a couple bits of entropy, with normal reading speed (that
> uses the auditory processing -- the reason you can't count the number
> of words in a limerick without using your fingers) around 150 wpm.
> Let's call that 25 bps.
> 
> 40 bps is then about the "square" (in log terms :) of our evolved language
> processing speeds. 
> 
> Maybe not such a bad estimate.
> 
> The same disparity is then seen between normal good voice compression
> and the actually bitrate we operate on. I.e. 25 bps "back-end" processing 
> speed, yet voice compression being still 1000s of bps.
> 
> In some applications  over the years I've used speach-to-text and text-to-speach
> over very low speed lines as a slap-dash compression method. With a bit
> of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still in
> use, but I don't think I've heard of it in preference to straight dsp methods.
> 
At the 1939 World Fair in New York City, Bell Telephone had an exhibit 
featuring the "vocoder".  An operator could synchronize speech by 
pressing "keys" on a console.  Bell thought the method could be used for 
"compressing" its telephone bandwidth requirements.

As it turned out, bandwidth was not the limiting factor in future 
telephone growth.  Simple system changes like separating the low-speed 
"housekeeping" functions from the in-band "information" were implemented.

I was involved in the testing of microwave frequencies in small circular 
wave-guide to allow for much wider band channels.  The need never 
evolved, and fiber optics replaced microwaves where it was needed.

As far as I know, any "compression" system requires some means of 
"storing" information so that it can be processed.  This was just being 
developed during Shannon's time.  It's obvious that "digital" storage is 
easier and more flexible than analog.

-- 
Virg Wall

[toc] | [prev] | [next] | [standalone]

#6706

On Nov 4, 12:50 pm, VWWall <vw...@large.invalid> wrote:
> k...@kymhorsell.com wrote:
> > In sci.math Robert Wessel <robertwess...@yahoo.com> wrote:
> > ...
> >> Consider that high keystroke rates pretty much require chords, and a
> >> chord is most certainly not three or four random keys, pressed at
> >> different times and forces - rather they're a small subset of the
> >> possible combinations of keys within a hand-span (or two hand-spans in
> >> some cases), hit at the same time, and with the same force (in fact
> >> there are less than about 10,000 chords).  And just like English, you
> >> can't really have random sequences, or you'd just have noise.
>
> >> Still, 40bps intuitively feels a bit light, but not by that much. I
> >> expect that if Shannon said something like that, he was considering
> >> music to be more along the lines of a MIDI stream than a digitized
> >> sound file.
> > ...
>
> > Perhaps we could use spoken language as a guide to the auditory bidrate
> > we've evolved to process. As Shannon estimated, each letter in English
> > has only a couple bits of entropy, with normal reading speed (that
> > uses the auditory processing -- the reason you can't count the number
> > of words in a limerick without using your fingers) around 150 wpm.
> > Let's call that 25 bps.
>
> > 40 bps is then about the "square" (in log terms :) of our evolved language
> > processing speeds.
>
> > Maybe not such a bad estimate.
>
> > The same disparity is then seen between normal good voice compression
> > and the actually bitrate we operate on. I.e. 25 bps "back-end" processing
> > speed, yet voice compression being still 1000s of bps.
>
> > In some applications  over the years I've used speach-to-text and text-to-speach
> > over very low speed lines as a slap-dash compression method. With a bit
> > of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still in
> > use, but I don't think I've heard of it in preference to straight dsp methods.
>
> At the 1939 World Fair in New York City, Bell Telephone had an exhibit
> featuring the "vocoder".  An operator could synchronize speech by
> pressing "keys" on a console.  Bell thought the method could be used for
> "compressing" its telephone bandwidth requirements.
>
> As it turned out, bandwidth was not the limiting factor in future
> telephone growth.  Simple system changes like separating the low-speed
> "housekeeping" functions from the in-band "information" were implemented.

   The cranks that work in signal processing are bandwidth obessed
with all
   technology, so that's where digital signal processing, Turing
Machines, Lasers,
   Holograms, miniature atomic clocks, Integrated Circuits, XML,
Optical Networks,
   USB,  LEDS, Flash Memory,  Orbital Solar Energy, and post New York
Times
   ergonomics all came from.


> I was involved in the testing of microwave frequencies in small circular
> wave-guide to allow for much wider band channels.  The need never
> evolved, and fiber optics replaced microwaves where it was needed.
>
> As far as I know, any "compression" system requires some means of
> "storing" information so that it can be processed.  This was just being
> developed during Shannon's time.  It's obvious that "digital" storage is
> easier and more flexible than analog.
>
> --
> Virg Wall

[toc] | [prev] | [next] | [standalone]

#6712

On Nov 4, 12:50 pm, VWWall <vw...@large.invalid> wrote:
> k...@kymhorsell.com wrote:
> > In sci.math Robert Wessel <robertwess...@yahoo.com> wrote:
> > ...
> >> Consider that high keystroke rates pretty much require chords, and a
> >> chord is most certainly not three or four random keys, pressed at
> >> different times and forces - rather they're a small subset of the
> >> possible combinations of keys within a hand-span (or two hand-spans in
> >> some cases), hit at the same time, and with the same force (in fact
> >> there are less than about 10,000 chords).  And just like English, you
> >> can't really have random sequences, or you'd just have noise.
>
> >> Still, 40bps intuitively feels a bit light, but not by that much. I
> >> expect that if Shannon said something like that, he was considering
> >> music to be more along the lines of a MIDI stream than a digitized
> >> sound file.
> > ...
>
> > Perhaps we could use spoken language as a guide to the auditory bidrate
> > we've evolved to process. As Shannon estimated, each letter in English
> > has only a couple bits of entropy, with normal reading speed (that
> > uses the auditory processing -- the reason you can't count the number
> > of words in a limerick without using your fingers) around 150 wpm.
> > Let's call that 25 bps.
>
> > 40 bps is then about the "square" (in log terms :) of our evolved language
> > processing speeds.
>
> > Maybe not such a bad estimate.
>
> > The same disparity is then seen between normal good voice compression
> > and the actually bitrate we operate on. I.e. 25 bps "back-end" processing
> > speed, yet voice compression being still 1000s of bps.
>
> > In some applications  over the years I've used speach-to-text and text-to-speach
> > over very low speed lines as a slap-dash compression method. With a bit
> > of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still in
> > use, but I don't think I've heard of it in preference to straight dsp methods.
>
> At the 1939 World Fair in New York City, Bell Telephone had an exhibit
> featuring the "vocoder".  An operator could synchronize speech by
> pressing "keys" on a console.  Bell thought the method could be used for
> "compressing" its telephone bandwidth requirements.
>
> As it turned out, bandwidth was not the limiting factor in future
> telephone growth.  Simple system changes like separating the low-speed
> "housekeeping" functions from the in-band "information" were implemented.
>
> I was involved in the testing of microwave frequencies in small circular
> wave-guide to allow for much wider band channels.  The need never
> evolved, and fiber optics replaced microwaves where it was needed.
>
> As far as I know, any "compression" system requires some means of
> "storing" information so that it can be processed.  This was just being
> developed during Shannon's time.  It's obvious that "digital" storage is
> easier and more flexible than analog.

  But that's only really the first stage or filtering set of signal
processing itself.
  So, that's also why the people that actually know how the rest of
the stuff works
  started working on satellites, lasers, digital controllers, optical
networks,
  holographics, USB, digital TV, and nanotechnology.


> --
> Virg Wall

[toc] | [prev] | [next] | [standalone]

#6688

"Robert Wessel" <robertwessel2@yahoo.com> wrote in message 
news:r6d7b7l6nt23kcu3n0cisoe85j0d6go5no@4ax.com...
| On Fri, 4 Nov 2011 20:33:48 +1100, "Peter Webb"
| <webbfamily@optusnetDIESPAMDIE.com.au> wrote:
|
| >
| >
| >
| >He estimated music contains 40 bits/second entropy.
| >How close is MP3 to that?
| >
| >___________________________________
| >I doubt surprised Shannon said that, and if he did its somewhere between
| >meaningless and wrong.
| >
| >CD quality mp3s are roughly equivalent to 178 kbps, over 4,000 times his
| >estimate. But then you can encode a lot of sounds that most people would 
not
| >consider music. And it stereo, so you can halve it if Shannon was talking
| >about mono.
| >
| >And how do you define music, except as sound? And random sound waveforms
| >cannot be compressed on average at all. To get a smaller figure for 
music,
| >you have to define what subsets of sounds are music. Lots of luck.
| >
| >The real number probably lies somewhere between 178 kbps and 40 bps. The
| >actual number is the base 2 logarithm of the number of different 1 second
| >sound bites that the ear can distuingish and would consider as music. I
| >doubt even Shannon would have known the answer to that equation.
|
|
| I expect that if Shannon said something like that, he was considering
| music to be more along the lines of a MIDI stream than a digitized
| sound file.
|
| Assuming a pianist can make 20 keystroke per second (and there are
| scores that require that), 40 bits is a bit light, since you need to
| refine the timing a bit, and specify the force of the keystroke, as
| well as deal with the releases and the pedals (although those should
| need a relatively low bit rate).
|
| But all that together would certainly be less than 1000 bits per
| second.  That assumes that music is actually random keystrokes at
| random times and random forces, at at very high keystroke rate - which
| it is, of course, not, no more than English sentences are random
| sequences of letters.
|
| Consider that high keystroke rates pretty much require chords, and a
| chord is most certainly not three or four random keys, pressed at
| different times and forces - rather they're a small subset of the
| possible combinations of keys within a hand-span (or two hand-spans in
| some cases), hit at the same time, and with the same force (in fact
| there are less than about 10,000 chords).  And just like English, you
| can't really have random sequences, or you'd just have noise.
|
| Still, 40bps intuitively feels a bit light, but not by that much. I
| expect that if Shannon said something like that, he was considering
| music to be more along the lines of a MIDI stream than a digitized
| sound file.
|
| Assuming a pianist can make 20 keystroke per second (and there are
| scores that require that), 40 bits is a bit light, since you need to
| refine the timing a bit, and specify the force of the keystroke, as
| well as deal with the releases and the pedals (although those should
| need a relatively low bit rate).
|
| But all that together would certainly be less than 1000 bits per
| second.  That assumes that music is actually random keystrokes at
| random times and random forces, at at very high keystroke rate - which
| it is, of course, not, no more than English sentences are random
| sequences of letters.
|
| Consider that high keystroke rates pretty much require chords, and a
| chord is most certainly not three or four random keys, pressed at
| different times and forces - rather they're a small subset of the
| possible combinations of keys within a hand-span (or two hand-spans in
| some cases), hit at the same time, and with the same force (in fact
| there are less than about 10,000 chords).  And just like English, you
| can't really have random sequences, or you'd just have noise.
|
| Still, 40bps intuitively feels a bit light for some of the busiest
| music, but not by that much.  For more typical passages, it seems
| adequate.

The difference between MIDI and audio compression is the
synthesizer. The same pianist can play 20 notes a second on
a church organ or harpsichord and it will sound bloody awful,
the pipes don't have the same percussive attack that a hammer
on a string has. Same with speech, it is the explosive attack of
the consonant that makes the word, not the drawn out vowel
which is often changed by accent. Orz-dry-lee-uns still write
"Australians" but change the sound of the vowels as we all do.
Much of the spoken word is cliché, "it's like" "know what I
mean".  The first four notes of Beethoven's 5th are a cliché,
it wouldn't matter what instrument they were played on, you'd
still recognise the phrase. Audio compression carries more
information than MIDI, it has to.

[toc] | [prev] | [next] | [standalone]

#6683

On 04/11/2011 09:33, Peter Webb wrote:
>
>
>
> He estimated music contains 40 bits/second entropy.
> How close is MP3 to that?
>
> ___________________________________
> I doubt surprised Shannon said that, and if he did its somewhere between
> meaningless and wrong.
>

If Shannon did make the statement attributed to him above (and I 
seriously doubt it) - I would like to see references.

> CD quality mp3s are roughly equivalent to 178 kbps, over 4,000 times his
> estimate. But then you can encode a lot of sounds that most people would
> not consider music. And it stereo, so you can halve it if Shannon was
> talking about mono.
>
> And how do you define music, except as sound? And random sound waveforms


Have you never seen sheet music? That is what Shannon was estimating - 
the bitrate for describing music in the abstract. There are a finite 
number of notes, durations and amplitudes in a classical composition.

I suspect 40 bits/sec is still far too tight, but a midi stream using a 
high end reconstruction codec represents a pretty good example of what 
is possible by way of compression for *music* as opposed to voice or a 
random noise stream.

> cannot be compressed on average at all. To get a smaller figure for
> music, you have to define what subsets of sounds are music. Lots of luck.

I think that may have been his intention although I don't actually 
recall seeing the 40 bit/s number originally attributed to him above.

As I said if he did anything I think he was estimating the information 
content of music in the already concise form of an orchestral score.
>
> The real number probably lies somewhere between 178 kbps and 40 bps. The
> actual number is the base 2 logarithm of the number of different 1
> second sound bites that the ear can distuingish and would consider as
> music. I doubt even Shannon would have known the answer to that equation.

Maybe he didn't consider all the alternative atonal and continuous 
frequency synthesisers of arbitrary waveforms that are possible now - or 
perhaps it was a subtle dig at Stockhausen and Schoenberg as not music.

Be interesting if the reference to this paper can be found to see how he 
allocated those 40 bits/s....

I reckon at a bare minimum about 7 to the note, 8 to amplitude, 6 
duration, 5 to the instrument - and it is already obvious that you 
cannot encode more than a single note per second at this bitrate.

Can anyone provide a citation to this alleged paper on music bitrate?

-- 
Regards,
Martin Brown

[toc] | [prev] | [next] | [standalone]

#6685

"Martin Brown" <|||newspam|||@nezumi.demon.co.uk> wrote in message 
news:I9Psq.13644$Mg.12196@newsfe13.iad...
> On 04/11/2011 09:33, Peter Webb wrote:
>>
>>
>>
>> He estimated music contains 40 bits/second entropy.
>> How close is MP3 to that?
>>
>> ___________________________________
>> I doubt surprised Shannon said that, and if he did its somewhere between
>> meaningless and wrong.
>>
>
> If Shannon did make the statement attributed to him above (and I seriously 
> doubt it) - I would like to see references.
>
>> CD quality mp3s are roughly equivalent to 178 kbps, over 4,000 times his
>> estimate. But then you can encode a lot of sounds that most people would
>> not consider music. And it stereo, so you can halve it if Shannon was
>> talking about mono.
>>
>> And how do you define music, except as sound? And random sound waveforms
>
>
> Have you never seen sheet music? That is what Shannon was estimating - the 
> bitrate for describing music in the abstract. There are a finite number of 
> notes, durations and amplitudes in a classical composition.
>

Far more plausible.

If you don't consider singing to be music.


> I suspect 40 bits/sec is still far too tight, but a midi stream using a 
> high end reconstruction codec represents a pretty good example of what is 
> possible by way of compression for *music* as opposed to voice or a random 
> noise stream.
>

I wonder what Jimi Hendrix playing Star Spangled Banner would sound like as 
a midi stream?



>> cannot be compressed on average at all. To get a smaller figure for
>> music, you have to define what subsets of sounds are music. Lots of luck.
>
> I think that may have been his intention although I don't actually recall 
> seeing the 40 bit/s number originally attributed to him above.
>
> As I said if he did anything I think he was estimating the information 
> content of music in the already concise form of an orchestral score.

= midi. A very plausible explanation, but one which requires a very limitted 
definition of music.


>>
>> The real number probably lies somewhere between 178 kbps and 40 bps. The
>> actual number is the base 2 logarithm of the number of different 1
>> second sound bites that the ear can distuingish and would consider as
>> music. I doubt even Shannon would have known the answer to that equation.
>
> Maybe he didn't consider all the alternative atonal and continuous 
> frequency synthesisers of arbitrary waveforms that are possible now - or 
> perhaps it was a subtle dig at Stockhausen and Schoenberg as not music.
>

Yeah, I was going to use a synthesiser as an example of an instrument that 
couldn't be compressed, but then I realised that Shannon didn't have to 
worry about digital synthesisers in his day.


> Be interesting if the reference to this paper can be found to see how he 
> allocated those 40 bits/s....
>
> I reckon at a bare minimum about 7 to the note, 8 to amplitude, 6 
> duration, 5 to the instrument - and it is already obvious that you cannot 
> encode more than a single note per second at this bitrate.
>
> Can anyone provide a citation to this alleged paper on music bitrate?
>
> -- 
> Regards,
> Martin Brown 

[toc] | [prev] | [next] | [standalone]

#6705

On 11/4/2011 5:31 AM, Martin Brown wrote:

> Have you never seen sheet music? That is what Shannon was estimating -
> the bitrate for describing music in the abstract. There are a finite
> number of notes, durations and amplitudes in a classical composition.
>
> I suspect 40 bits/sec is still far too tight, but a midi stream using a
> high end reconstruction codec represents a pretty good example of what
> is possible by way of compression for *music* as opposed to voice or a
> random noise stream.

Which is why we are talking about midi. Recording SOUND is a different 
thing. Note that a cymbal crash is very close to white noise. This means 
that exact reproduction allows for little compression. Of course all 
music is not cymbal crashes (or usually so) which is where MP3 comes from.

> Be interesting if the reference to this paper can be found to see how he
> allocated those 40 bits/s....
>
> I reckon at a bare minimum about 7 to the note, 8 to amplitude, 6
> duration, 5 to the instrument - and it is already obvious that you
> cannot encode more than a single note per second at this bitrate.

Sorry NOT "obvious". You need to learn more about information theory!
Note that you have encoded ONE note but looking at sheet music one 
observed that MANY notes are identical!  Hence there is redundancy that 
can be eliminated for compression.

[toc] | [prev] | [next] | [standalone]

#6707

On 11/4/2011 6:31 AM, Martin Brown wrote:
> On 04/11/2011 09:33, Peter Webb wrote:

   ...

> I suspect 40 bits/sec is still far too tight, but a midi stream using a
> high end reconstruction codec represents a pretty good example of what
> is possible by way of compression for *music* as opposed to voice or a
> random noise stream.
>
>> cannot be compressed on average at all. To get a smaller figure for
>> music, you have to define what subsets of sounds are music. Lots of luck.
>
> I think that may have been his intention although I don't actually
> recall seeing the 40 bit/s number originally attributed to him above.
>
> As I said if he did anything I think he was estimating the information
> content of music in the already concise form of an orchestral score.
>>
>> The real number probably lies somewhere between 178 kbps and 40 bps. The
>> actual number is the base 2 logarithm of the number of different 1
>> second sound bites that the ear can distuingish and would consider as
>> music. I doubt even Shannon would have known the answer to that equation.
>
> Maybe he didn't consider all the alternative atonal and continuous
> frequency synthesisers of arbitrary waveforms that are possible now - or
> perhaps it was a subtle dig at Stockhausen and Schoenberg as not music.

There's a case to be made that it really isn't music. Oompah-pah bands, 
Mozart, Bach, Ellington, that characteristic sound of an orchestra 
tuning up, African jungle drums, all have a characteristic fractal 
pattern also found in mountain profiles. Schoenberg, Stockhausen, and 
Webern have patterns that clearly differ. I don't have the reference any 
more. I hope someone can supply it.

   ...

Jerry
-- 
Engineering is the art of making what you want from things you can get.

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.dsp

csiph-web