Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
| From | robert bristow-johnson <rbj@audioimagination.com> |
|---|---|
| Newsgroups | comp.dsp |
| Subject | Re: information theory? |
| Date | 2011-11-01 16:04 -0400 |
| Organization | A noiseless patient Spider |
| Message-ID | <j8pjcs$4fe$1@dont-email.me> (permalink) |
| References | <7f70f946-bf5b-4546-a232-5b7353a542cd@u37g2000prh.googlegroups.com> <4eaf7bf5.547777542@www.eternal-september.org> <5b93b1ba-5a4b-48eb-ba30-1ff212fd7839@f36g2000vbm.googlegroups.com> <9cd896f5-11aa-4600-82c6-a1cb0966f956@j36g2000prh.googlegroups.com> |
On 11/1/11 1:29 PM, RichD wrote:
> On Nov 1, Chris<chris.sant...@gmail.com> wrote:
>> Do you have a specific question? Or just want to lurk and learn?
>
> I'd like to lurk someplace, where there are discussions
> of current topics. It's such a rich field, there are always
> new ideas.
>
>> Information Theory was the hardest class I took in grad school. So
>> abstract, and so much probability.... but it's a cool topic
>> nonetheless.
>
> I read Shannon's paper long go, and found it fascinating
> ever since. In the most general sense, it's applicable
> to so many areas. For instance, people have recast
> thermodynamics as information processing.
even though i have a *feel* for the connection to thermodynamic entropy,
i really do not understand what the mathematical connection is between
SUM{ p{M_i} * I{M_i} }
i
where M_i = the i_th message
p{M_i} is the probability of the i_th message
I{M_i} is the information content of the i_th message and is equal to
-log( p{M_i} )
(the base of the log only changes the units that information is measured
with. log2() means information measured in "bits".)
now how is that above summation related to thermodynamic entropy which is
integral{ 1/T dQ}
i wouldn't mind seeing the connection here. perhaps glen or Clay (who
are the other physikers hanging here?) might have an answer.
> Or, you
> could model the entire universe as a computer, and
> information theory applies. It even describes human
> nature - manipulating symbols is what separates us
> from the apes.
oh, i dunno. that sounds like the kinda stuff we talk about as we sit
around picking our noses. apes don't manipulate signals?
--
r b-j rbj@audioimagination.com
"Imagination is more important than knowledge."
Back to comp.dsp | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
information theory? RichD <r_delaney2001@yahoo.com> - 2011-10-31 20:55 -0700
Re: information theory? eric.jacobsen@ieee.org (Eric Jacobsen) - 2011-11-01 04:57 +0000
Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-01 05:58 -0700
Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-01 10:29 -0700
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 11:13 -0700
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 14:39 -0700
Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-01 16:04 -0400
OT: information theory? Jerry Avins <jya@ieee.org> - 2011-11-01 18:06 -0400
Re: information theory? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-11-01 22:46 +0000
Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-01 22:38 -0500
Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-11-02 17:06 -0500
Re: information theory? "steveu" <steveu@n_o_s_p_a_m.coppice.org> - 2011-11-02 19:41 -0500
Re: information theory? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-11-03 06:58 -0500
Re: information theory? "GO-HERE .NL" <gdewilde@gmail.com> - 2011-11-03 08:57 -0700
Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-02 19:44 -0500
Re: information theory? "Christopher J. Henrich" <chenrich@monmouth.com> - 2011-11-02 22:40 -0400
Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-03 23:21 -0500
Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 08:47 +0000
Re: information theory? (copyright question followup) JohnF <john@please.see.sig.for.email.com> - 2011-11-02 09:48 +0000
Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 10:02 +0000
Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 16:46 +0000
Re: information theory? (copyright question followup) Bernd Jendrissek <bernd.jendrissek@gmail.com> - 2011-11-02 15:04 -0700
Re: information theory? (copyright question followup) David Eather <eather@tpg.com.au> - 2011-11-03 04:00 +1000
Re: information theory? (copyright question followup) unruh <unruh@physics.ubc.ca> - 2011-11-02 19:05 +0000
Re: information theory? (copyright question followup) "J. Clarke" <jclarkeusenet@cox.net> - 2011-11-24 08:42 -0500
Re: information theory? (copyright question followup) Richard Outerbridge <outer@interlog.com> - 2011-11-25 01:37 -0500
Re: information theory? (copyright question followup) Les Cargill <lcargill99@comcast.com> - 2011-11-25 11:37 -0600
Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:38 -0700
Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-03 08:51 +0000
Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-02 08:02 -0700
Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-02 09:38 -0700
Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:51 -0700
Re: information theory? VWWall <vwall@large.invalid> - 2011-11-02 15:08 -0700
Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-11-02 19:59 -0400
Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-04 20:41 +0000
Re: information theory? "krw@att.bizzzzzzzzzzzz" <krw@att.bizzzzzzzzzzzz> - 2011-11-04 16:59 -0500
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 15:28 -0700
Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 21:09 -0400
Re: information theory? ggr@nope.ucsd.edu (Greg Rose) - 2011-11-05 02:11 +0000
Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-05 11:53 +0000
Re: information theory? "Mike Terry" <news.dead.person.stones@darjeeling.plus.com> - 2011-11-05 17:32 +0000
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 18:41 -0800
Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-03 21:54 -0700
Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 03:42 -0500
Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 20:33 +1100
Re: information theory? Robert Wessel <robertwessel2@yahoo.com> - 2011-11-04 05:10 -0500
Re: information theory? kym@kymhorsell.com - 2011-11-04 10:50 +0000
Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 22:14 +1100
Re: information theory? "Jesse F. Hughes" <jesse@phiwumbda.org> - 2011-11-04 09:13 -0400
Re: information theory? Spehro Pefhany <speffSNIP@interlogDOTyou.knowwhat> - 2011-11-04 10:32 -0400
Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 18:02 +0000
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 15:54 -0800
Re: information theory? VWWall <vwall@large.invalid> - 2011-11-04 09:50 -0700
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 12:41 -0700
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 13:40 -0700
Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:35 +0000
Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-04 10:31 +0000
Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 21:51 +1100
Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 14:18 -0500
Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 15:56 -0400
Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-12-21 17:49 -0800
Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-12-21 20:07 -0600
Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-12-22 03:08 -0500
Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-12-22 08:30 +0000
Re: information theory? John Devereux <john@devereux.me.uk> - 2011-12-22 10:12 +0000
Re: information theory? "Peter Webb" <r.peter.webbbbb@gmail.com> - 2011-12-22 21:14 +1100
Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:00 +0000
Re: information theory? Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2011-11-04 09:45 +0000
Re: information theory? unruh <unruh@physics.ubc.ca> - 2011-11-04 19:05 +0000
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 17:11 -0800
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 06:23 -0800
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 11:29 -0800
Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-08 04:41 -0800
Re: information theory? "Michael A. Terrell" <mike.terrell@earthlink.net> - 2011-11-08 13:46 -0500
Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-03 03:32 -0500
Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:39 -0700
Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:28 -0700
csiph-web