Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.dsp > #6597 > unrolled thread

information theory?

Back to article view | Back to comp.dsp

Contents

  information theory? RichD <r_delaney2001@yahoo.com> - 2011-10-31 20:55 -0700
    Re: information theory? eric.jacobsen@ieee.org (Eric Jacobsen) - 2011-11-01 04:57 +0000
      Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-01 05:58 -0700
        Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-01 10:29 -0700
          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 11:13 -0700
            Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-01 14:39 -0700
          Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-01 16:04 -0400
            OT: information theory? Jerry Avins <jya@ieee.org> - 2011-11-01 18:06 -0400
            Re: information theory? glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2011-11-01 22:46 +0000
          Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-01 22:38 -0500
            Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-11-02 17:06 -0500
              Re: information theory? "steveu" <steveu@n_o_s_p_a_m.coppice.org> - 2011-11-02 19:41 -0500
              Re: information theory? Andrew Haley <andrew29@littlepinkcloud.invalid> - 2011-11-03 06:58 -0500
                Re: information theory? "GO-HERE .NL" <gdewilde@gmail.com> - 2011-11-03 08:57 -0700
            Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-02 19:44 -0500
              Re: information theory? "Christopher J. Henrich" <chenrich@monmouth.com> - 2011-11-02 22:40 -0400
                Re: information theory? Lofty Goat <rlwatkins@gmail.com> - 2011-11-03 23:21 -0500
          Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 08:47 +0000
            Re: information theory? (copyright question followup) JohnF <john@please.see.sig.for.email.com> - 2011-11-02 09:48 +0000
              Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 10:02 +0000
                Re: information theory? (copyright question followup) Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-02 16:46 +0000
                Re: information theory? (copyright question followup) Bernd Jendrissek <bernd.jendrissek@gmail.com> - 2011-11-02 15:04 -0700
              Re: information theory? (copyright question followup) David Eather <eather@tpg.com.au> - 2011-11-03 04:00 +1000
                Re: information theory? (copyright question followup) unruh <unruh@physics.ubc.ca> - 2011-11-02 19:05 +0000
              Re: information theory? (copyright question followup) "J. Clarke" <jclarkeusenet@cox.net> - 2011-11-24 08:42 -0500
                Re: information theory? (copyright question followup) Richard Outerbridge <outer@interlog.com> - 2011-11-25 01:37 -0500
                  Re: information theory? (copyright question followup) Les Cargill <lcargill99@comcast.com> - 2011-11-25 11:37 -0600
            Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:38 -0700
              Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-03 08:51 +0000
          Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-02 08:02 -0700
            Re: information theory? robert bristow-johnson <rbj@audioimagination.com> - 2011-11-02 09:38 -0700
            Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-02 11:51 -0700
              Re: information theory? VWWall <vwall@large.invalid> - 2011-11-02 15:08 -0700
                Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-11-02 19:59 -0400
                  Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-04 20:41 +0000
                    Re: information theory? "krw@att.bizzzzzzzzzzzz" <krw@att.bizzzzzzzzzzzz> - 2011-11-04 16:59 -0500
                      Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 15:28 -0700
                      Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 21:09 -0400
                        Re: information theory? ggr@nope.ucsd.edu (Greg Rose) - 2011-11-05 02:11 +0000
                        Re: information theory? Frederick Williams <freddywilliams@btinternet.com> - 2011-11-05 11:53 +0000
                      Re: information theory? "Mike Terry" <news.dead.person.stones@darjeeling.plus.com> - 2011-11-05 17:32 +0000
                  Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 18:41 -0800
                Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-11-03 21:54 -0700
                  Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 03:42 -0500
                  Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 20:33 +1100
                    Re: information theory? Robert Wessel <robertwessel2@yahoo.com> - 2011-11-04 05:10 -0500
                      Re: information theory? kym@kymhorsell.com - 2011-11-04 10:50 +0000
                        Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 22:14 +1100
                          Re: information theory? "Jesse F. Hughes" <jesse@phiwumbda.org> - 2011-11-04 09:13 -0400
                            Re: information theory? Spehro Pefhany <speffSNIP@interlogDOTyou.knowwhat> - 2011-11-04 10:32 -0400
                              Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 18:02 +0000
                                Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 15:54 -0800
                        Re: information theory? VWWall <vwall@large.invalid> - 2011-11-04 09:50 -0700
                          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 12:41 -0700
                          Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-04 13:40 -0700
                      Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:35 +0000
                    Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-11-04 10:31 +0000
                      Re: information theory? "Peter Webb" <webbfamily@optusnetDIESPAMDIE.com.au> - 2011-11-04 21:51 +1100
                      Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-04 14:18 -0500
                      Re: information theory? Jerry Avins <jya@ieee.org> - 2011-11-04 15:56 -0400
                      Re: information theory? RichD <r_delaney2001@yahoo.com> - 2011-12-21 17:49 -0800
                        Re: information theory? Les Cargill <lcargill99@comcast.com> - 2011-12-21 20:07 -0600
                        Re: information theory? Richard Outerbridge <outer@interlog.com> - 2011-12-22 03:08 -0500
                          Re: information theory? Martin Brown <|||newspam|||@nezumi.demon.co.uk> - 2011-12-22 08:30 +0000
                            Re: information theory? John Devereux <john@devereux.me.uk> - 2011-12-22 10:12 +0000
                            Re: information theory? "Peter Webb" <r.peter.webbbbb@gmail.com> - 2011-12-22 21:14 +1100
                    Re: information theory? "Androcles" <Headmaster@Hogwarts.physics.October.2011> - 2011-11-04 11:00 +0000
                  Re: information theory? Casper H.S. Dik <Casper.Dik@OrSPaMcle.COM> - 2011-11-04 09:45 +0000
                    Re: information theory? unruh <unruh@physics.ubc.ca> - 2011-11-04 19:05 +0000
                Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-06 17:11 -0800
                  Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 06:23 -0800
                    Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-07 11:29 -0800
                    Re: information theory? jim <retnuh2011@gmail.com> - 2011-11-08 04:41 -0800
                      Re: information theory? "Michael A. Terrell" <mike.terrell@earthlink.net> - 2011-11-08 13:46 -0500
              Re: information theory? "BJACOBY@teranews.com" <benj@iwaynet.net> - 2011-11-03 03:32 -0500
              Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:39 -0700
              Re: information theory? Chris <chris.santoro@gmail.com> - 2011-11-03 08:28 -0700

Page 1 of 4  [1] 2 3 4  Next page →

#6597 — information theory?

Are there any boards, or private blogs, dedicated
to discussion of information theory?

Rich

[toc] | [next] | [standalone]

#6598

On Mon, 31 Oct 2011 20:55:57 -0700 (PDT), RichD
<r_delaney2001@yahoo.com> wrote:

>Are there any boards, or private blogs, dedicated
>to discussion of information theory?
>
>Rich

Depending on the question comp.dsp is not a bad place.   There are a
fair number of comm people there.


Eric Jacobsen
Anchor Hill Communications
www.anchorhill.com

[toc] | [prev] | [next] | [standalone]

#6601

Do you have a specific question? Or just want to lurk and learn?

Information Theory was the hardest class I took in grad school. So
abstract, and so much probability.... but it's a cool topic
nonetheless.

On Nov 1, 12:57 am, eric.jacob...@ieee.org (Eric Jacobsen) wrote:
> On Mon, 31 Oct 2011 20:55:57 -0700 (PDT), RichD
>
> <r_delaney2...@yahoo.com> wrote:
> >Are there any boards, or private blogs, dedicated
> >to discussion of information theory?
>
> >Rich
>
> Depending on the question comp.dsp is not a bad place.   There are a
> fair number of comm people there.
>
> Eric Jacobsen
> Anchor Hill Communicationswww.anchorhill.com

[toc] | [prev] | [next] | [standalone]

#6608

On Nov 1, Chris <chris.sant...@gmail.com> wrote:
> Do you have a specific question? Or just want to lurk and learn?

I'd like to lurk someplace, where there are discussions
of current topics.  It's such a rich field, there are always
new ideas.

> Information Theory was the hardest class I took in grad school. So
> abstract, and so much probability.... but it's a cool topic
> nonetheless.

I read Shannon's paper long go, and found it fascinating
ever since.  In the most general sense, it's applicable
to so many areas.  For instance, people have recast
thermodynamics as information processing.  Or, you
could model the entire universe as a computer, and
information theory applies.  It even describes human
nature - manipulating symbols is what separates us
from the apes.

> > >Are there any boards, or private blogs, dedicated
> > >to discussion of information theory?

--
Rich

[toc] | [prev] | [next] | [standalone]

#6612

On Nov 1, 1:29 pm, RichD <r_delaney2...@yahoo.com> wrote:
> On Nov 1, Chris <chris.sant...@gmail.com> wrote:
>
> > Do you have a specific question? Or just want to lurk and learn?
>
> I'd like to lurk someplace, where there are discussions
> of current topics.  It's such a rich field, there are always
> new ideas.
>
> > Information Theory was the hardest class I took in grad school. So
> > abstract, and so much probability.... but it's a cool topic
> > nonetheless.
>
> I read Shannon's paper long go, and found it fascinating
> ever since.  In the most general sense, it's applicable
> to so many areas.  For instance, people have recast
> thermodynamics as information processing.  Or, you
> could model the entire universe as a computer, and
> information theory applies.

   Well, you can. But it's still going to take the people
   working in thermodynamics another several hundred
   years to discover that when base all your science
   on Classical Geometry, you will always come
   to that conclusion, regardless of the technology.

   Which still doesn't imply that the people who
   understand number theory aren't going to be
   working on post pyramid compilers, rather than
   more sandstone libraries for them.

   The people who work in engineering will be
   working on lasers, holographics, nanotechnolgy, and self-
replicating machines.
   Regardless of what the jerks do with flourescent lightbulbs.

   And the people who working in signal processing will be working
   on 21st Helicopters, regardless of what any of the clowns do with
radar.


 It even describes human
> nature - manipulating symbols is what separates us
> from the apes.
>
> > > >Are there any boards, or private blogs, dedicated
> > > >to discussion of information theory?
>
> --
> Rich

[toc] | [prev] | [next] | [standalone]

#6614

On Nov 1, 2:13 pm, jim <retnuh2...@gmail.com> wrote:
> On Nov 1, 1:29 pm, RichD <r_delaney2...@yahoo.com> wrote:
>
>
>
>
>
>
>
>
>
> > On Nov 1, Chris <chris.sant...@gmail.com> wrote:
>
> > > Do you have a specific question? Or just want to lurk and learn?
>
> > I'd like to lurk someplace, where there are discussions
> > of current topics.  It's such a rich field, there are always
> > new ideas.
>
> > > Information Theory was the hardest class I took in grad school. So
> > > abstract, and so much probability.... but it's a cool topic
> > > nonetheless.
>
> > I read Shannon's paper long go, and found it fascinating
> > ever since.  In the most general sense, it's applicable
> > to so many areas.  For instance, people have recast
> > thermodynamics as information processing.  Or, you
> > could model the entire universe as a computer, and
> > information theory applies.
>
>    Well, you can. But it's still going to take the people
>    working in thermodynamics another several hundred
>    years to discover that when base all your science
>    on Classical Geometry, you will always come
>    to that conclusion, regardless of the technology.

    Which is mostly a way of telling the loons:

    If you could get the moon on classical geometry,
    they wouldn't be paying real engineers, several millions dollars a
year,
    and paying people working in thermodynamics, minimum wage for
    waxing Edsels for Dow Chemical.



>
>    Which still doesn't imply that the people who
>    understand number theory aren't going to be
>    working on post pyramid compilers, rather than
>    more sandstone libraries for them.
>
>    The people who work in engineering will be
>    working on lasers, holographics, nanotechnolgy, and self-
> replicating machines.
>    Regardless of what the jerks do with flourescent lightbulbs.
>
>    And the people who working in signal processing will be working
>    on 21st Helicopters, regardless of what any of the clowns do with
> radar.
>
>  It even describes human
>
>
>
>
>
>
>
> > nature - manipulating symbols is what separates us
> > from the apes.
>
> > > > >Are there any boards, or private blogs, dedicated
> > > > >to discussion of information theory?
>
> > --
> > Rich

[toc] | [prev] | [next] | [standalone]

#6613

On 11/1/11 1:29 PM, RichD wrote:
> On Nov 1, Chris<chris.sant...@gmail.com>  wrote:
>> Do you have a specific question? Or just want to lurk and learn?
>
> I'd like to lurk someplace, where there are discussions
> of current topics.  It's such a rich field, there are always
> new ideas.
>
>> Information Theory was the hardest class I took in grad school. So
>> abstract, and so much probability.... but it's a cool topic
>> nonetheless.
>
> I read Shannon's paper long go, and found it fascinating
> ever since.  In the most general sense, it's applicable
> to so many areas.  For instance, people have recast
> thermodynamics as information processing.

even though i have a *feel* for the connection to thermodynamic entropy, 
i really do not understand what the mathematical connection is between

     SUM{ p{M_i} * I{M_i} }
      i

where M_i = the i_th message
p{M_i} is the probability of the i_th message
I{M_i} is the information content of the i_th message and is equal to

     -log( p{M_i} )

(the base of the log only changes the units that information is measured 
with.  log2() means information measured in "bits".)

now how is that above summation related to thermodynamic entropy which is

     integral{ 1/T dQ}

i wouldn't mind seeing the connection here.  perhaps glen or Clay (who 
are the other physikers hanging here?) might have an answer.


>  Or, you
> could model the entire universe as a computer, and
> information theory applies.  It even describes human
> nature - manipulating symbols is what separates us
> from the apes.

oh, i dunno.  that sounds like the kinda stuff we talk about as we sit 
around picking our noses.  apes don't manipulate signals?


-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

[toc] | [prev] | [next] | [standalone]

#6615 — OT: information theory?

On 11/1/2011 4:04 PM, robert bristow-johnson wrote:

   ...

> oh, i dunno. that sounds like the kinda stuff we talk about as we sit
> around picking our noses. apes don't manipulate signals?

Even my dog manipulates symbols, sometimes in very creative ways. There 
was an orangutan who regularly unlocked his cage at night, letting 
himself and his family into the savanna-like yard outside it. He used a 
piece of wire as a lock pick, and rebent it to fit behind his lip when 
he didn't need to use it. It took the keepers a long time to figure it 
all out. Can we in good conscience lock up creatures like that?

Jerry
-- 
Engineering is the art of making what you want from things you can get.

[toc] | [prev] | [next] | [standalone]

#6616

robert bristow-johnson <rbj@audioimagination.com> wrote:
(snip)

> even though i have a *feel* for the connection to thermodynamic entropy, 
> i really do not understand what the mathematical connection is between

>     SUM{ p{M_i} * I{M_i} }
>      i

> where M_i = the i_th message
> p{M_i} is the probability of the i_th message
> I{M_i} is the information content of the i_th message and is equal to

>     -log( p{M_i} )

> (the base of the log only changes the units that information is measured 
> with.  log2() means information measured in "bits".)

> now how is that above summation related to thermodynamic entropy which is

>     integral{ 1/T dQ}

> i wouldn't mind seeing the connection here.  perhaps glen or 
> Clay (who are the other physikers hanging here?) might have 
> an answer.

I am not sure I can explain it so well, either.  

Thermodynamic entropy is related to the number of states a
system can be in, which  I believe can trace back to the above sum.

If you take a container with a partition down the middle, put
one gas in one side, and a different on in the other side, then
consider if you select a molecule at random, what the probability
of it being gas one or gas two.  For each side, there is 100%
chance of it being the appropriate type.  Now remove the partition
and let them start mixing.  The entropy increases as they mix, and
the probability eventually approaches 50% each.  

In the case of individual molecules, it is a sum.  With enough,
you approximate it as an integral.

-- glen

[toc] | [prev] | [next] | [standalone]

#6617

On 11/1/2011 12:29 PM, RichD wrote:

> I read Shannon's paper long go, and found it fascinating
> ever since.  In the most general sense, it's applicable
> to so many areas.  For instance, people have recast
> thermodynamics as information processing.  Or, you
> could model the entire universe as a computer, and
> information theory applies.  It even describes human
> nature - manipulating symbols is what separates us
> from the apes.

Actually, Shannon's "paper" is a book. And the unfortunate part is that 
physicists in particular for the most part have developed little 
understanding of Shannon's association of information with entropy. This 
one fact alone blows a huge hole in classical physics. Yet, (nearly) all 
physicists go blindly on barfing the sad dogma of how entropy always 
increases, how the universe is "running down" etc. etc. They know even 
less about the relationships between signal processing and 
Electromagnetic theory. They can't understand the transform 
relationships between the amplitude of a wave in free space at one point 
and it's far field angular expansion. They are snowed by self-inductance 
because they haven't bothered to understand the nature of feedback 
systems. In short they are stuck in ignorance and the past and have NO 
desire to move out of it.

But the unfortunate facts are that information theory especially 
combined with things like classical thermodynamics holds the key to 
physics in the 21st century! And what exactly IS physics going to be in 
the 21st century? I can tell you. It will be the physics of the things 
that physics of today has so LOUDLY avoided. Namely, the physics of 
life! The grass under my brick sidewalk is ALWAYS striving to take the 
chaos of the dirt down there and turn it into ORDER! It insists on doing 
this every year in spite of my best efforts to stop it! But still 
"traditional" science pretends to move "forward" while ignoring the fact 
that Shannon shot all their dogma to hell back in the MIDDLE of the the 
20th century.

[toc] | [prev] | [next] | [standalone]

#6644

BJACOBY@teranews.com wrote:
> On 11/1/2011 12:29 PM, RichD wrote:
>
>> I read Shannon's paper long go, and found it fascinating
>> ever since. In the most general sense, it's applicable
>> to so many areas. For instance, people have recast
>> thermodynamics as information processing. Or, you
>> could model the entire universe as a computer, and
>> information theory applies. It even describes human
>> nature - manipulating symbols is what separates us
>> from the apes.
>
> Actually, Shannon's "paper" is a book. And the unfortunate part is that
> physicists in particular for the most part have developed little
> understanding of Shannon's association of information with entropy. This
> one fact alone blows a huge hole in classical physics. Yet, (nearly) all
> physicists go blindly on barfing the sad dogma of how entropy always
> increases, how the universe is "running down" etc. etc. They know even
> less about the relationships between signal processing and
> Electromagnetic theory. They can't understand the transform
> relationships between the amplitude of a wave in free space at one point
> and it's far field angular expansion. They are snowed by self-inductance
> because they haven't bothered to understand the nature of feedback
> systems. In short they are stuck in ignorance and the past and have NO
> desire to move out of it.
>
> But the unfortunate facts are that information theory especially
> combined with things like classical thermodynamics holds the key to
> physics in the 21st century! And what exactly IS physics going to be in
> the 21st century? I can tell you. It will be the physics of the things
> that physics of today has so LOUDLY avoided. Namely, the physics of
> life! The grass under my brick sidewalk is ALWAYS striving to take the
> chaos of the dirt down there and turn it into ORDER! It insists on doing
> this every year in spite of my best efforts to stop it! But still
> "traditional" science pretends to move "forward" while ignoring the fact
> that Shannon shot all their dogma to hell back in the MIDDLE of the the
> 20th century.
>


Eh? Physics guys loudly accept that entropy can go up locally. They
just  say universal ( meaning at the scale of the universe ) entropy
is what goes up. Sagan at one point assigned values to the relative 
entropy/order increases from reading something, for example.

Penrose/Hawking and everybody else since have used
communications theory quite a bit to do physics. I
don't even think it's settled yet.

Penroses' "Emperor's New Mind" misses quite a bit ( he
skyhooks some stuff ) but that alone should show they're
thinking about it.

If physics has a problem, it is that the experiments have all gotten
too expensive for a 1920s-1950s level of progress to occur. That, and
string  theory becoming too precious and abstract.

--
Les Cargill

[toc] | [prev] | [next] | [standalone]

#6652

>BJACOBY@teranews.com wrote:
>> On 11/1/2011 12:29 PM, RichD wrote:
>>
>>> I read Shannon's paper long go, and found it fascinating
>>> ever since. In the most general sense, it's applicable
>>> to so many areas. For instance, people have recast
>>> thermodynamics as information processing. Or, you
>>> could model the entire universe as a computer, and
>>> information theory applies. It even describes human
>>> nature - manipulating symbols is what separates us
>>> from the apes.
>>
>> Actually, Shannon's "paper" is a book. And the unfortunate part is that
>> physicists in particular for the most part have developed little
>> understanding of Shannon's association of information with entropy.
This
>> one fact alone blows a huge hole in classical physics. Yet, (nearly)
all
>> physicists go blindly on barfing the sad dogma of how entropy always
>> increases, how the universe is "running down" etc. etc. They know even
>> less about the relationships between signal processing and
>> Electromagnetic theory. They can't understand the transform
>> relationships between the amplitude of a wave in free space at one
point
>> and it's far field angular expansion. They are snowed by
self-inductance
>> because they haven't bothered to understand the nature of feedback
>> systems. In short they are stuck in ignorance and the past and have NO
>> desire to move out of it.
>>
>> But the unfortunate facts are that information theory especially
>> combined with things like classical thermodynamics holds the key to
>> physics in the 21st century! And what exactly IS physics going to be in
>> the 21st century? I can tell you. It will be the physics of the things
>> that physics of today has so LOUDLY avoided. Namely, the physics of
>> life! The grass under my brick sidewalk is ALWAYS striving to take the
>> chaos of the dirt down there and turn it into ORDER! It insists on
doing
>> this every year in spite of my best efforts to stop it! But still
>> "traditional" science pretends to move "forward" while ignoring the
fact
>> that Shannon shot all their dogma to hell back in the MIDDLE of the the
>> 20th century.
>>
>
>
>Eh? Physics guys loudly accept that entropy can go up locally. They
>just  say universal ( meaning at the scale of the universe ) entropy
>is what goes up. Sagan at one point assigned values to the relative 
>entropy/order increases from reading something, for example.
>
>Penrose/Hawking and everybody else since have used
>communications theory quite a bit to do physics. I
>don't even think it's settled yet.
>
>Penroses' "Emperor's New Mind" misses quite a bit ( he
>skyhooks some stuff ) but that alone should show they're
>thinking about it.
>
>If physics has a problem, it is that the experiments have all gotten
>too expensive for a 1920s-1950s level of progress to occur. That, and
>string  theory becoming too precious and abstract.

In basic high school physics and chemistry, when you first learn about
entropy, you learn about it rising and falling through various processes.
Its there where you learn the, perhaps, surprising result that the *net*
entropy along any chain of events always increases. Why would even someone
limited to high school science find anything anomalous about localised
entropy in information theory?

Steve

[toc] | [prev] | [next] | [standalone]

#6658

In sci.crypt Les Cargill <lcargill99@comcast.com> wrote:
> 
> Penroses' "Emperor's New Mind" misses quite a bit ( he skyhooks some
> stuff ) but that alone should show they're thinking about it.

_The Road to Reality_ goes into a lot of detail.

Andrew.

[toc] | [prev] | [next] | [standalone]

#6665

On Nov 3, 12:58 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:
> In sci.crypt Les Cargill <lcargil...@comcast.com> wrote:
>
>
>
> > Penroses' "Emperor's New Mind" misses quite a bit ( he skyhooks some
> > stuff ) but that alone should show they're thinking about it.
>
> _The Road to Reality_ goes into a lot of detail.
>
> Andrew.

The truth will set you free, but first it will make you look stupid.

- gaby de wilde

[toc] | [prev] | [next] | [standalone]

#6653

On Tue, 01 Nov 2011 22:38:03 -0500, BJACOBY@teranews.com wrote:

> On 11/1/2011 12:29 PM, RichD wrote:
> 
>> I read Shannon's paper long go, and found it fascinating ever since. 
>> In the most general sense, it's applicable to so many areas.  For
>> instance, people have recast thermodynamics as information processing. 
>> Or, you could model the entire universe as a computer, and information
>> theory applies.  It even describes human nature - manipulating symbols
>> is what separates us from the apes.
> 
> Actually, Shannon's "paper" is a book. And the unfortunate part is that
> physicists in particular for the most part have developed little
> understanding of Shannon's association of information with entropy. This
> one fact alone blows a huge hole in classical physics. Yet, (nearly) all
> physicists go blindly on barfing the sad dogma of how entropy always
> increases, how the universe is "running down" etc. etc. They know even
> less about the relationships between signal processing and
> Electromagnetic theory. They can't understand the transform
> relationships between the amplitude of a wave in free space at one point
> and it's far field angular expansion. They are snowed by self-inductance
> because they haven't bothered to understand the nature of feedback
> systems. In short they are stuck in ignorance and the past and have NO
> desire to move out of it.
> 
> But the unfortunate facts are that information theory especially
> combined with things like classical thermodynamics holds the key to
> physics in the 21st century! And what exactly IS physics going to be in
> the 21st century? I can tell you. It will be the physics of the things
> that physics of today has so LOUDLY avoided. Namely, the physics of
> life! The grass under my brick sidewalk is ALWAYS striving to take the
> chaos of the dirt down there and turn it into ORDER! It insists on doing
> this every year in spite of my best efforts to stop it! But still
> "traditional" science pretends to move "forward" while ignoring the fact
> that Shannon shot all their dogma to hell back in the MIDDLE of the the
> 20th century.

This almost makes sense... at least at first.

But the earliest I can remember someone doing otherwise, in earnest, is 
1977, with the publication of Kantor's "Information Mechanics".  There 
are probably earlier examples.

-- 
RLW

[toc] | [prev] | [next] | [standalone]

#6654

In article <DP-dnaStmoSaeSzTnZ2dnUVZ5oadnZ2d@giganews.com>, Lofty Goat
<rlwatkins@gmail.com> wrote:

> 
> But the earliest I can remember someone doing otherwise, in earnest, is 
> 1977, with the publication of Kantor's "Information Mechanics".  There 
> are probably earlier examples.
I am delighted to learn that somebody besides me knows about that book.
What do you think of it?

-- 
Chris Henrich
http://www.mathinteract.com
God just doesn't fit inside a single religion.

[toc] | [prev] | [next] | [standalone]

#6674

On Wed, 02 Nov 2011 22:40:13 -0400, Christopher J. Henrich wrote:

> In article <DP-dnaStmoSaeSzTnZ2dnUVZ5oadnZ2d@giganews.com>, Lofty Goat
> <rlwatkins@gmail.com> wrote:
> 
> 
>> But the earliest I can remember someone doing otherwise, in earnest, is
>> 1977, with the publication of Kantor's "Information Mechanics".  There
>> are probably earlier examples.

> I am delighted to learn that somebody besides me knows about that book.
> What do you think of it?

While hesitant to offer an opinion after thirty years, what it said, 
indirectly, about the limits and potential of computational machinery 
seemed most interesting, something which seems now to be coming full 
circle, as well as what seemed a pretty novel approach, at the time, to 
reasoning about the World Of Quanta.

And I'll confess that I've been away from physics so long that if I were 
to re-read it I'd probably not understand but a fraction.  You'll find 
others hereabouts I'm sure whose judgments will be more useful.

-- 
RLW

[toc] | [prev] | [next] | [standalone]

#6620

On 01/11/2011 17:29, RichD wrote:
> On Nov 1, Chris<chris.sant...@gmail.com>  wrote:
>> Do you have a specific question? Or just want to lurk and learn?
>
> I'd like to lurk someplace, where there are discussions
> of current topics.  It's such a rich field, there are always
> new ideas.
>
>> Information Theory was the hardest class I took in grad school. So
>> abstract, and so much probability.... but it's a cool topic
>> nonetheless.
>
> I read Shannon's paper long go, and found it fascinating
> ever since.  In the most general sense, it's applicable
> to so many areas.  For instance, people have recast
> thermodynamics as information processing.  Or, you
> could model the entire universe as a computer, and
> information theory applies.  It even describes human
> nature - manipulating symbols is what separates us
> from the apes.
>
>>>> Are there any boards, or private blogs, dedicated
>>>> to discussion of information theory?

You don't really say which part of information theory you are interested 
in - but some of the more interesting modern developments stem from 
Bayes theorem and the principle of maximum entropy.

You could do worse than look for the collected works of Ed Jaynes - it 
used to be available online at one time but now it is published.

http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133

David MacKays book Information Theory, Inference and Learning might be a 
cheaper alternative.

Not an easy read and assumes higher mathematics as a starting point.

-- 
Regards,
Martin Brown

[toc] | [prev] | [next] | [standalone]

#6621 — Re: information theory? (copyright question followup)

In sci.math Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
> [[snip]]
> You could do worse than look for the collected works of Ed Jaynes - it 
> used to be available online at one time but now it is published.
> 
> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133

Just curious -- if it was freely available online at one time,
and I assume legally at that time, how does later publishing
a paper copy retroactively affect online legality? I'd have
assumed any originally-legal online info remains legal regardless
of who else prints it on paper. Is that wrong?
-- 
John Forkosh  ( mailto:  j@f.com  where j=john and f=forkosh )

[toc] | [prev] | [next] | [standalone]

#6622 — Re: information theory? (copyright question followup)

On 02/11/2011 09:48, JohnF wrote:
> In sci.math Martin Brown<|||newspam|||@nezumi.demon.co.uk>  wrote:
>> [[snip]]
>> You could do worse than look for the collected works of Ed Jaynes - it
>> used to be available online at one time but now it is published.
>>
>> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133
>
> Just curious -- if it was freely available online at one time,
> and I assume legally at that time, how does later publishing
> a paper copy retroactively affect online legality? I'd have
> assumed any originally-legal online info remains legal regardless
> of who else prints it on paper. Is that wrong?

I don't know. ISTR it was online as a part of a global proofing effort. 
When that was finished it looks like the online copy was removed.

(at least if it is still there I can't see it now)

It was at: http://bayes.wustl.edu/etj/prob.html

Still linked to from Bayesian resources at Cornell:

http://www.astro.cornell.edu/staff/loredo/bayes/


-- 
Regards,
Martin Brown

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | comp.dsp

csiph-web