Re: IBM demos cognitive computer chips

From	BGB <cr88192@hotmail.com>
Newsgroups	comp.arch
Subject	Re: IBM demos cognitive computer chips
Date	2011-08-20 00:15 -0700
Organization	albasani.net
Message-ID	<j2nn8o$9m2$1@news.albasani.net> (permalink)
References	<ggtgp-4F4320.18023918082011@netnews.mchsi.com> <agi3q.116877$eG6.66461@newsfe09.iad> <oex3q.19896$iK1.12792@newsfe05.iad> <j2mbn8$706$1@news.albasani.net> <68z3q.135743$8G4.70421@newsfe17.iad>

Show all headers | View raw

On 8/19/2011 12:50 PM, Robert Myers wrote:
> On 8/19/2011 2:52 PM, BGB wrote:
>
>>
>> a problem I think is the lack of much in-depth information as to what
>> exactly neurons actually do, and to some extent, what is done by a lot
>> of electronic neural nets sounds mostly like glorified vector/matrix
>> multiplies combined with a feedback-driven adaptation mechanism or
>> similar.
>>
>
> You said a mouthful.
>

well, mostly just stating it how it comes off (from what things I have 
read, ...).

if one has N axons, and N dendrites, with a 1:1 mapping, and the output 
is the sum of each axon multiplied by each weight, essentially this is a 
dot-product.

doing this for N outputs, essentially we have an N input vector being 
transformed into an N output vector via N dot products each with N 
multiplies.

essentially, this is a vector*matrix multiply.

looking on Wikipedia: apparently this is about right, except that each 
output is also generally scaled by 1/N run through tanh (appears to 
serve mostly as a range clamp function).

then presumably this is followed by updating the weights, say, for each:
relation[j][i]=axon[i]*output[j];
f=weight[j][i]*(1.0-rate)+relation[j][i]*rate;
weight[j][i]=clamp(f, -1.0, 1.0); //maybe tanh here as well?
then possibly scaling weight[j] by 1.0/(sigma(N,i=0)weight[j][i])
(above just a naive guess, I don't know what they are actually using).

granted, there is probably a lot more to it than this though, but I 
haven't really looked much into it.

>>
>> granted, I also dislike how much some people are prone to
>> trivialize/demean the Von Neumann architecture, as it is well proven at,
>> actually working, and being generally useful... (vs most of its "better"
>> alternatives).
>>
>
> Actually, I belong to the camp that thinks that the Von Neumann
> architecture has been worked to the boundaries of its inherent
> limitations, and well beyond, actually.
>

possibly, but the architecture is also well proven (it has worked well 
for a long time), and switching to a fundamentally different 
architecture (if it, for example, breaks software compatibility) would 
be a bit of a hard sell.

granted, internally many chips are using the Harvard architecture, but 
externally look and behave as if they followed the Von Neumann 
architecture (for example, we traditionally have a big flat address 
space with both code and data pages).

it is possible that whatever is done internally, will need to behave as 
if it followed this model for sake of being usable.

an example would be if ones' neural-net chips showed up as large 
memory-mapped devices (where neural activation, weights, ... could be 
inspected and modified as needed, or saved/restored from disk files, ...).

however, it is unlikely neural-nets could effectively replace 
traditional architectures, if anything, because they would be ill-suited 
to running existing software.

so, if anything, they would likely have to coexist, with software spread 
between multiple devices.

more so, I have doubts that a "fully generic" neural net (every neuron 
potentially connected to every other) can likely be easily scaled up to 
the proposed levels (millions of neurons), without needing to compromise 
somewhere (essentially, creating some sort of "auxiliary architecture" 
within the network):
say, some sets of neurons connect to adjacent neighbors;
other ones connect between nearby clusters;
others are linked across the network, or represent a shared "global 
signaling state" or similar.

ultimately, the "shape" of the above network could influence the 
behavior of the network as a whole.

> The proposed program sounds a bit like surveying the battlefield at
> Gettysburg on July 4, 1863 with a design for a new rifle and claiming
> that you could have changed the outcome with your technology (maybe a
> plausible claim if your new technology was a modern assault rifle).
>
> As a way of exploring wildly different ways of doing computing, the IBM,
> DARPA, and BU programs might actually make sense. As yet another attack
> on the problem of artificial intelligence, it makes about as much sense
> as Pickett's charge.
>

fair enough.

I don't think it particularly likely that either neural net chips, or 
for that matter, conventional x86 or ARM chips, or GPUs, will suddenly 
lead to "intelligence".

actually, at the moment I more lean towards a "weak AI" position, 
whereby one can emulate some aspects of intelligent agents, but I have 
my doubts as to whether or not human-like intelligence (Strong AI) is 
necessarily actually possible (in the near term, or possibly at all).

Thread

IBM demos cognitive computer chips Brett Davis <ggtgp@yahoo.com> - 2011-08-18 18:02 -0500
  Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-18 20:37 -0400
    Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-19 13:40 -0400
      Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-19 11:52 -0700
        Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-19 15:50 -0400
          Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-20 00:15 -0700
            Re: IBM demos cognitive computer chips Stephen Fuld <SFuld@alumni.cmu.edu.invalid> - 2011-08-20 09:22 -0700
              Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-20 14:10 -0400
        Re: IBM demos cognitive computer chips Quadibloc <jsavard@ecn.ab.ca> - 2011-08-21 05:16 -0700
          Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-21 11:56 -0700
            Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-21 15:29 -0400
              Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-21 13:19 -0700
            Re: IBM demos cognitive computer chips "Andy \"Krazy\" Glew" <andy@SPAM.comp-arch.net> - 2011-08-22 09:41 -0400
              Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-22 12:22 -0700
                Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-22 13:01 -0700
              Re: IBM demos cognitive computer chips "Andy \"Krazy\" Glew" <andy@SPAM.comp-arch.net> - 2011-08-23 10:11 -0400
              Re: IBM demos cognitive computer chips Eric  Northup <digitaleric@gmail.com> - 2011-08-23 09:36 -0700
                Re: IBM demos cognitive computer chips "Andy \"Krazy\" Glew" <andy@SPAM.comp-arch.net> - 2011-08-24 22:15 -0400
                Re: IBM demos cognitive computer chips "Andy \"Krazy\" Glew" <andy@SPAM.comp-arch.net> - 2011-08-25 09:07 -0400
      Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-20 01:41 -0400
      Re: IBM demos cognitive computer chips Stephen Fuld <SFuld@alumni.cmu.edu.invalid> - 2011-08-20 08:55 -0700
        Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-20 16:10 -0400
          Re: IBM demos cognitive computer chips Stephen Fuld <SFuld@alumni.cmu.edu.invalid> - 2011-08-21 21:11 -0700
            Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-22 12:04 -0400
              Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-22 12:22 -0400
            Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-22 14:24 -0400
              Re: IBM demos cognitive computer chips Quadibloc <jsavard@ecn.ab.ca> - 2011-08-22 19:32 -0700
                Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-22 23:21 -0700
                Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-23 09:49 -0700
                Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-23 09:46 -0700
                Re: IBM demos cognitive computer chips BGB <cr88192@hotmail.com> - 2011-08-23 15:05 -0700
                Re: IBM demos cognitive computer chips Piotr Wyderski <peter.pan@neverland.mil> - 2011-08-26 16:04 +0200
                Re: IBM demos cognitive computer chips Quadibloc <jsavard@ecn.ab.ca> - 2011-08-26 09:58 -0700
                Re: IBM demos cognitive computer chips ChrisQ <meru@devnull.com> - 2011-08-29 14:56 +0100
                Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-29 11:06 -0400
                Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-29 16:10 -0400
                Re: IBM demos cognitive computer chips EricP <ThatWouldBeTelling@thevillage.com> - 2011-08-29 17:48 -0400
                Re: IBM demos cognitive computer chips Robert Myers <rbmyersusa@gmail.com> - 2011-08-29 19:24 -0400
    Re: IBM demos cognitive computer chips ggtgp@yahoo.com - 2012-03-30 21:54 -0700

csiph-web