Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #101275 > unrolled thread

Why is there difference between cmd line and .py file?

Back to article view | Back to comp.lang.python

Contents

  Why is there difference between cmd line and .py file? Robert <rxjwg98@gmail.com> - 2016-01-05 12:25 -0800
    Re: Why is there difference between cmd line and .py file? John Gordon <gordon@panix.com> - 2016-01-05 20:37 +0000
      Re: Why is there difference between cmd line and .py file? Robert <rxjwg98@gmail.com> - 2016-01-05 12:45 -0800
        Re: Why is there difference between cmd line and .py file? Joel Goldstick <joel.goldstick@gmail.com> - 2016-01-05 15:58 -0500
          Re: Why is there difference between cmd line and .py file? Robert <rxjwg98@gmail.com> - 2016-01-05 13:05 -0800
          Re: Why is there difference between cmd line and .py file? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-01-06 00:21 +0100
      Re: Why is there difference between cmd line and .py file? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-01-06 16:25 +1100
    Re: Why is there difference between cmd line and .py file? Robert <rxjwg98@gmail.com> - 2016-01-05 12:41 -0800
      Re: Why is there difference between cmd line and .py file? Thomas 'PointedEars' Lahn <PointedEars@web.de> - 2016-01-06 00:25 +0100
        Re: Why is there difference between cmd line and .py file? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-01-06 16:18 +1100
    Re: Why is there difference between cmd line and .py file? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2016-01-06 16:33 +1100

#101275 — Why is there difference between cmd line and .py file?

Hi,

I run below code, which is downloaded from link:

http://stackoverflow.com/questions/15513792/expectation-maximization-coin-toss-examples?rq=1



////////////
# represent the experiments
head_counts = np.array([5,9,8,4,7])
tail_counts = 10-head_counts
experiments = zip(head_counts,tail_counts)

# initialise the pA(heads) and pB(heads)
pA_heads = np.zeros(100); pA_heads[0] = 0.60
pB_heads = np.zeros(100); pB_heads[0] = 0.50

# E-M begins!
delta = 0.001  
j = 0 # iteration counter
improvement = float('inf')
while (improvement>delta):
    expectation_A = np.zeros((5,2), dtype=float) 
    expectation_B = np.zeros((5,2), dtype=float)
    for i in range(0,len(experiments)):
        e = experiments[i] # i'th experiment
        ll_A = get_mn_log_likelihood(e,np.array([pA_heads[j],1-pA_heads[j]])) # loglikelihood of e given coin A
        ll_B = get_mn_log_likelihood(e,np.array([pB_heads[j],1-pB_heads[j]])) # loglikelihood of e given coin B

        weightA = math.exp(ll_A) / ( math.exp(ll_A) + math.exp(ll_B) ) # corresponding weight of A proportional to likelihood of A 
        weightB = math.exp(ll_B) / ( math.exp(ll_A) + math.exp(ll_B) ) # corresponding weight of B proportional to likelihood of B                            

        expectation_A[i] = np.dot(weightA, e) 
        expectation_B[i] = np.dot(weightB, e)

    print "sum(expectation_A)[0]", sum(expectation_A)[0]
    pA_heads[j+1] = sum(expectation_A)[0] / sum(sum(expectation_A)); 
    pB_heads[j+1] = sum(expectation_B)[0] / sum(sum(expectation_B));
//////////
The print message is: 
sum(expectation_A)[0] 21.2974818963
sum(expectation_A)[0] 19.2093824201
sum(expectation_A)[0] 19.4102767983
sum(expectation_A)[0] 19.7538328728
sum(expectation_A)[0] 19.9753285438
sum(expectation_A)[0] 20.0879016403
sum(expectation_A)[0] 20.139820175
sum(expectation_A)[0] 20.1628374165



When I check below in detail in interactive mode (Canopy Python shell),
 
sum(expectation_A)[0]

it has error:
/////////////
sum(expectation_A)[0]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-145-d6f33dff0343> in <module>()
----> 1 sum(expectation_A)[0]

IndexError: invalid index to scalar variable. 
//////////////

I can see expectation_A content with:

In[146]:expectation_A
Out[146]: 
array([[ 0.52278641,  0.52278641],
       [ 8.55858656,  0.95095406],
       [ 6.75024946,  1.68756237],
       [ 0.1260128 ,  0.1890192 ],
       [ 4.20520218,  1.80222951]])

It looks like 
sum(expectation_A)[0]

can run in .py file, while it cannot be run in python shell.
Is it true?

Thanks,

[toc] | [next] | [standalone]

#101276

In <e9f1d94f-4dd6-49b5-bf5c-bbcdc8d304e8@googlegroups.com> Robert <rxjwg98@gmail.com> writes:

> ////////////
> # represent the experiments
> head_counts = np.array([5,9,8,4,7])

The code doesn't define 'np', so this line should produce an error.

The code you linked contains this import:

    import numpy as np

However you didn't show it here, so I wonder if you posted the real code.

> sum(expectation_A)[0]
> ---------------------------------------------------------------------------
> IndexError                                Traceback (most recent call last)
> <ipython-input-145-d6f33dff0343> in <module>()
> ----> 1 sum(expectation_A)[0]

> IndexError: invalid index to scalar variable. 
> //////////////

The built-in function sum() returns a single value, not a list, so this
is a reasonable error.

I suspect the code actually intended to call numpy.sum(), which does
return a list (or something like a list).

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon@panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"

[toc] | [prev] | [next] | [standalone]

#101278

On Tuesday, January 5, 2016 at 3:37:53 PM UTC-5, John Gordon wrote:
> In <e9f1d94f-4dd6-49b5-bf5c-bbcdc8d304e8@googlegroups.com> Robert <r--@gmail.com> writes:
> 
> > ////////////
> > # represent the experiments
> > head_counts = np.array([5,9,8,4,7])
> 
> The code doesn't define 'np', so this line should produce an error.
> 
> The code you linked contains this import:
> 
>     import numpy as np
> 
> However you didn't show it here, so I wonder if you posted the real code.
> 
> > sum(expectation_A)[0]
> > ---------------------------------------------------------------------------
> > IndexError                                Traceback (most recent call last)
> > <ipython-input-145-d6f33dff0343> in <module>()
> > ----> 1 sum(expectation_A)[0]
> 
> > IndexError: invalid index to scalar variable. 
> > //////////////
> 
> The built-in function sum() returns a single value, not a list, so this
> is a reasonable error.
> 
> I suspect the code actually intended to call numpy.sum(), which does
> return a list (or something like a list).
> 
> -- 
> John Gordon                   A is for Amy, who fell down the stairs
> gordon@panix.com              B is for Basil, assaulted by bears
>                                 -- Edward Gorey, "The Gashlycrumb Tinies"

Thanks, John. When I typed my new thought, your reply came. You are right.
 The attached message missed numpy import (In my file, it had).

Now, I cannot use np.sum. It has an error, see below please. How can I use
the numpy sum()?

Thanks,
/////////
import numpy as np

In [154]: np.sum(expectation_A)[0]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-154-1e2cbda689a3> in <module>()
----> 1 np.sum(expectation_A)[0]

IndexError: invalid index to scalar variable.

[toc] | [prev] | [next] | [standalone]

#101279

On Tue, Jan 5, 2016 at 3:45 PM, Robert <rxjwg98@gmail.com> wrote:

> On Tuesday, January 5, 2016 at 3:37:53 PM UTC-5, John Gordon wrote:
> > In <e9f1d94f-4dd6-49b5-bf5c-bbcdc8d304e8@googlegroups.com> Robert <
> r--@gmail.com> writes:
> >
> > > ////////////
> > > # represent the experiments
> > > head_counts = np.array([5,9,8,4,7])
> >
> > The code doesn't define 'np', so this line should produce an error.
> >
> > The code you linked contains this import:
> >
> >     import numpy as np
> >
> > However you didn't show it here, so I wonder if you posted the real code.
> >
> > > sum(expectation_A)[0]
> > >
> ---------------------------------------------------------------------------
> > > IndexError                                Traceback (most recent call
> last)
> > > <ipython-input-145-d6f33dff0343> in <module>()
> > > ----> 1 sum(expectation_A)[0]
> >
> > > IndexError: invalid index to scalar variable.
> > > //////////////
> >
> > The built-in function sum() returns a single value, not a list, so this
> > is a reasonable error.
> >
> > I suspect the code actually intended to call numpy.sum(), which does
> > return a list (or something like a list).
> >
> > --
> > John Gordon                   A is for Amy, who fell down the stairs
> > gordon@panix.com              B is for Basil, assaulted by bears
> >                                 -- Edward Gorey, "The Gashlycrumb Tinies"
>
> Thanks, John. When I typed my new thought, your reply came. You are right.
>  The attached message missed numpy import (In my file, it had).
>
> Now, I cannot use np.sum. It has an error, see below please. How can I use
> the numpy sum()?
>
> Thanks,
> /////////
> import numpy as np
>
> In [154]: np.sum(expectation_A)[0]
> ---------------------------------------------------------------------------
> IndexError                                Traceback (most recent call last)
> <ipython-input-154-1e2cbda689a3> in <module>()
> ----> 1 np.sum(expectation_A)[0]
>
> IndexError: invalid index to scalar variable.
>

I've not used numpy, but you should print expectation_A to see what's in
it.  It may be empty, causing the index.  It may be that expectation_A is
an integer, not a list

>
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

[toc] | [prev] | [next] | [standalone]

#101280

On Tuesday, January 5, 2016 at 3:58:44 PM UTC-5, Joel Goldstick wrote:
> On Tue, Jan 5, 2016 at 3:45 PM, Robert <rxjwg98@gmail.com> wrote:
> 
> > On Tuesday, January 5, 2016 at 3:37:53 PM UTC-5, John Gordon wrote:
> > > In <e9f1d94f-4dd6-49b5-bf5c-bbcdc8d304e8@googlegroups.com> Robert <
> > r--@gmail.com> writes:
> > >
> > > > ////////////
> > > > # represent the experiments
> > > > head_counts = np.array([5,9,8,4,7])
> > >
> > > The code doesn't define 'np', so this line should produce an error.
> > >
> > > The code you linked contains this import:
> > >
> > >     import numpy as np
> > >
> > > However you didn't show it here, so I wonder if you posted the real code.
> > >
> > > > sum(expectation_A)[0]
> > > >
> > ---------------------------------------------------------------------------
> > > > IndexError                                Traceback (most recent call
> > last)
> > > > <ipython-input-145-d6f33dff0343> in <module>()
> > > > ----> 1 sum(expectation_A)[0]
> > >
> > > > IndexError: invalid index to scalar variable.
> > > > //////////////
> > >
> > > The built-in function sum() returns a single value, not a list, so this
> > > is a reasonable error.
> > >
> > > I suspect the code actually intended to call numpy.sum(), which does
> > > return a list (or something like a list).
> > >
> > > --
> > > John Gordon                   A is for Amy, who fell down the stairs
> > > gordon@panix.com              B is for Basil, assaulted by bears
> > >                                 -- Edward Gorey, "The Gashlycrumb Tinies"
> >
> > Thanks, John. When I typed my new thought, your reply came. You are right.
> >  The attached message missed numpy import (In my file, it had).
> >
> > Now, I cannot use np.sum. It has an error, see below please. How can I use
> > the numpy sum()?
> >
> > Thanks,
> > /////////
> > import numpy as np
> >
> > In [154]: np.sum(expectation_A)[0]
> > ---------------------------------------------------------------------------
> > IndexError                                Traceback (most recent call last)
> > <ipython-input-154-1e2cbda689a3> in <module>()
> > ----> 1 np.sum(expectation_A)[0]
> >
> > IndexError: invalid index to scalar variable.
> >
> 
> I've not used numpy, but you should print expectation_A to see what's in
> it.  It may be empty, causing the index.  It may be that expectation_A is
> an integer, not a list
> 
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
> 
> 
> 
> -- 
> Joel Goldstick
> http://joelgoldstick.com/stats/birthdays

Thanks. It has sum method. OK now.

[toc] | [prev] | [next] | [standalone]

#101281

Joel Goldstick wrote:

> On Tue, Jan 5, 2016 at 3:45 PM, Robert <rxjwg98@gmail.com> wrote:
>> import numpy as np
>>
>> In [154]: np.sum(expectation_A)[0]
>> […]
>> IndexError: invalid index to scalar variable.
> 
> I've not used numpy, but you should print expectation_A to see what's in
> it.  It may be empty, causing the index.

Did you mean “IndexError” instead of “index”?

> It may be that expectation_A is an integer, not a list

Please think about this again.

| $ python3
| Python 3.4.4 (default, Dec 21 2015, 09:19:42)
| [GCC 5.3.1 20151219] on linux
| Type "help", "copyright", "credits" or "license" for more information.
| >>> import numpy
| >>> print(numpy.sum.__doc__)
| 
| Sum of array elements over a given axis.

The problem is instead that np.sum(…) *returns* a scalar value (as expected 
from a summation method when passed only scalar values) and _not_ a list 
(insofar the error message is misleading; there is no scalar *variable* 
causing the problem, but a scalar return *value*):

|     Parameters
|     ----------
|     a : array_like
|         Elements to sum.
| […]
| 
|     Returns
|     -------
|     sum_along_axis : ndarray
|         An array with the same shape as `a`, with the specified
|         axis removed.   If `a` is a 0-d array, or if `axis` is None, a 
|         scalar is returned.  If an output array is specified, a reference
|         to `out` is returned.
| 
| […]
| 
|     Examples
|     --------
|     >>> np.sum([0.5, 1.5])
|     2.0
| 
`----

So you cannot use the index notation with the *return* value.  Try e.g. 
42[0]; it does not work because it does not make sense.

It is more likely that the index notation was misplaced: 

np.sum(expectation_A[0])

would make sense if the first element of the iterable referred to by 
“expectation_A” were a numeric scalar or a reference to a list.


Please trim your quotes to the relevant minimum.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#101286

On Wednesday 06 January 2016 07:37, John Gordon wrote:

> The built-in function sum() returns a single value, not a list, so this
> is a reasonable error.

Not quite. It depends on what arguments you give it.

py> a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
py> sum(a, [])
[1, 2, 3, 4, 5, 6, 7, 8, 9]


But your point is well taken. In this case, the call:

sum(expectation_A)

defaults to an initial value of 0, so expectation_A almost certainly is a 
list of numbers, in which case sum will return a single number, not a list 
or array.

-- 
Steve

[toc] | [prev] | [next] | [standalone]

#101277

On Tuesday, January 5, 2016 at 3:26:15 PM UTC-5, Robert wrote:
> Hi,
> 
> I run below code, which is downloaded from link:
> 
> http://stackoverflow.com/questions/15513792/expectation-maximization-coin-toss-examples?rq=1
> 
> 
> 
> ////////////
> # represent the experiments
> head_counts = np.array([5,9,8,4,7])
> tail_counts = 10-head_counts
> experiments = zip(head_counts,tail_counts)
> 
> # initialise the pA(heads) and pB(heads)
> pA_heads = np.zeros(100); pA_heads[0] = 0.60
> pB_heads = np.zeros(100); pB_heads[0] = 0.50
> 
> # E-M begins!
> delta = 0.001  
> j = 0 # iteration counter
> improvement = float('inf')
> while (improvement>delta):
>     expectation_A = np.zeros((5,2), dtype=float) 
>     expectation_B = np.zeros((5,2), dtype=float)
>     for i in range(0,len(experiments)):
>         e = experiments[i] # i'th experiment
>         ll_A = get_mn_log_likelihood(e,np.array([pA_heads[j],1-pA_heads[j]])) # loglikelihood of e given coin A
>         ll_B = get_mn_log_likelihood(e,np.array([pB_heads[j],1-pB_heads[j]])) # loglikelihood of e given coin B
> 
>         weightA = math.exp(ll_A) / ( math.exp(ll_A) + math.exp(ll_B) ) # corresponding weight of A proportional to likelihood of A 
>         weightB = math.exp(ll_B) / ( math.exp(ll_A) + math.exp(ll_B) ) # corresponding weight of B proportional to likelihood of B                            
> 
>         expectation_A[i] = np.dot(weightA, e) 
>         expectation_B[i] = np.dot(weightB, e)
> 
>     print "sum(expectation_A)[0]", sum(expectation_A)[0]
>     pA_heads[j+1] = sum(expectation_A)[0] / sum(sum(expectation_A)); 
>     pB_heads[j+1] = sum(expectation_B)[0] / sum(sum(expectation_B));
> //////////
> The print message is: 
> sum(expectation_A)[0] 21.2974818963
> sum(expectation_A)[0] 19.2093824201
> sum(expectation_A)[0] 19.4102767983
> sum(expectation_A)[0] 19.7538328728
> sum(expectation_A)[0] 19.9753285438
> sum(expectation_A)[0] 20.0879016403
> sum(expectation_A)[0] 20.139820175
> sum(expectation_A)[0] 20.1628374165
> 
> 
> 
> When I check below in detail in interactive mode (Canopy Python shell),
>  
> sum(expectation_A)[0]
> 
> it has error:
> /////////////
> sum(expectation_A)[0]
> ---------------------------------------------------------------------------
> IndexError                                Traceback (most recent call last)
> <ipython-input-145-d6f33dff0343> in <module>()
> ----> 1 sum(expectation_A)[0]
> 
> IndexError: invalid index to scalar variable. 
> //////////////
> 
> I can see expectation_A content with:
> 
> In[146]:expectation_A
> Out[146]: 
> array([[ 0.52278641,  0.52278641],
>        [ 8.55858656,  0.95095406],
>        [ 6.75024946,  1.68756237],
>        [ 0.1260128 ,  0.1890192 ],
>        [ 4.20520218,  1.80222951]])
> 
> It looks like 
> sum(expectation_A)[0]
> 
> can run in .py file, while it cannot be run in python shell.
> Is it true?
> 
> Thanks,

I just wonder that the cmd line function sum may be different from the 
.py file used. One is numpy package, the other is a general one. Then,
how can I further make it clear for this guess?
Thanks,

[toc] | [prev] | [next] | [standalone]

#101282

Robert wrote:

> I just wonder that the cmd line function sum may be different from the
> .py file used. One is numpy package, the other is a general one. Then,
> how can I further make it clear for this guess?

Among other things:

print(sum.__doc__)

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.

[toc] | [prev] | [next] | [standalone]

#101285

On Wednesday 06 January 2016 10:25, Thomas 'PointedEars' Lahn wrote:

> Robert wrote:
> 
>> I just wonder that the cmd line function sum may be different from the
>> .py file used. One is numpy package, the other is a general one. Then,
>> how can I further make it clear for this guess?
> 
> Among other things:
> 
> print(sum.__doc__)


Better:

help(sum)

in the interactive shell.


-- 
Steve

[toc] | [prev] | [next] | [standalone]

#101287

On Wednesday 06 January 2016 07:25, Robert wrote:

> Why is there difference between cmd line and .py file?

Almost certainly because you are not running exactly the same code each 
time.


> I run below code, which is downloaded from link:

Your code fails on the first line with 

    NameError: name 'np' is not defined


so you are obviously running something different from what you post here. 
Since we don't know what code you are actually running, we cannot tell you 
why it behaves differently when run as a .py file and in the interactive 
interpreter.

But I am 99% sure that it is because you are running slightly different 
pieces of code.

My *guess* is that in the code that works correctly, you are running:

import numpy as np
from numpy import sum

at the beginning of the .py script, but in the interactive shell, you are 
just running:

import numpy as np


But there may be other differences as well.



-- 
Steve

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web