Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!feeder1.cambriumusenet.nl!feed.tweaknews.nl!194.109.133.84.MISMATCH!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <4e1fd009$0$29986$c3e8da3$5496439d@news.astraweb.com>
References: <4e1fd009$0$29986$c3e8da3$5496439d@news.astraweb.com>
Date: Fri, 15 Jul 2011 11:02:08 -0600
Subject: Re: None versus MISSING sentinel -- request for design feedback
From: Eric Snow <ericsnowcurrently@gmail.com>
To: "Steven D'Aprano" <steve+comp.lang.python@pearwood.info>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1070.1310749330.1164.python-list@python.org>
Lines: 116
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:9556

On Thu, Jul 14, 2011 at 11:28 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Hello folks,
>
> I'm designing an API for some lightweight calculator-like statistics
> functions, such as mean, standard deviation, etc., and I want to support
> missing values. Missing values should be just ignored. E.g.:
>
> mean([1, 2, MISSING, 3]) =3D> 6/3 =3D 2 rather than 6/4 or raising an err=
or.
>
> My question is, should I accept None as the missing value, or a dedicated
> singleton?
>
> In favour of None: it's already there, no extra code required. People may
> expect it to work.
>
> Against None: it's too easy to mistakenly add None to a data set by mista=
ke,
> because functions return None by default.

Good point.

>
> In favour of a dedicated MISSING singleton: it's obvious from context. It=
's
> not a lot of work to implement compared to using None. Hard to accidental=
ly
> include it by mistake. If None does creep into the data by accident, you
> get a nice explicit exception.

Also good points.

>
> Against MISSING: users may expect to be able to choose their own sentinel=
 by
> assigning to MISSING. I don't want to support that.
>
>
> I've considered what other packages do:-
>
> R uses a special value, NA, to stand in for missing values. This is more =
or
> less the model I wish to follow.
>
> I believe that MATLAB treats float NANs as missing values. I consider thi=
s
> an abuse of NANs and I won't be supporting that :-P

I was just thinking of this.  :)

>
> Spreadsheets such as Excel, OpenOffice and Gnumeric generally ignore blan=
k
> cells, and give you a choice between ignoring text and treating it as zer=
o.
> E.g. with cells set to [1, 2, "spam", 3] the AVERAGE function returns 2 a=
nd
> the AVERAGEA function returns 1.5.
>
> numpy uses masked arrays, which is probably over-kill for my purposes; I =
am
> gratified to see it doesn't abuse NANs:
>
>>>> import numpy as np
>>>> a =3D np.array([1, 2, float('nan'), 3])
>>>> np.mean(a)
> nan
>
> numpy also treats None as an error:
>
>>>> a =3D np.array([1, 2, None, 3])
>>>> np.mean(a)
> Traceback (most recent call last):
> =A0File "<stdin>", line 1, in <module>
> =A0File "/usr/lib/python2.5/site-packages/numpy/core/fromnumeric.py", lin=
e
> 860, in mean
> =A0 =A0return mean(axis, dtype, out)
> TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
>
>
> I would appreciate any comments, advice or suggestions.
>

Too bad there isn't a good way to "freeze" a name, i.e. indicate that
any attempt to rebind it is an exception.  Trying to rebind None is a
SyntaxError, but a NameError or something would be fine.  Then the
downside of using your own sentinel here goes away.

In reality, using Missing may be your best bet anyway.  If there were
a convention for indicating a name should not be re-bound (like a
single leading underscore indicates "private"), you could use that
(all caps?).  Since "we're all consenting adults" it would probably be
good enough to make sure others know that Missing should not be
re-bound...

I might have said to use NotImplemented instead of None, but it can be
re-bound and the name isn't as helpful for your use case.

Another solution, perhaps ugly or confusing, is to use something like
two underscores as the name for your sentinel:

mean([1, 2, __, 3])

Still it seems like using Missing (or whatever) would be better than None.

-eric

>
> --
> Steven
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>