Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #36316

Re: Numpy outlier removal

Path csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <oscar.j.benjamin@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.013
X-Spam-Evidence '*H*': 0.97; '*S*': 0.00; 'dict': 0.09; 'excluding': 0.09; 'explanation': 0.09; 'cc:addr:python-list': 0.10; 'cc:name:python list': 0.16; 'include.': 0.16; 'integers.': 0.16; 'numpy': 0.16; 'operation.': 0.16; 'robust.': 0.16; 'to:addr:pearwood.info': 0.16; 'to:addr:steve+comp.lang.python': 0.16; "to:name:steven d'aprano": 0.16; 'wrote:': 0.17; 'jan': 0.18; 'cc:2**0': 0.23; 'thus': 0.24; 'cc:addr:python.org': 0.25; 'header:In-Reply-To:1': 0.25; 'values': 0.26; 'message- id:@mail.gmail.com': 0.27; "d'aprano": 0.29; 'far,': 0.29; 'steven': 0.29; 'array': 0.29; 'no,': 0.29; "i'm": 0.29; 'running': 0.32; 'received:google.com': 0.34; 'text': 0.34; 'list': 0.35; 'received:209.85': 0.35; 'really': 0.36; 'received:209': 0.37; 'data': 0.37; 'subject:: ': 0.38; 'sure': 0.38; 'where': 0.40; 'skip:" 10': 0.40; 'header:Received:5': 0.40; 'first': 0.61; 'more': 0.63; '2013': 0.84; 'oscar': 0.84; 'subject:removal': 0.84; 'technique': 0.93
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=VRdcwuSa6q4RwTwiG1URKmx0jTNpinrOGmF3tHcxIVg=; b=NJ3drRO6K57IxDDcfUfbzoSNBfOzNjcwJIVQsToTrIOWpCkbQjHQ2sZF/vat9gStxT Q55SPeNCcvSjE952wUDFnPcSnFmPHAnZxWL7Vu1qewQ4pm0tBmfiv42BFpzfGKbUklvk ngdy/S8ngXzC311+PoXgmjYakp07WZKY0xExe29u4JbGR0SzN8Z1y18d/jdIiQefVPa3 L2IjJlSe7xaFixfbyX1Sw2vdDzOgG88J5NNObmvzt0IZyW2QvFPZOG/A9zDMbUsRy7Sg kaDoKiyu0qCMaZdzeyy8ZX4apprCVC4M8lHpvZ9LuwPp1kudas2JFzg6Ukxh72E0p7xJ 9Cew==
MIME-Version 1.0
In-Reply-To <50ea28e7$0$30003$c3e8da3$5496439d@news.astraweb.com>
References <mailman.179.1357501521.2939.python-list@python.org> <50ea28e7$0$30003$c3e8da3$5496439d@news.astraweb.com>
Date Mon, 7 Jan 2013 02:29:27 +0000
Subject Re: Numpy outlier removal
From Oscar Benjamin <oscar.j.benjamin@gmail.com>
To "Steven D'Aprano" <steve+comp.lang.python@pearwood.info>
Content-Type text/plain; charset=ISO-8859-1
Cc Python List <python-list@python.org>
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.15
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.205.1357525775.2939.python-list@python.org> (permalink)
Lines 25
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1357525775 news.xs4all.nl 6945 [2001:888:2000:d::a6]:34361
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:36316

Show key headers only | View raw

On 7 January 2013 01:46, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Sun, 06 Jan 2013 19:44:08 +0000, Joseph L. Casale wrote:
>
>> I have a dataset that consists of a dict with text descriptions and
>> values that are integers. If required, I collect the values into a list
>> and create a numpy array running it through a simple routine:
>>
>> data[abs(data - mean(data)) < m * std(data)]
>>
>> where m is the number of std deviations to include.
>
> I'm not sure that this approach is statistically robust. No, let me be
> even more assertive: I'm sure that this approach is NOT statistically
> robust, and may be scientifically dubious.

Whether or not this is "statistically robust" requires more
explanation about the OP's intention. Thus far, the OP has not given
any reason/motivation for excluding data or even for having any data
in the first place! It's hard to say whether any technique applied is
really accurate/robust without knowing *anything* about the purpose of
the operation.


Oscar

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Numpy outlier removal "Joseph L. Casale" <jcasale@activenetwerx.com> - 2013-01-06 19:44 +0000
  Re: Numpy outlier removal Hans Mulder <hansmu@xs4all.nl> - 2013-01-06 23:33 +0100
    RE: Numpy outlier removal "Joseph L. Casale" <jcasale@activenetwerx.com> - 2013-01-06 22:50 +0000
    Re: Numpy outlier removal MRAB <python@mrabarnett.plus.com> - 2013-01-06 23:18 +0000
  Re: Numpy outlier removal Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-07 01:46 +0000
    Re: Numpy outlier removal "Paul Simon" <psimon@sonic.net> - 2013-01-06 18:21 -0800
    Re: Numpy outlier removal Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-01-07 02:29 +0000
      Re: Numpy outlier removal Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-07 05:11 +0000
        Re: Numpy outlier removal Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-01-07 15:20 +0000
          [Offtopic] Line fitting [was Re: Numpy outlier removal] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-07 17:58 +0000
            Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Chris Angelico <rosuav@gmail.com> - 2013-01-08 06:43 +1100
              Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-08 02:06 +0000
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Chris Angelico <rosuav@gmail.com> - 2013-01-08 17:35 +1100
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Robert Kern <robert.kern@gmail.com> - 2013-01-08 15:55 +0000
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Chris Angelico <rosuav@gmail.com> - 2013-01-09 07:14 +1100
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-09 07:50 +0000
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Robert Kern <robert.kern@gmail.com> - 2013-01-08 22:59 +0000
            Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-01-07 22:32 +0000
              Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-08 01:23 +0000
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Terry Reedy <tjreedy@udel.edu> - 2013-01-08 04:07 -0500
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Maarten <maarten.sneep@knmi.nl> - 2013-01-08 08:47 -0800
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Maarten <maarten.sneep@knmi.nl> - 2013-01-08 08:47 -0800
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-01-09 00:02 +0000
                Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-01-08 13:50 +0000
            Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Jason Friedman <jason@powerpull.net> - 2013-01-08 19:22 -0700
            Re: [Offtopic] Line fitting [was Re: Numpy outlier removal] Jason Friedman <jason@powerpull.net> - 2013-01-08 19:23 -0700
        Re: Numpy outlier removal Robert Kern <robert.kern@gmail.com> - 2013-01-07 15:35 +0000
    RE: Numpy outlier removal "Joseph L. Casale" <jcasale@activenetwerx.com> - 2013-01-07 02:12 +0000

csiph-web