Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <oscar.j.benjamin@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.008
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'subject:: [': 0.03; 'algorithm': 0.03; 'output': 0.04; 'sanity': 0.07; '(1,': 0.09; 'regression': 0.09; 'terry': 0.09; 'cc:addr:python-list': 0.10; '1.5,': 0.16; 'benjamin': 0.16; 'cc:name:python list': 0.16; 'subject:Line': 0.16; 'to:addr:pearwood.info': 0.16; 'to:addr:steve+comp.lang.python': 0.16; "to:name:steven d'aprano": 0.16; 'mon,': 0.16; 'wrote:': 0.17; 'jan': 0.18; 'subject:] ': 0.19; 'cc:2**0': 0.23; 'minutes.': 0.23; 'cc:addr:python.org': 0.25; 'header:In-Reply-To:1': 0.25; 'looks': 0.26; 'fit': 0.26; 'message-id:@mail.gmail.com': 0.27; 'lines': 0.28; "d'aprano": 0.29; 'steven': 0.29; 'subject: [': 0.29; 'code': 0.31; '(and': 0.32; 'not.': 0.32; 'could': 0.32; 'getting': 0.33; 'received:google.com': 0.34; 'wrong': 0.34; 'said,': 0.35; 'subject:]': 0.35; 'received:209.85': 0.35; 'really': 0.36; 'wanted': 0.36; 'visual': 0.36; "didn't": 0.36; 'anything': 0.36; 'should': 0.36; 'problems': 0.36; 'received:209': 0.37; 'far': 0.37; 'data': 0.37; 'subject:: ': 0.38; 'some': 0.38; 'easily': 0.39; 'little': 0.39; 'where': 0.40; 'header:Received:5': 0.40; 'think': 0.40; 'your': 0.60; 'situation': 0.62; 'more': 0.63; 'decided': 0.65; 'to,': 0.65; 'hour': 0.69; '2013': 0.84; 'oscar': 0.84; 'refusing': 0.84; 'subject:removal': 0.84; 'visually': 0.84; 'ruler': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=tXG/fLtTZ2J5DEq8zeUKyi8aCRFQmYtdpMltIG5UqJg=; b=mWXk0ChmNJH7zEOPpV0eiwZ5ygiFXXSjFb9TYc2pjpa59twisVFWONZuqVZDAb0aIv zALYPswBNB7TyLpniitSqmnxLXxdAMHgrCP2/tS5o52ysompmBeeJK63LdqriIbHakLD R6OtNiVtoI3a/yM63ao4LIMR27yojfKPVz70Py06+nZjvG7NOUW2vnSqLyo4vysMMqhy RFpGXmle46Nkns0e3tbsFxZFY6rrQQDhcjjcTLVQGw40T43/YxTwYhtpYPjtUf2qpnTL 7uenWKIMSTz5mRKs9HL4ktK+1hq9W0zyo2jPq8k+JMcFvsCcVKK8T78l7EwUk91DMTl1 1/fg==
MIME-Version: 1.0
In-Reply-To: <50eb7513$0$30003$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.179.1357501521.2939.python-list@python.org> <50ea28e7$0$30003$c3e8da3$5496439d@news.astraweb.com> <mailman.205.1357525775.2939.python-list@python.org> <50ea58f0$0$21851$c3e8da3$76491128@news.astraweb.com> <mailman.223.1357572059.2939.python-list@python.org> <50eb0cd2$0$30003$c3e8da3$5496439d@news.astraweb.com> <mailman.239.1357597977.2939.python-list@python.org> <50eb7513$0$30003$c3e8da3$5496439d@news.astraweb.com>
Date: Tue, 8 Jan 2013 13:50:59 +0000
Subject: Re: [Offtopic] Line fitting [was Re: Numpy outlier removal]
From: Oscar Benjamin <oscar.j.benjamin@gmail.com>
To: "Steven D'Aprano" <steve+comp.lang.python@pearwood.info>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Python List <python-list@python.org>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.275.1357653482.2939.python-list@python.org>
Lines: 25
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1357653482 news.xs4all.nl 6986 [2001:888:2000:d::a6]:60007
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:36430

On 8 January 2013 01:23, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> On Mon, 07 Jan 2013 22:32:54 +0000, Oscar Benjamin wrote:
>
> [...]
>> I also think it would
>> be highly foolish to go so far with refusing to eyeball data that you
>> would accept the output of some regression algorithm even when it
>> clearly looks wrong.
>
> I never said anything of the sort.
>
> I said, don't fit lines to data by eye. I didn't say not to sanity check
> your straight line fit is reasonable by eyeballing it.

I should have been a little clearer. That was the situation when I
decided to just use a (digital) ruler - although really it was more of
a visual bisection (1, 2, 1.5, 1.25...). The regression result was
clearly wrong (and also invalid for the reasons Terry has described).
Some of the problems were easily fixable and others were not. I could
have spent an hour getting the code to make the line go where I wanted
it to, or I could just fit the line visually in about 2 minutes.


Oscar