Path: csiph.com!x330-a1.tempe.blueboxinc.net!feeder1.hal-mli.net!weretis.net!feeder4.news.weretis.net!news.netcologne.de!newsfeed-fusi2.netcologne.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail From: Ingo Thies Newsgroups: comp.graphics.apps.gnuplot Subject: Re: Fitting: How does gnuplot calculate the covariance matrix? Date: Mon, 11 Apr 2011 22:13:51 +0200 Lines: 45 Message-ID: <90h5nvFh1hU1@mid.individual.net> References: <9088euFi3iU1@mid.individual.net> <4DA1E330.20903@t-online.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Trace: individual.net Q/gCZ/Qvdo1TZv22bs/XlQEgSTw4TmVR6nEBxQJ6+yNaxj44ex Cancel-Lock: sha1:QmQUiPcDFtt1hj6Gx6AQxjdfaAo= User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; de; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 In-Reply-To: <4DA1E330.20903@t-online.de> Xref: x330-a1.tempe.blueboxinc.net comp.graphics.apps.gnuplot:235 Am 2011-04-10 19:04, schrieb Hans-Bernhard Bröker: > Nobody expects chi^2 to be zero. The expectation is for the reduced > chisquare (chisq/ndf) to be about 1.0. You might have misunderstood me. If you want to calculate the 1-sigma ellipse from the actual chi^2 in the (a,b) map (this cannot be done with gnuplot yet, but with a simple independent program), one has to transform the chi^2 into the exclusion probability (via the incomplete gamma function and the degree of freedom), and then transform this into sigma via the (inverse) error function. But before you throw the chi^2 into this algorithm, you have to substract the chi_min^2, the chi^2 of the best-fit (a,b). I.e. "0 sigma" corresponds to chi^2=chi_min^2, not 0. Otherwise you get too small error contours, and for bad fits, the 1-sigma contour might not even exist. I have to admit that I had indeed neglected to do this correction in Thies & Kroupa (2007), Figure 9. Plotted there are the 95% and 99% (i.e. 2 and 2.6 sigma) contours for the fitted parameters of four initial mass functions (IMF). The fit to the Taurus IMF was relatively poor, with the probability being >95%, making the 2-sigma contour disappear without the correction mentioned above (however, the main result is not affected by this lapsus). This shows the consequence of missing this correction. Here the paper on arxiv: What has all this to do with gnuplot? Well, if the errors of the data are near-Gaussian, you can approximate the error contour in the (a,b) space by an ellipse. The axis vectors of this ellipse are, as far as I remember, the eigenvectors of the covariance matrix which is closely related to the correlation matrix by multiplying/dividing with/by the product of the standard errors of a and b. However, the error corridor you get from walking around the error ellipse and picking (a,b) and use it in f(x), is too narrow. As far as I remember, it is similar to the 1-sigma (a,b) error contour if chi_min^2 is not subtracted. This, the other way round, corresponds to too small a,b standard errors. -- Gruß, Ingo