Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #2288 > unrolled thread

fitting from x to y and from y to x

Started byJean Dupont <jeandupont115@gmail.com>
First post2014-02-04 01:29 -0800
Last post2014-02-06 11:54 -0800
Articles 7 — 5 participants

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  fitting from x to y and from y to x Jean Dupont <jeandupont115@gmail.com> - 2014-02-04 01:29 -0800
    Re: fitting from x to y and from y to x Karl <mail.kfr@gmx.net> - 2014-02-04 13:05 +0100
      Re: fitting from x to y and from y to x Jean Dupont <jeandupont115@gmail.com> - 2014-02-04 06:57 -0800
    Re: fitting from x to y and from y to x Ethan A Merritt <merritt@u.washington.edu> - 2014-02-04 12:06 -0800
      Re: fitting from x to y and from y to x Jean Dupont <jeandupont115@gmail.com> - 2014-02-05 06:59 -0800
        Re: fitting from x to y and from y to x Ethan A Merritt <EAMerritt@gmail.com> - 2014-02-05 08:12 -0800
          Re: fitting from x to y and from y to x Jean Dupont <jeandupont314@gmail.com> - 2014-02-06 11:54 -0800

#2288 — fitting from x to y and from y to x

FromJean Dupont <jeandupont115@gmail.com>
Date2014-02-04 01:29 -0800
Subjectfitting from x to y and from y to x
Message-ID<ff8c63b2-6c6a-406f-b109-53e535b59ddd@googlegroups.com>
It happens quite often that one is measuring two data sets, lets call them x and y. It's common also to have a measurement error on x (delta_x) and on y (delta_y).
Then a fitting on f(x) with some parameters (a, b, c...)is performed using delta_y
which when done properly will give reliable values for a, b, c...
It's also interesting however to fit the inverse function of f(x) with y as independent variable and x as dependent variable using delta_x in the fitting.
This will give slightly different values for a,b,c ...One could then take the mean
of both sets a,b,c for an end-result.

I was wondering whether the procedure above can be implemented elegantly in Gnuplot without having to calculate the inverse function explicitly first.


kind regards,
jean

[toc] | [next] | [standalone]

#2289

FromKarl <mail.kfr@gmx.net>
Date2014-02-04 13:05 +0100
Message-ID<lcql34$un3$1@news.rz.uni-karlsruhe.de>
In reply to#2288
Am 04.02.2014 10:29, schrieb Jean Dupont:
> It happens quite often that one is measuring two data sets, lets call them x and y. It's common also to have a measurement error on x (delta_x) and on y (delta_y).
> Then a fitting on f(x) with some parameters (a, b, c...)is performed using delta_y
> which when done properly will give reliable values for a, b, c...
> It's also interesting however to fit the inverse function of f(x) with y as independent variable and x as dependent variable using delta_x in the fitting.
> This will give slightly different values for a,b,c ...One could then take the mean
> of both sets a,b,c for an end-result.
> 
> I was wondering whether the procedure above can be implemented elegantly in Gnuplot without having to calculate the inverse function explicitly first.
> 

quite impossible without the inverse function (provided your function is
strictly monotonic and thus can even have an unambiguous inverse), i´d say.

and i fear that idea lacks some mathematical rigour. What should be the
rationale for taking the mean? If you want to publish that, prepare for
some referee to hang you for it. (OK, 95% of referees would either not
read, understand or care about it ;-) )

one approach could be to just do the fit with dx*dy as the error
variable. Remember that the error does nothing more than give a weight
dx^-2 to each data point (see "help fit"), so you don´t have to worry
about the units.

or you could try for a multi-branch fitting contraption, although i´m
not sure how (or if) that would work.

  Karl

[toc] | [prev] | [next] | [standalone]

#2290

FromJean Dupont <jeandupont115@gmail.com>
Date2014-02-04 06:57 -0800
Message-ID<c6b106b7-d6aa-4658-ae43-c2f600078679@googlegroups.com>
In reply to#2289
Op dinsdag 4 februari 2014 13:05:54 UTC+1 schreef Karl:
> Am 04.02.2014 10:29, schrieb Jean Dupont:
> > It happens quite often that one is measuring two data sets, lets call them x and y. It's common also to have a measurement error on x (delta_x) and on y (delta_y).
> > Then a fitting on f(x) with some parameters (a, b, c...)is performed using delta_y
> > which when done properly will give reliable values for a, b, c...
> > It's also interesting however to fit the inverse function of f(x) with y as independent variable and x as dependent variable using delta_x in the fitting.
> > This will give slightly different values for a,b,c ...One could then take the mean
> > of both sets a,b,c for an end-result.
> > 
> > I was wondering whether the procedure above can be implemented elegantly in Gnuplot without having to calculate the inverse function explicitly first.
> > 
>
> quite impossible without the inverse function (provided your function is
> strictly monotonic and thus can even have an unambiguous inverse), i´d say.
>
My functions are almost always strictly monotonic and have an unambiguous inverse, but writing it out algebraically might indeed be cumbersome.

> and i fear that idea lacks some mathematical rigour. What should be the
> rationale for taking the mean? If you want to publish that, prepare for
> some referee to hang you for it. (OK, 95% of referees would either not
> read, understand or care about it ;-) )
I got the idea from an approach I have seen applied in the context of linear regression analysis (I do know that that differs from what Gnuplot does):
First the regression line from y to x was determined, then the regression line from x to y, then the mean value of slope and intercept was taken for a final
regression line. 
Because I found it strange that only one error dy is used when fitting with Gnuplot while both dx and dy are known, I first was wondering whether there are methods for fitting using dx and dy simultaneously, as far as I understood
this is not possible in Gnuplot, hence my question

kind regards,
jean



> one approach could be to just do the fit with dx*dy as the error
> variable. Remember that the error does nothing more than give a weight
> dx^-2 to each data point (see "help fit"), so you don´t have to worry
> about the units.
>
> or you could try for a multi-branch fitting contraption, although i´m
> not sure how (or if) that would work.
>
>   Karl

[toc] | [prev] | [next] | [standalone]

#2291

FromEthan A Merritt <merritt@u.washington.edu>
Date2014-02-04 12:06 -0800
Message-ID<lcrh92$9mh$1@dont-email.me>
In reply to#2288
Jean Dupont wrote:

> It happens quite often that one is measuring two data sets, lets call them
> x and y. It's common also to have a measurement error on x (delta_x) and
> on y (delta_y). Then a fitting on f(x) with some parameters (a, b, c...)is
> performed using delta_y which when done properly will give reliable values
> for a, b, c... It's also interesting however to fit the inverse function
> of f(x) with y as independent variable and x as dependent variable using
> delta_x in the fitting. This will give slightly different values for a,b,c
> ...One could then take the mean of both sets a,b,c for an end-result.
> 
> I was wondering whether the procedure above can be implemented elegantly
> in Gnuplot without having to calculate the inverse function explicitly
> first.

This has been discussed before several times.  See for example
 http://gnuplot.10905.n7.nabble.com/fit-NLLS-and-Levenberg-Marquardt-td9090.html

No, you don't necessarily need to have inverse functions defined.
However in order to allow properly for errors on x you need a 
different treatment of the minimization procedure.
I would use a maximum-likelihood approach. 
This is not built into gnuplot, but if a search will probably turn
up examples of incorporating gnuplot into the process.
I found this example in just a few minutes of looking:

 http://nbviewer.ipython.org/github/fonnesbeck/Bios366/blob/master/notebooks/Section3_1-Univariate-and-Multivariate-Optimization.ipynb

	Ethan

> 
> 
> kind regards,
> jean

[toc] | [prev] | [next] | [standalone]

#2292

FromJean Dupont <jeandupont115@gmail.com>
Date2014-02-05 06:59 -0800
Message-ID<c3130ce1-a4f8-4a04-950f-fd7313a7cbc2@googlegroups.com>
In reply to#2291
Op dinsdag 4 februari 2014 21:06:56 UTC+1 schreef Ethan A Merritt:
> Jean Dupont wrote:
> 
> 
> 
> > It happens quite often that one is measuring two data sets, lets call them
> 
> > x and y. It's common also to have a measurement error on x (delta_x) and
> 
> > on y (delta_y). Then a fitting on f(x) with some parameters (a, b, c...)is
> 
> > performed using delta_y which when done properly will give reliable values
> 
> > for a, b, c... It's also interesting however to fit the inverse function
> 
> > of f(x) with y as independent variable and x as dependent variable using
> 
> > delta_x in the fitting. This will give slightly different values for a,b,c
> 
> > ...One could then take the mean of both sets a,b,c for an end-result.
> 
> > 
> 
> > I was wondering whether the procedure above can be implemented elegantly
> 
> > in Gnuplot without having to calculate the inverse function explicitly
> 
> > first.
> 
> 
> 
> This has been discussed before several times.  See for example
> 
>  http://gnuplot.10905.n7.nabble.com/fit-NLLS-and-Levenberg-Marquardt-td9090.html
> 
> 
> 
> No, you don't necessarily need to have inverse functions defined.
> 
> However in order to allow properly for errors on x you need a 
> 
> different treatment of the minimization procedure.
> 
> I would use a maximum-likelihood approach. 
> 
> This is not built into gnuplot, but if a search will probably turn
> 
> up examples of incorporating gnuplot into the process.
> 
> I found this example in just a few minutes of looking:
> 
> 
> 
>  http://nbviewer.ipython.org/github/fonnesbeck/Bios366/blob/master/notebooks/Section3_1-Univariate-and-Multivariate-Optimization.ipynb
> 
> 
> 
> 	Ethan
Thank you Ethan for this very interesting reply

kind regards,
jean

[toc] | [prev] | [next] | [standalone]

#2293

FromEthan A Merritt <EAMerritt@gmail.com>
Date2014-02-05 08:12 -0800
Message-ID<lctnpr$24f$1@dont-email.me>
In reply to#2292
Jean Dupont wrote:

> Op dinsdag 4 februari 2014 21:06:56 UTC+1 schreef Ethan A Merritt:
>> Jean Dupont wrote:
>> 
>> 
>> 
>> > It happens quite often that one is measuring two data sets, lets
>> > call them
>> 
>> > x and y. It's common also to have a measurement error on x
>> > (delta_x) and
>> 
>> > on y (delta_y). Then a fitting on f(x) with some parameters (a, b,
>> > c...)is
>> 
>> > performed using delta_y which when done properly will give reliable
>> > values
>> 
>> > for a, b, c... It's also interesting however to fit the inverse
>> > function
>> 
>> > of f(x) with y as independent variable and x as dependent variable
>> > using
>> 
>> > delta_x in the fitting. This will give slightly different values
>> > for a,b,c
>> 
>> > ...One could then take the mean of both sets a,b,c for an
>> > end-result.
>> 
>> > 
>> 
>> > I was wondering whether the procedure above can be implemented
>> > elegantly
>> 
>> > in Gnuplot without having to calculate the inverse function
>> > explicitly
>> 
>> > first.
>> 
>> 
>> 
>> This has been discussed before several times.  See for example
>> 
>>  http://gnuplot.10905.n7.nabble.com/fit-NLLS-and-Levenberg-Marquardt-td9090.html
>> 
>> 
>> 
>> No, you don't necessarily need to have inverse functions defined.
>> 
>> However in order to allow properly for errors on x you need a
>> 
>> different treatment of the minimization procedure.
>> 
>> I would use a maximum-likelihood approach.
>> 
>> This is not built into gnuplot, but if a search will probably turn
>> 
>> up examples of incorporating gnuplot into the process.
>> 
>> I found this example in just a few minutes of looking:
>> 
>> 
>> 
>>  http://nbviewer.ipython.org/github/fonnesbeck/Bios366/blob/master/notebooks/Section3_1-Univariate-and-Multivariate-Optimization.ipynb
>> 
>> 
>> 
>> Ethan
> Thank you Ethan for this very interesting reply

I forgot to mention that there is a patch against current gnuplot
on the SourceForge tracker that adds a new fitting mode that can
use error estimates on both x and y.  I have not looked at this
myself, but I would be interested in hearing from test users.
It is more likely to be included in a future gnuplot version if
there are multiple people advocating for it.

https://sourceforge.net/p/gnuplot/patches/585/

The patch is based on the "effective variance method" 
(Jay Orear, Am. J. Phys., Vol. 50, No. 10, October 1982)

	Ethan


> 
> kind regards,
> jean

[toc] | [prev] | [next] | [standalone]

#2297

FromJean Dupont <jeandupont314@gmail.com>
Date2014-02-06 11:54 -0800
Message-ID<468378f1-2e8e-4e71-920f-85790e9e72b8@googlegroups.com>
In reply to#2293
Op woensdag 5 februari 2014 17:12:50 UTC+1 schreef Ethan A Merritt:
> Jean Dupont wrote:
> > Op dinsdag 4 februari 2014 21:06:56 UTC+1 schreef Ethan A Merritt:
> >> Jean Dupont wrote:
> >> 
> >> 
> >> 
> >> > It happens quite often that one is measuring two data sets, lets
> >> > call them
> >> 
> >> > x and y. It's common also to have a measurement error on x
> >> > (delta_x) and
> >> 
> >> > on y (delta_y). Then a fitting on f(x) with some parameters (a, b,
> >> > c...)is
> >> 
> >> > performed using delta_y which when done properly will give reliable
> >> > values
> >> 
> >> > for a, b, c... It's also interesting however to fit the inverse
> >> > function
> >> 
> >> > of f(x) with y as independent variable and x as dependent variable
> >> > using
> >> 
> >> > delta_x in the fitting. This will give slightly different values
> >> > for a,b,c
> >> 
> >> > ...One could then take the mean of both sets a,b,c for an
> >> > end-result.
> >> 
> >> > 
> >> 
> >> > I was wondering whether the procedure above can be implemented
> >> > elegantly
> >> 
> >> > in Gnuplot without having to calculate the inverse function
> >> > explicitly
> >> 
> >> > first.
> >> 
> >> 
> >> 
> >> This has been discussed before several times.  See for example
> >> 
> >>  http://gnuplot.10905.n7.nabble.com/fit-NLLS-and-Levenberg-Marquardt-td9090.html
> >> 
> >> 
> >> 
> >> No, you don't necessarily need to have inverse functions defined.
> >> 
> >> However in order to allow properly for errors on x you need a
> >> 
> >> different treatment of the minimization procedure.
> >> 
> >> I would use a maximum-likelihood approach.
> >> 
> >> This is not built into gnuplot, but if a search will probably turn
> >> 
> >> up examples of incorporating gnuplot into the process.
> >> 
> >> I found this example in just a few minutes of looking:
> >> 
> >> 
> >> 
> >>  http://nbviewer.ipython.org/github/fonnesbeck/Bios366/blob/master/notebooks/Section3_1-Univariate-and-Multivariate-Optimization.ipynb
> >> 
> >> 
> >> 
> >> Ethan
> > Thank you Ethan for this very interesting reply
> I forgot to mention that there is a patch against current gnuplot
> on the SourceForge tracker that adds a new fitting mode that can
> use error estimates on both x and y.  I have not looked at this
> myself, but I would be interested in hearing from test users.
> It is more likely to be included in a future gnuplot version if
> there are multiple people advocating for it.
> https://sourceforge.net/p/gnuplot/patches/585/
> The patch is based on the "effective variance method" 
> (Jay Orear, Am. J. Phys., Vol. 50, No. 10, October 1982)
> 	Ethan
Could you describe how to compile the code with this patch? (I'm using
Linux)

kind regards,
jean

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web