Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.graphics.apps.gnuplot > #1879 > unrolled thread

fitting to more than 5 variables

Back to article view | Back to comp.graphics.apps.gnuplot

Contents

  fitting to more than 5 variables trg818@gmail.com - 2013-06-07 17:18 -0700
    Re: fitting to more than 5 variables sfeam <sfeam@users.sourceforge.net> - 2013-06-08 15:26 -0700
      Re: fitting to more than 5 variables trg818@gmail.com - 2013-06-09 16:35 -0700
        Re: fitting to more than 5 variables sfeam <sfeam@users.sourceforge.net> - 2013-06-10 16:28 -0700
          Re: fitting to more than 5 variables sfeam <sfeam@users.sourceforge.net> - 2013-06-11 15:59 -0700

#1879 — fitting to more than 5 variables

Hi,
it seems that by default, the non-linear least squares fitting algorithm implemented in Gnuplot is restricted to at most 5 independent variables. I have a dataset that I need to fit to 9 independent variables. Is there a way to do this in Gnuplot, e.g., by setting some environment variable, or is there absolutely no way to achieve this? If it can't be done, are there plans to implement a method that allows for more flexibility, and what alternatives do I have?
Thomas

[toc] | [next] | [standalone]

#1881

trg818@gmail.com wrote:

> Hi,
> it seems that by default, the non-linear least squares fitting
> algorithm implemented in Gnuplot is restricted to at most 5
> independent variables. I have a dataset that I need to fit to 9
> independent variables. Is there a way to do this in Gnuplot, e.g., by
> setting some environment variable, or is there absolutely no way to
> achieve this? If it can't be done, are there plans to implement a
> method that allows for more flexibility, and what alternatives do I
> have? Thomas

Please show us the fit command you are trying to use,
and the error message it gives.

There is not a 5-parameter limit in defining a function. 
E.g. this command is accepted
  F(q1,q2,q3,q4,q5,q6,q7,q8,q9) = q1+q2+q3+q4+q5+q6+q7+q8+q9

Also there is not a 5-parameter limit in the "via" keyword. 
E.g. this command is accepted
  fit Q(x,y) using 1:2:3:(1) via q1,q2,q3,q4,q5,q6,q7,q8

Even the combination is accepted:
  fit F(q1,q2,q3,q4,q5,q6,q7,q8,q9) 'foo.dat'  using 1:2:3 \
      via q1,q2,q3,q4,q5,q6,q7,q8,q9

So what limit have you encountered, exactly?

[toc] | [prev] | [next] | [standalone]

#1883

On Sunday, June 9, 2013 12:26:55 AM UTC+2, sfeam wrote:
> There is not a 5-parameter limit in defining a function. 
Correct, but that's not the problem.
> Also there is not a 5-parameter limit in the "via" keyword. 
>   fit Q(x,y) using 1:2:3:(1) via q1,q2,q3,q4,q5,q6,q7,q8
> Even the combination is accepted:
>   fit F(q1,q2,q3,q4,q5,q6,q7,q8,q9) 'foo.dat'  using 1:2:3 \
>       via q1,q2,q3,q4,q5,q6,q7,q8,q9
It is accepted because you are fitting a function of only 3 independent variables that has 9 parameters/coefficients. My function has eight (not nine, as I originally wrote) independent variables, i.e., the data space is eight-dimensional, and there are 9 parameters:

P(l,T,d,XS,XC,XS2,XN,N)=a*l+ \
(b+c*d)/T+d*log10(1.-XS)+e*log10(1.-XC)+f*log10(1.-XS2)+g*log10(1.-XN)+h*N+i
fit P(l,T,d,XS,XC,XS2,XN,N) 'DU.dat' via a,b,c,d,e,f,g,h,i

The error I get is:
         undefined variable: l
and if I remove the term with the l and try with only 8 variables, the error is that T is an undefined variable, and so on.
> So what limit have you encountered, exactly?
The online help says: "The fit function may have up to five independent variables." This is with Gnuplot v.4.6 patchlevel 1 on a Mac (macports install).

Thomas

[toc] | [prev] | [next] | [standalone]

#1885

trg818@gmail.com wrote:

> On Sunday, June 9, 2013 12:26:55 AM UTC+2, sfeam wrote:
>> There is not a 5-parameter limit in defining a function.
> Correct, but that's not the problem.
>> Also there is not a 5-parameter limit in the "via" keyword.
>>   fit Q(x,y) using 1:2:3:(1) via q1,q2,q3,q4,q5,q6,q7,q8
>> Even the combination is accepted:
>>   fit F(q1,q2,q3,q4,q5,q6,q7,q8,q9) 'foo.dat'  using 1:2:3 \
>>       via q1,q2,q3,q4,q5,q6,q7,q8,q9
> It is accepted because you are fitting a function of only 3 independent
> variables that has 9 parameters/coefficients. My function has eight (not
> nine, as I originally wrote) independent variables, i.e., the data space
> is eight-dimensional, and there are 9 parameters:
> 
> P(l,T,d,XS,XC,XS2,XN,N)=a*l+ \
> (b+c*d)/T+d*log10(1.-XS)+e*log10(1.-XC)+f*log10(1.-XS2)+g*log10(1.-XN)+h*N+i
> fit P(l,T,d,XS,XC,XS2,XN,N) 'DU.dat' via a,b,c,d,e,f,g,h,i
> 
> The error I get is:
>          undefined variable: l
> and if I remove the term with the l and try with only 8 variables, the
> error is that T is an undefined variable, and so on.
>> So what limit have you encountered, exactly?
> The online help says: "The fit function may have up to five independent
> variables." This is with Gnuplot v.4.6 patchlevel 1 on a Mac (macports
> install).

OK, I understand now.  

The limitation comes not from the number of variables, but from the fact
that the maximum number of input data columns is 7, since this is the
fixed size of the internal structure that holds each data point.

The run-time memory requirements of gnuplot increase proportionally with
the size of this internal data structure.  I.e. redefining the structure
to hold 10 values (8 parameter dimensions + 2 spatial coords) would increase
the memory required by ~42%.  This is trivial for small data sets, but can
become an issue for very large data sets.  Historically there has been more
user demand to handle huge data sets than demand to handle small data sets
with high dimensionality.  

On the other hand, historically these limits were a consequence of 
16- or 32-bit memory indexing. One could argue that the trade-off should be
revisited in the era of 64-bit computing.  Bastian Maerkisch has been working
to incorporate improvements from several sources into gnuplot's fit routines
in the development version.  I do not recall seeing discussion of allowing
higher dimensionality in the data, but maybe I missed it.

If you are willing to be a guinea pig for testing new code, perhaps you should
raise this issue on the gnuplot development mailing list:
 gnuplot-beta@lists.sourceforge.net

I can imagine a stupid-but-easy solution that simply doubles the size of the
data structure on 64-bit systems. This would affect all plotting and fitting.
A cleaner solution might be to allocate space dynamically in the fit routines
if additional dimensions are required.

	Ethan

[toc] | [prev] | [next] | [standalone]

#1887

sfeam wrote:

> I can imagine a stupid-but-easy solution that simply doubles the size of
> the data structure on 64-bit systems. This would affect all plotting and
> fitting. A cleaner solution might be to allocate space dynamically in the
> fit routines if additional dimensions are required.

Having now delved into the code a bit, I think that increasing the data
storage only for fitting would be trivial.  So I expect that the limit of
5 independent parameters can be removed in the development version to
allow testing.  If it tests out OK, the change will appear in the next
released version.

	Ethan

[toc] | [prev] | [standalone]

Back to top | Article view | comp.graphics.apps.gnuplot

csiph-web