Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #7089

Re: Findminimum too slow for iterative reweighted least

From Oliver Ruebenkoenig <ruebenko@wolfram.com>
Newsgroups comp.soft-sys.math.mathematica
Subject Re: Findminimum too slow for iterative reweighted least
Date 2011-12-10 12:37 +0000
Organization Steven M. Christensen and Associates, Inc and MathTensor, Inc.
Message-ID <jbvjq3$j2f$1@smc.vnet.net> (permalink)
References <201112091055.FAA03827@smc.vnet.net>

Show all headers | View raw



On Fri, 9 Dec 2011, Alberto Maydeu-Olivares wrote:

> Hi, I'm trying to use FindMinimum to minimize a weighted least squares
> function where the weight matrix needs to be updated at each
> iteration. The weight matrix involves the inverse of a symbolic
> matrix. This should be fast if FindMinimum first evaluates the
> symbolic matrix, then takes the inverse. Given how long it takes, it's
> obviously not doing that. I wonder if anyone can suggest a way to
> force FindMinimum to do that. I obviously tried Evaluate, but does not
> work. At attach a toy example so that you can try. Finding the minimum
> of this toy example takes a full second. It takes minutes to do a
> realistic size job, It should not take more than a couple of seconds.
>
> Thanks a lot for your help.
>
>
> model = {l1^2 + ps, l2*l1, l2^2 + ps, l3*l2, l3*l1, l3^2 + ps};
> theta = {l1, l2, l3, ps};
> j = Outer[D, model, theta];
> data = {2., .42, 3., .56, .48, 1.};
> startval = Transpose[{theta, {.5, .5, .5, .5}}];
> e = data - model;
> mat = {{2 (l1^2 + ps)^2, 2 l1 l2 (l1^2 + ps), 2 l1^2 l2^2,
>    2 l2 l3 (l1^2 + ps), 2 l1 l2^2 l3,
>    2 l2^2 l3^2}, {2 l1 l2 (l1^2 + ps),
>    ps (l2^2 + ps) + l1^2 (2 l2^2 + ps), 2 l1 l2 (l2^2 + ps),
>    l1 l3 (l1^2 + l2^2 + ps), l2 l3 (l1^2 + l2^2 + ps),
>    2 l1 l2 l3^2}, {2 l1^2 l2^2, 2 l1 l2 (l2^2 + ps), 2 (l2^2 + ps)^2,
>     2 l1^2 l2 l3, 2 l1 l3 (l2^2 + ps),
>    2 l1^2 l3^2}, {2 l2 l3 (l1^2 + ps), l1 l3 (l1^2 + l2^2 + ps),
>    2 l1^2 l2 l3, l2^2 l3^2 + (l1^2 + ps) (l3^2 + ps),
>    l1 l2 (2 l3^2 + ps), 2 l2 l3 (l3^2 + ps)}, {2 l1 l2^2 l3,
>    l2 l3 (l1^2 + l2^2 + ps), 2 l1 l3 (l2^2 + ps),
>    l1 l2 (2 l3^2 + ps), l1^2 l3^2 + (l2^2 + ps) (l3^2 + ps),
>    2 l1 l3 (l3^2 + ps)}, {2 l2^2 l3^2, 2 l1 l2 l3^2, 2 l1^2 l3^2,
>    2 l2 l3 (l3^2 + ps), 2 l1 l3 (l3^2 + ps), 2 (l3^2 + ps)^2}};
>
> (*brute force approach to benchmark*)
> w = Inverse[mat];
> irls = FindMinimum[e. w  . e, Evaluate[Sequence @@ startval], Gradient
> -> -2 e.   w .j]; // Timing
>
> (*this should work, in fact, it takes almost twice as long*)
> w := Inverse[mat];
> irls = FindMinimum[Evaluate[e. w  . e],  Evaluate[Sequence @@
> startval], Gradient -> Evaluate[-2 e.   w .j]]; // Timing
>
>

Alberto,

try

ewe = e.w.e;
ewj = -2 e.w.j;
irls = FindMinimum[ewe, Evaluate[Sequence @@ startval],
     Gradient -> ewj]; // AbsoluteTiming

which is somewhat faster. Using Method -> "Newton" gets rid of the 
messages but takes longer. Perhaps a Hessian would help. Or use, say, 
Method -> "ConjugateGradient" which is againn faster but has messages. You 
could use ParallelTry with different methods.

Oliver

Back to comp.soft-sys.math.mathematica | Previous | Next — Next in thread | Find similar | Unroll thread


Thread

Re: Findminimum too slow for iterative reweighted least Oliver Ruebenkoenig <ruebenko@wolfram.com> - 2011-12-10 12:37 +0000
  Re: Findminimum too slow for iterative reweighted least "Oleksandr Rasputinov" <oleksandr_rasputinov@hmamail.com> - 2011-12-11 09:01 +0000

csiph-web