Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #15821 > unrolled thread

Filtering data from numerical minimization

Started bySebastian <sebhofer@gmail.com>
First post2011-02-04 06:42 +0000
Last post2011-02-04 11:58 +0000
Articles 2 — 2 participants

Back to article view | Back to comp.soft-sys.math.mathematica


Contents

  Filtering data from numerical minimization Sebastian <sebhofer@gmail.com> - 2011-02-04 06:42 +0000
    Re: Filtering data from numerical minimization Ray Koopman <koopman@sfu.ca> - 2011-02-04 11:58 +0000

#15821 — Filtering data from numerical minimization

FromSebastian <sebhofer@gmail.com>
Date2011-02-04 06:42 +0000
SubjectFiltering data from numerical minimization
Message-ID<iig75a$rkb$1@smc.vnet.net>
I have a function f(n,x,y,z) that I want to numerically minimize with
respect to x,y,z for a list of n=1...N. The function is pretty
complicated and also contains other parameters, which are fixed as far
as optimization is concerned. I use NMinimize to create a table for
different values of n. This effectively leaves me with 4 lists (f_min,
x_min,...), where f_min = f(x_min, y_min, z_min). I then plot these
lists against the vector n.

Depending on the other parameters the minimization sometimes works
well, and sometimes it doesn't. The problem is the following: while
the overall form of the curves is often easily recognizable by eye,
there is some noise on top of it, i.e. some of the points (every 5th
say) are just way off. As I don't think that f behaves that way, but
rather NMinimize fails to find the correct value, I'd like to filter
that noise. Filtering the points by hand, however, doesn't seem like a
feasible solution. Does anyone know a good way to tackle this problem?
I already tried running MovingMedian which works reasonably well but
I'd like to keep the rest of the data untouched if possible.

I'm happy for all suggestions!
TIA, Sebastian

[toc] | [next] | [standalone]


#15982

FromRay Koopman <koopman@sfu.ca>
Date2011-02-04 11:58 +0000
Message-ID<iigpl3$21k$1@smc.vnet.net>
In reply to#15821
On Feb 3, 10:42 pm, Sebastian <sebho...@gmail.com> wrote:
> I have a function f(n,x,y,z) that I want to numerically minimize with
> respect to x,y,z for a list of n=1...N. The function is pretty
> complicated and also contains other parameters, which are fixed as far
> as optimization is concerned. I use NMinimize to create a table for
> different values of n. This effectively leaves me with 4 lists (f_min,
> x_min,...), where f_min = f(x_min, y_min, z_min). I then plot these
> lists against the vector n.
>
> Depending on the other parameters the minimization sometimes works
> well, and sometimes it doesn't. The problem is the following: while
> the overall form of the curves is often easily recognizable by eye,
> there is some noise on top of it, i.e. some of the points (every 5th
> say) are just way off. As I don't think that f behaves that way, but
> rather NMinimize fails to find the correct value, I'd like to filter
> that noise. Filtering the points by hand, however, doesn't seem like a
> feasible solution. Does anyone know a good way to tackle this problem?
> I already tried running MovingMedian which works reasonably well but
> I'd like to keep the rest of the data untouched if possible.
>
> I'm happy for all suggestions!
> TIA, Sebastian

Try Tukey's "4253h,twice" -- t4253h below. I've wrapped the multiline
functions in parentheses so that I wouldn't have to worry about where
the CRs come.

med42[y_] := ( (* running medians of 4, recentered by
                  running medians of 2; endpoints copied *)
Join[{y[[1]]},
     ListCorrelate[{.5,.5}, Join[{{.5,.5}.Take[y,2]},
                                 Median/@Partition[y,4,1],
                                 {{.5,.5}.Take[y,-2]}]],
     {y[[-1]]}] );

med5[y_] := ( (* running medians of 5, endpoints copied *)
Join[{y[[1]], Median@Take[y,3]},
     Median/@Partition[y,5,1],
     {Median@Take[y,-3], y[[-1]]}] );

med3e[y_] := ( (* running medians of 3, endpoints adjusted *)
Join[{Median@{{3.,-2.}.Take[#, 2], y[[ 1]], #[[ 1]]}}, #,
     {Median@{{-2.,3.}.Take[#,-2], y[[-1]], #[[-1]]}}]& [
Median /@ Partition[y,3,1] ] );

hann[y_] := ( (* 1:2:1-weighted running means; endpoints copied *)
Join[{y[[1]]}, ListCorrelate[{.25,.5,.25},y], {y[[-1]]}] );

s4253h[y_] := hann @ med3e @ med5 @ med42 @ y;

t4253h[y_] := # + s4253h[y - #] & [s4253h @ y];

y = {60,70,54,56,70,66,53,95,70,69,56,70,70,60,60,
     60,50,50,48,59,50,60,70,54,46,57,57,51,51,59};

t4253h[y]

{60., 60.3594, 60.5781, 60.9375, 62.2109, 65.0078,
 67.8438, 69.2813, 69.8438, 70.0508, 69.5313, 67.8711,
 65.5625, 63.0078, 59.9336, 55.793, 51.8281, 50.2344,
 50.2031, 52.0352, 55.6055, 57.375, 57.1328, 56.4063,
 55.4141, 54.5781, 54.1875, 54.2148, 54.5742, 55.2031}

[toc] | [prev] | [standalone]


Back to top | Article view | comp.soft-sys.math.mathematica


csiph-web