Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #8706
| From | Ray Koopman <koopman@sfu.ca> |
|---|---|
| Newsgroups | comp.soft-sys.math.mathematica |
| Subject | Re: Kolmogorov Smirnov in two or more dimensions is in Mathematica 8.0.4 |
| Date | 2012-02-23 10:48 +0000 |
| Organization | Steven M. Christensen and Associates, Inc and MathTensor, Inc. |
| Message-ID | <ji55hg$77q$1@smc.vnet.net> (permalink) |
| References | <ji2g4f$k8d$1@smc.vnet.net> |
On Feb 22, 2:30 am, maria giovanna dainotti
<mariagiovannadaino...@yahoo.it> wrote:
> Dear Andy Ross,
> thanks for your suggestion. I applied the method
> DistributionFitTest[data1, data2,
> {"TestDataTable","SzekelyEnergy"},Method->{"MonteCarlo","MonteCarloSamples"->1000}]
> and it works, but when I try to minimize the function I have a problem.
>
> For[j = 1, j < n + 1,
> fn = Function[{s, m},
> x = Table[{Data1[[j, 1]] + s, Data1[[j, 2]] + m}, {j, 1, size}];
> -2*Log[
> DistributionFitTest[x, Data0, {"TestDataTable", "SzekelyEnergy"},
> Method -> {"MonteCarlo", "MonteCarloSamples" -> 1000}][[1, 1,
> 2, 3]]]
> ]]
>
> With Data1
> {{0.894606, 14.0865}, {0.811846, 13.7631}, {1.00287,
> 14.2577}, {0.798426, 13.8605}, {0.765556, 14.0297}, {1.06602,
> 14.144}, {0.965456, 14.2174}, {0.879326, 14.6311}, {0.831876,
> 14.006}, {0.695166, 14.2763}, {1.05657, 14.2558}, {1.05519,
> 13.7674}, {0.811626, 14.1137}, {0.910116, 13.9954}, {0.933376,
> 13.9232}, {0.870476, 14.2965}, {0.713326, 14.7321}, {1.13929,
> 13.9139}, {1.1878, 14.0376}, {0.998686, 13.9364}, {1.10588,
> 14.0096}, {1.06213, 14.2319}, {0.797846, 14.9199}, {0.768916,
> 14.5909}, {0.944476, 14.8957}, {1.09493, 14.6853}, {1.03171,
> 14.5485}, {0.838416, 15.1119}, {1.03875, 14.3372}, {0.695036,
> 15.0422}, {0.860336, 14.9023}, {0.903736, 15.1527}, {0.798196,
> 14.7923}, {0.695146, 15.5481}, {1.08506, 15.2012}, {1.07127,
> 15.2615}, {0.919576, 15.5636}, {1.01526, 15.4415}, {1.15206,
> 15.0784}, {0.844826, 15.6197}, {0.897866, 15.887}, {0.762816,
> 15.7318}, {0.867886, 15.743}, {0.991576, 15.7403}, {0.898346,
> 15.6946}, {0.996826, 15.6467}, {1.0194, 15.6681}, {0.830686,
> 15.5817}, {0.982806, 15.6863}, {0.883306, 16.3492}, {0.928786,
> 15.9975}, {0.932786, 15.9091}, {1.08748, 15.8808}, {0.963906,
> 16.4309}, {0.896236, 16.7413}, {0.750366, 16.7619}, {0.945936,
> 16.6388}, {1.064, 16.7059}, {0.947046, 17.0593}, {1.0258,
> 16.3197}, {0.973976, 16.9003}, {0.829206, 17.183}, {0.917956,
> 17.0182}, {0.954016, 16.8625}, {0.842906, 17.2803}, {1.10261,
> 17.0396}, {1.02155, 17.3819}, {0.671206, 17.5726}, {1.04596,
> 17.076}, {1.08985, 17.3047}, {1.05809, 17.2071}, {0.916836,
> 17.6237}, {0.815676, 17.5559}, {0.943866, 18.1943}, {1.02348,
> 16.8333}, {1.00346, 16.9493}, {1.15636, 17.0909}, {1.11935,
> 17.5003}}
>
> and Data0
>
> {{0.97304, 14.1829}, {0.98663, 14.1295}, {0.98284, 14.3172}, {0.81423,
> 14.3466}, {0.97303, 14.5966}, {0.87122, 14.8435}, {0.90252,
> 14.9036}, {0.81887, 15.1177}, {1.07722, 14.6849}, {0.86684,
> 15.6456}, {0.86664, 14.7034}, {0.78728, 15.0898}, {1.10336,
> 14.4085}, {1.12014, 14.7281}, {0.95923, 14.4988}, {0.89942,
> 15.3173}, {0.83841, 14.8422}, {0.99105, 14.8813}, {1.111,
> 14.5964}, {0.93255, 15.5019}, {1.03142, 14.8009}, {1.00661,
> 15.0827}, {0.93255, 15.3064}, {1.10023, 14.7189}, {0.8797,
> 15.1038}, {1.0013, 14.6755}, {0.87673, 15.0952}, {0.84131,
> 15.7345}, {1.06392, 15.3528}, {1.00138, 14.9835}, {0.77803,
> 15.4637}, {0.76795, 15.3611}, {0.98328, 15.1047}, {0.89193,
> 14.8321}, {0.72882, 15.909}, {0.77123, 15.7902}, {0.86218,
> 15.6637}, {0.84381, 15.536}, {0.99263, 15.8903}, {1.0805,
> 15.1453}, {0.85316, 15.7793}, {0.85186, 15.9119}, {1.10898,
> 15.7583}, {1.03365, 15.7393}, {0.84783, 16.1911}, {1.10979,
> 16.0031}, {1.05238, 16.015}, {0.90259, 16.4864}, {0.84963,
> 15.9818}, {1.09221, 16.0088}, {1.0443, 15.8326}, {0.8945,
> 16.1927}, {0.83015, 16.2776}, {0.7551, 16.5538}, {1.05947,
> 15.9138}, {1.06189, 15.6061}, {1.05889, 16.0743}, {0.85216,
> 16.1568}, {0.72597, 16.7657}, {0.93638, 16.3583}, {0.81968,
> 16.2711}, {0.95022, 16.3549}, {1.04536, 16.18}, {1.0786,
> 16.0967}, {0.9385, 15.8504}, {0.95024, 15.9338}, {0.76753,
> 17.1461}, {1.18224, 16.0437}, {0.96447, 16.4908}, {0.98235,
> 16.0892}, {1.06151, 16.699}, {0.79052, 16.5207}, {1.15863,
> 16.3223}, {1.00795, 16.2444}, {1.07284, 16.6536}, {1.04796,
> 16.865}, {0.84226, 16.7247}, {1.04712, 16.2673}}
>
> [...]
Here is an implementation of the Szekely-Energy comparison of your
two data sets. It saves some time by taking advantage of the fact
that your two sample sizes are equal. Perhaps more importantly, it
uses a centered orthonormal basis for the space of the pooled data.
This gives both variables equal opportunity to influence the test
statistic. If this is not done, the test statistic tends to ignore
variables with smaller variances.
Based on 10^6 samples, the approximate p-value is .0587 .
n1 = Length[data1 = {{0.894606, 14.0865}, ..., {1.11935, 17.5003}}]
n0 = Length[data0 = {{0.97304, 14.1829}, ..., {1.04712, 16.2673}}]
n = Length[data = Join[data1, data0]]
78
78
156
zz = Transpose@Orthogonalize[Transpose@data - Mean@data];
d = Outer[Norm[#1-#2]&, zz, zz, 1];
a = Total[Take[d, n1, -n0], 2]
883.529
reps = 1*^6
Timing[
N@Total @ UnitStep[ Table[
s = RandomSample[r = Range@n, n1]; r[[s]] = 0;
t = SparseArray[r] /. SparseArray[_,_,_,x_] :> x[[3]];
Total[d[[s,t]], 2], {reps}] - a] / reps ]
1000000
{3901.83, 0.058742}
Back to comp.soft-sys.math.mathematica | Previous | Next — Previous in thread | Find similar | Unroll thread
Re: Kolmogorov Smirnov in two or more dimensions is in Mathematica 8.0.4 maria giovanna dainotti <mariagiovannadainotti@yahoo.it> - 2012-02-22 10:30 +0000 Re: Kolmogorov Smirnov in two or more dimensions is in Mathematica 8.0.4 Ray Koopman <koopman@sfu.ca> - 2012-02-23 10:48 +0000
csiph-web