Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #6094 > unrolled thread

Problems with DistributionFitTest

Started byfd <fdimer@gmail.com>
First post2011-11-01 05:07 +0000
Last post2011-11-01 11:58 +0000
Articles 2 — 2 participants

Back to article view | Back to comp.soft-sys.math.mathematica


Contents

  Problems with DistributionFitTest fd <fdimer@gmail.com> - 2011-11-01 05:07 +0000
    Re: Problems with DistributionFitTest Ray Koopman <koopman@sfu.ca> - 2011-11-01 11:58 +0000

#6094 — Problems with DistributionFitTest

Fromfd <fdimer@gmail.com>
Date2011-11-01 05:07 +0000
SubjectProblems with DistributionFitTest
Message-ID<j8nuqm$egv$1@smc.vnet.net>
Dear Group

I'm not a specialist in statistics, but I spoke to one who found this
behaviour dubious.

Before using DistributionFitTest I was doing some tests with the
normal distribution, like this

data = RandomVariate[NormalDistribution[], 10000];

DistributionFitTest[data]

0.0312946

According to the documentation "A small p-value suggests that it is
unlikely that the data came from dist", and that the test assumes the
data is normally distributed

I found this result for the p-value to be really low, if I re-run the
code I often get what I would expect (a number greater than 0.5) but
it is not at all rare to obtain p values smaller than 0.05 and even
smaller. Through multiple re-runs I notice it fluctuates by orders of
magnitude.

The statistician I consulted with found this weird since the data was
drawn from a a normal distribution and the sample size is big,
especially because the Pearson X2 test also fluctuates like this:

H=DistributionFitTest[data, Automatic, "HypothesisTestData"];

H["TestDataTable", All]

Is this a real issue?

Any thougths

Best regards
Felipe



[toc] | [next] | [standalone]


#6102

FromRay Koopman <koopman@sfu.ca>
Date2011-11-01 11:58 +0000
Message-ID<j8omtr$lf2$1@smc.vnet.net>
In reply to#6094
On Oct 31, 10:07 pm, fd <fdi...@gmail.com> wrote:
> Dear Group
>
> I'm not a specialist in statistics, but I spoke to one who found this
> behaviour dubious.
>
> Before using DistributionFitTest I was doing some tests with the
> normal distribution, like this
>
> data = RandomVariate[NormalDistribution[], 10000];
>
> DistributionFitTest[data]
>
> 0.0312946
>
> According to the documentation "A small p-value suggests that it is
> unlikely that the data came from dist", and that the test assumes the
> data is normally distributed
>
> I found this result for the p-value to be really low, if I re-run the
> code I often get what I would expect (a number greater than 0.5) but
> it is not at all rare to obtain p values smaller than 0.05 and even
> smaller. Through multiple re-runs I notice it fluctuates by orders of
> magnitude.
>
> The statistician I consulted with found this weird since the data was
> drawn from a a normal distribution and the sample size is big,
> especially because the Pearson X2 test also fluctuates like this:
>
> H=DistributionFitTest[data, Automatic, "HypothesisTestData"];
>
> H["TestDataTable", All]
>
> Is this a real issue?
>
> Any thougths
>
> Best regards
> Felipe

If the data were generated by the distribution for which you are
testing then, no matter what the sample size is, the p-value is a
sample from a Uniform[0,1] distribution, so 5% of the time it should
be < .05, etc.

[toc] | [prev] | [standalone]


Back to top | Article view | comp.soft-sys.math.mathematica


csiph-web