Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #5873 > unrolled thread

Re: Finding distribution parameters for a mixture of product distributions

Started byDarren Glosemeyer <darreng@wolfram.com>
First post2011-10-26 21:39 +0000
Last post2011-10-26 21:39 +0000
Articles 1 — 1 participant

Back to article view | Back to comp.soft-sys.math.mathematica


Contents

  Re: Finding distribution parameters for a mixture of product distributions Darren Glosemeyer <darreng@wolfram.com> - 2011-10-26 21:39 +0000

#5873 — Re: Finding distribution parameters for a mixture of product distributions

FromDarren Glosemeyer <darreng@wolfram.com>
Date2011-10-26 21:39 +0000
SubjectRe: Finding distribution parameters for a mixture of product distributions
Message-ID<j89ulo$1n$1@smc.vnet.net>
On 10/22/2011 5:06 AM, Koslicki wrote:
> I am trying to estimate parameters for a mixture of product
> distributions, and Mathematica does not like my inclusion of product
> distributions.
>
> Ignoring the stupidity of the following choice of distributions,
> consider the following:
>
> In[142]:= data=RandomVariate[MixtureDistribution[{.1,.9},
> {BernoulliDistribution[0],BernoulliDistribution[1]}],100];
>
> In[140]:= dist1=MixtureDistribution[{p,1-p},
> {BernoulliDistribution[a1],BernoulliDistribution[a2]}];
>
> In[141]:= FindDistributionParameters[data,dist1]
>
> Out[141]= {p->0.261107,a1->0.919502,a2->0.960777}
>
> So everything works just fine.
>
>
> Yet if I want to look at a *product* of Bernoulli distribution
> instead, I get the following:
>
> In[143]:= data=RandomVariate[MixtureDistribution[{.1,.9},
> {ProductDistribution[BernoulliDistribution[0],BernoulliDistribution[0]],ProductDistribution[BernoulliDistribution[1],BernoulliDistribution[1]]}],
> 100];
>
> In[144]:= dist1=MixtureDistribution[{p,1-p},
> {ProductDistribution[BernoulliDistribution[a1],BernoulliDistribution[a2]],ProductDistribution[BernoulliDistribution[a3],BernoulliDistribution[a4]]}];
>
> In[145]:= FindDistributionParameters[data,dist1]
>
> During evaluation of In[145]:= FindDistributionParameters::ntsprt: One
> or more data points are not in support of the distribution ...
>
>
> How can it be that one of the data points is not in the support of the
> distribution when I chose the data points from the distribution
> itself?
>
> This problem seems to arise whenever I am trying to find parameters
> for a mixture of product distributions...
>
> Any thoughts?
>
> Thanks,
>
> ~David
>

The estimation code uses internal utility functions to weed out bad 
combinations of distributions and data before calling off to 
optimization routines. That code needs to be improved to handle this 
particular case, and I will look into that for a future release. It 
seems to work fine for mixtures of products of betas, so I suspect it 
may be specifically mixtures of products of discrete distributions that 
are problematic. I would be interested to see what other examples you've 
run into trouble with so I can use those as test cases as well.

While I do not recommend doing the following in general, a possible way 
around this for now is to add a definition for the internal validation 
function so this case will no longer get kicked out as being invalid.

In[1]:= data = RandomVariate[MixtureDistribution[{.1, .9}, 
{ProductDistribution[BernoulliDistribution[0], BernoulliDistribution[0]],
      ProductDistribution[BernoulliDistribution[1], 
BernoulliDistribution[1]]}], 100];

In[2]:= dist1 = MixtureDistribution[{p, 1 - p}, 
{ProductDistribution[BernoulliDistribution[a1], BernoulliDistribution[a2]],
     ProductDistribution[BernoulliDistribution[a3], 
BernoulliDistribution[a4]]}];


The case we will add is for mixtures of products of discrete 
distributions. The check will be weaker than it should be (we will only 
check that the data points are lists of integers of the right length), 
so you will need to make sure that all the integers you feed in for 
these cases are in range.


In[3]:= Unprotect[MixtureDistribution];

In[4]:= MixtureDistribution /: 
Statistics`Library`AcceptableDistributedDataQ[MixtureDistribution[_?VectorQ, 

     distlist : 
{ProductDistribution[__?Statistics`Library`DiscreteUnivariateDistributionQ] 
..}]?
    DistributionParameterQ, data_?(MatrixQ[#, IntegerQ] &)] := True /; 
Dimensions[data][[2]] === Length[distlist[[1]]]

In[5]:= Protect[MixtureDistribution];

In[6]:= FindDistributionParameters[data, dist1]

Out[6]= {p -> 0.909997, a1 -> 1., a2 -> 1., a3 -> 5.31528*10^-8, a4 -> 
5.31528*10^-8}


Darren Glosemeyer
Wolfram Research

[toc] | [standalone]


Back to top | Article view | comp.soft-sys.math.mathematica


csiph-web