3.11 Mixtures of Distributions

3.11  Mixtures of Distributions

Rrandom variable has a mixed distribution if its value will be obtained by randomly drawing from one of the values to be obtained for two or more other random variables. The random variable’s distribution is a mixture of the other random variables’ distributions.

Consider an experiment. You randomly draw two numbers, one from an N(0,4) distribution and the other from an N(0,9) distribution.10 Next, you flip a fair coin. If it comes up “heads”, you set X equal to the number drawn from the N(0,4) distribution. Otherwise, you set X equal to the number drawn from the N(0,9) distribution. The number X that will result from this experiment has a mixed normal distribution with PDF

[3.126]

This is the weighted average of the PDFs of the two normal distributions. More generally, consider m random variables Xk, each with PDF ϕk. Define m weights ξk > 0 that sum to 1. Then the random variable X that has PDF

[3.127]

has a mixed distribution.

3.11.1 Parameters of mixed distributions

Consider a random variable X with a mixed distribution as described above. The Xk have means μk and standard deviations σk. Then X has mean μ and standard deviation σ given by

[3.128]

[3.129]

Calculating a q-quantile of X requires that we solve a nonlinear system of equations, which can be done with Newton’s method. The q-quantile is that value x such that

[3.130]

so we seek probabilities q1, q2, … , qm such that

[3.131]

while

[3.132]

The desired q-quantile x of X then equals any of—all of—these:

[3.133]

These conditions are motivated for the case m = 2 in Exhibit 3.23.

Exhibit 3.23: This exhibit illustrates a mixture of two distributions with a graphical analogy to simple addition. Random variable X at the bottom is a mixture of normal random variables X1 and X2. Weights are ξ1 = 0.4 and ξ2 = 0.6. With the distributions stacked one above the other, you can see how the shapes of the upper two determine the shape of the bottom one. To find a q-quantile x of X, we must find probabilities q1 and q2 such that ξ1q1 + ξ2q2 = q while the q1-quantile x1 of X1 equals the q2-quantile x2 of X2.
3.11.2 Mixed-normal distributions

Since a normal distribution is defined by a mean and standard deviation, a mixed-normal distribution Nm(μ,σ2,ξ) is defined with a vector μ of means, a vector σ2 of variances, and a vector ξ of weights:

[3.134]

where the weights ξk > 0 sum to 1.

Mixed-normal distributions are useful for modeling multimodal or leptokurtic distributions. Exhibit 3.24 illustrates PDFs for two mixed-normal distributions. The first is weighted 0.6 in an N(–1,1) distribution and 0.4 in an N(2,1) distribution to achieve a bimodal distribution. The second is evenly weighted in N(0,1) and N(0,9) distributions to achieve a leptokurtic distribution.

Exhibit 3.24: PDFs for two mixed normal distributions are illustrated. The first is weighted 0.6 in an N(–1,1) distribution and 0.4 in an N(2,1) distribution to achieve a bimodal distribution. The second is evenly weighted in N(0,1) and N(0,9) distributions to achieve a leptokurtic distribution.
3.11.3 Mixed joint-normal distributions

While our discussion of mixed distributions has focused on random variables, similar concepts generalize for random vectors.

Market professionals often observe that market correlations seem exaggerated during large market swings. This phenomenon can be modeled with a mixture of joint-normal distributions—one with low variances and modest correlations and the other with high variances and more extreme correlations.

Consider vectors of n-dimensional mean vectors, n × n covariance matrices, and scalar weights:

[3.135]

where the weights ξk > 0 sum to 1. These define a mixed joint-normal distribution  with PDF

[3.136]

where ϕk(x) ~ Nn(μk,Σk).

Exercises
3.39

Derive formulas [3.128] and [3.129].
Solution

3.40

Consider random variable X ~ N 2(μ,σ2,ξ), where:

[3.137]

Calculate the mean, standard deviation, and .25-quantile of X.
Solution