3.11 Mixtures of Distributions

Rrandom variable has a mixed distribution if its value will be obtained by randomly drawing from one of the values to be obtained for two or more other random variables. The random variable’s distribution is a mixture of the other random variables’ distributions.

Consider an experiment. You randomly draw two numbers, one from an N(0,4) distribution and the other from an N(0,9) distribution.10 Next, you flip a fair coin. If it comes up “heads”, you set X equal to the number drawn from the N(0,4) distribution. Otherwise, you set X equal to the number drawn from the N(0,9) distribution. The number X that will result from this experiment has a mixed normal distribution with PDF

[3.126]

This is the weighted average of the PDFs of the two normal distributions. More generally, consider m random variables X_k, each with PDF ϕ_k. Define m weights ξ_k > 0 that sum to 1. Then the random variable X that has PDF

[3.127]

has a mixed distribution.

3.11.1 Parameters of mixed distributions

Consider a random variable X with a mixed distribution as described above. The X_k have means μ_k and standard deviations σ_k. Then X has mean μ and standard deviation σ given by

[3.128]

[3.129]

Calculating a q-quantile of X requires that we solve a nonlinear system of equations, which can be done with Newton’s method. The q-quantile is that value x such that

[3.130]

so we seek probabilities q₁, q₂, … , q_m such that

[3.131]

while

[3.132]

The desired q-quantile x of X then equals any of—all of—these:

[3.133]

These conditions are motivated for the case m = 2 in Exhibit 3.23.

Exhibit 3.23: This exhibit illustrates a mixture of two distributions with a graphical analogy to simple addition. Random variable X at the bottom is a mixture of normal random variables X₁ and X₂. Weights are ξ₁ = 0.4 and ξ₂ = 0.6. With the distributions stacked one above the other, you can see how the shapes of the upper two determine the shape of the bottom one. To find a q-quantile x of X, we must find probabilities q₁ and q₂ such that ξ₁q₁ + ξ₂q₂ = q while the q₁-quantile x₁ of X₁ equals the q₂-quantile x₂ of X₂.

3.11.2 Mixed-normal distributions

Since a normal distribution is defined by a mean and standard deviation, a mixed-normal distribution N^m(μ,σ²,ξ) is defined with a vector μ of means, a vector σ² of variances, and a vector ξ of weights:

[3.134]

where the weights ξ_k > 0 sum to 1.

Mixed-normal distributions are useful for modeling multimodal or leptokurtic distributions. Exhibit 3.24 illustrates PDFs for two mixed-normal distributions. The first is weighted 0.6 in an N(–1,1) distribution and 0.4 in an N(2,1) distribution to achieve a bimodal distribution. The second is evenly weighted in N(0,1) and N(0,9) distributions to achieve a leptokurtic distribution.

Exhibit 3.24: PDFs for two mixed normal distributions are illustrated. The first is weighted 0.6 in an N(–1,1) distribution and 0.4 in an N(2,1) distribution to achieve a bimodal distribution. The second is evenly weighted in N(0,1) and N(0,9) distributions to achieve a leptokurtic distribution.

3.11.3 Mixed joint-normal distributions

While our discussion of mixed distributions has focused on random variables, similar concepts generalize for random vectors.

Market professionals often observe that market correlations seem exaggerated during large market swings. This phenomenon can be modeled with a mixture of joint-normal distributions—one with low variances and modest correlations and the other with high variances and more extreme correlations.

Consider vectors of n-dimensional mean vectors, n × n covariance matrices, and scalar weights:

[3.135]