Expectation-Maximization (EM) for Finite Mixture Parameters Estimation

EM is a method for optimizing likelihood functions. It is good for situation where data might be missing or when other optimization methods fail.

EM needs some initial guessing for its operations. In this case, we must tell it the number of terms c in the mixture. Plus, we must also guess the initial parameters for each mixture component.

For example, consider this artificial two-term mixture data centered (means) at (-2,2) and (2,0) The covariance of each component is identity (var=1).

EM method consists of two steps. The Expectation step find posterior probability using the data points given the guess parameters of the mixtures (mixing coefficient, means and variances). The Maximization step finds the update mixing coefficient, new means and new variances. These two steps will be iterated until the maximum of errors of any three of them falls below a tolerant limit.

The initial parameters are:

c=2;

[n,d]=size(data); % n=# points, d=# dimension

tol=0.00001;      % each column represents a mean

max_it=100;

%initial guess

mu(:,1)=[-1 -1]';

mu(:,2)=[1 1]';

mix_cof = [0.2 0.8];

var_mat(:,:,1)=eye(d);

var_mat(:,:,2)=eye(d);

The MATLAB code for this task is quite long. Please refer to Example 9.11 of Martinez & Martinez 2008

The final output is:

converge after 39 iterations

p1 = 0.517600, p2 =0.482400

u=

   -2.1795    1.9905

    1.9790   -0.0184

var=

(:,:,1) =

    1.2498   -0.0522

   -0.0522    0.6819

(:,:,2) =

    0.7199   -0.0055

   -0.0055    1.0255