”
625.664留学生作业 写作、 辅导Python语言作业、 写作Java，c/c++编程设计作业
625.664 Computational Statistics
Problem Set 3
Associated Reading: Chapter 2: 2.2.2-2.2.2.2
Chapter 4: Introduction – Section 4.2
Complete the problems either by hand or using the computer and upload your final document
to the Blackboard course site. All final submittals are to be in PDF form. Please document any
code used to solve the problems and include it with your submission.
1. Problem 2.5. Only do parts (a), (b), (c), and (e).
2. Consider the following mixture of two normal densities:
p(x; ) = (x; 1, 21) + (1 )(x; 2, 22)
where = (, 1, 2, 21, 22) is the collection of all the parameters driving the model, with
denoting the the mixing proportion (and not the usual number), and is the standard
density for a normal random variable. Direct use of maximum likelihood is made difficult
here because of the expression of the likelihood function. So, we wish to simplify this
problem by use of the EM algorithm.
Let Zi be the random variable that denotes the class membership of of an observed xi
in
a given sample. Clearly, zi
is the missing data, with Zi {0, 1} and Zi Bernoulli().
The complete-data density for Y = (X, Z) is given by
p(xi, zi|) = [(x; 1, 21)]zi
[(1 )(x; 2, 22)](1zi)
(a) Show that the complete-data log-likelihood function is
l(; x, z) = log()ni=1zi +
ni=1zilog((xi; 1, 21)) + log(1 )(n ni=1zi)+
ni=1(1 zi) log((xi; 2, 22)).
For the Expectation step of the EM algorithm, we need Q(|
(k)) = E[l(; y)|x, (k)].
(b) Find Q(|
(k)) in terms of E[Zi|xi, (k)], again it is best to leave your answer as the
sum of four main terms. Also, if is fine to leave your answer in terms of the function
(xi; (k)1,(21)(k)).
(c) Show that, given
(k) and xi,E[Zi|xi, (k)] = P[Zi = 1|xi, (k)] =
(k)(xi; (k)1,(21)(k))(k)(xi; (k)1,(21)(k)) + (1 (k))(xi; (k)2,(22)(k))
For the Maximization step of the EM algorithm, we need to find the new set of parameters
(k+1) that maximizes Q(|(k)). This can be done in a straightforward manner by solving
Q(|(k)) = 0
for each of the parameters in = (, 1, 21, 2, 22). To simplify your notation let
(k)i =E[Zi|xi, (k)] and
(k) =ni=1 (k)i.
(d) First show that
(e) Now, use the fact that
(f) Suppose that you have observed the sample x1, x2, . . . , xn from the above distribution.
Write the psuedocode required to estimate using the EM algorithm. Be sure to include
some form of convergence criteria.
“
添加老师微信回复‘’官网 辅导‘’获取专业老师帮助，或点击联系老师1对1在线指导。
