Tài liệu Independent component analysis P10 doc

10

ICA by Minimization of

Mutual Information

An important approach for independent component analysis (ICA) estimation, in￾spired by information theory, is minimization of mutual information.

The motivation of this approach is that it may not be very realistic in many cases

to assume that the data follows the ICA model. Therefore, we would like to develop

an approach that does not assume anything about the data. What we want to have

is a general-purpose measure of the dependence of the components of a random

vector. Using such a measure, we could define ICA as a linear decomposition that

minimizes that dependence measure. Such an approach can be developed using

mutual information, which is a well-motivated information-theoretic measure of

statistical dependence.

One of the main utilities of mutual information is that it serves as a unifying

framework for many estimation principles, in particular maximum likelihood (ML)

estimation and maximization of nongaussianity. In particular, this approach gives a

rigorous justification for the heuristic principle of nongaussianity.

10.1 DEFINING ICA BY MUTUAL INFORMATION

10.1.1 Information-theoretic concepts

The information-theoretic concepts needed in this chapter were explained in Chap￾ter 5. Readers not familiar with information theory are advised to read that chapter

before this one.

221

Independent Component Analysis. Aapo Hyvarinen, Juha Karhunen, Erkki Oja ¨

Copyright  2001 John Wiley & Sons, Inc.

ISBNs: 0-471-40540-X (Hardback); 0-471-22131-7 (Electronic)