Determining stochastic dependence for normally distributed vectors using the chi-squared metric

摘要

A fundamental problem in information theory and pattern recognition involves computing and estimating the probability density function associated with a set of random variables. In estimating this density function, one can either assume that the form of the density function is known, and that we are merely estimating parameters that characterize the distribution or that no information about the density function is available. This problem has been extensively studied if the random variables are independent. If the random variables are dependent and are of the discrete sort, the problem of capturing this dependence between variables has been studied in Chow and Liu (IEEE Trans. Inf. Theory14, 462–467 (May 1968)). The analogous problem for normally distributed continuous random variables has been tackled in Chow et al. (Comput. Biomed. Res.12, 589–613 (1979)). In both these instances, the determination of the best dependence tree hinges on the well-known Expected Mutual Information Measure (EMIM) Metric. Recently Valiveti and Oommen studied the suitability of the chi-squared based metric in-lieu of the EMIM metric, for the discrete variable case (Pattern Recognition25, 1389–1400 (1992)). In this paper, we generalize the latter result and study the use of the chi-squared metric for determining dependence trees for normally distributed random vectors. We show that for such vectors, the chi-squared metric yields the optimal tree and that it is identical to the one obtained using the EMIM metric. The computation of the maximum likelihood estimate of the dependence tree is also discussed.