Fast search local extremum for maximal information coefficient (MIC)

摘要

Maximal information coefficient (MIC) is an indicator to explore the correlation between pairwise variables in large data sets, and the accuracy of MIC has an impact on the measure of dependence for each pair. To improve the equitability in an acceptable run-time, in this paper, an intelligent MIC (iMIC) is proposed for optimizing the partition on the y-axis to approximate the MIC with good accuracy. It is an iterative algorithm on quadratic optimization to generate a better characteristic matrix. During the process, the iMIC can quickly find out the local optimal value while using a lower number of iterations. It produces results that are close to the true MIC values by searching just n times, rather than n2 computations required for the previous method. In the compared experiments of 169 indexes about 202 countries from World Health Organization (WHO) data set, the proposed algorithm offers a better solution coupled with a reasonable run-time for MIC, and good performance search for the extreme values in fewer iterations. The iMIC develops the equitability keeping the satisfied accuracy with fast computational speed, potentially benefitting the relationship exploration in big data.