Maximal Information Coefficient – just a messed-up estimate of mutual information?

Theory papers almost never make it into top journals and this is why I have blogged about the paper ‘Detecting Novel Associations in Large Data Sets’ in Science by Reshef et al before (here and here). The reception in the statistics community was mixed and while Terry Speed seemed to love it, Rob Tibshirani started to point out weaknesses. And now other people have joined the discussion.

Justin B Kinney, a physicist who has a quantitative biology lab at CSHL, writes:

Mickey Atwal and I recently posted a preprint that challenges the primary claims of this [Reshef’s] article.

In essence we find that Reshef et al.’s claim that MIC has an “equitability” property while mutual information does not is incorrect. We provide mathematical proof of this, and also validate our findings by rerunning Reshef et al.’s simulations.

Thus, there appears to be no motivation for using MIC rather than simply estimating mutual information.

Frankly, as far as we can tell, MIC is just a messed up estimate of mutual information.

If you are interested, Justin Kinney’s preprint and more material by Reshef et al are available on arXiv:

• Justin B. Kinney, Gurinder S. Atwal, Equitability, mutual information, and the maximal information coefficient, http://arxiv.org/abs/1301.7745
• Reshef et al Equitability Analysis of the Maximal Information Coefficient, with Comparisons, http://arxiv.org/abs/1301.6314.

In a discussion on Andrew Gelman’s blog Michael Mitzenmacher, one of Reshef et al’s senior authors, seems to try to back out of the discussion:

We discuss the notion of equitability as a desirable heuristic property, as underscored by our use of words like “roughly equal” and “similar” instead of “equal” when discussing it. Philosophically, we have been using equitability as an approximate property (..)

In my own work I am so pragmatic, I just use whatever works and does the job. But as soon as it comes to mathematical statements something like “roughly equal”, “desirable heuristic property”, and “approximate property” seem to be a way to weasel out of discussion… I’m not impressed at all by Mitzenmacher’s response.

Florian