Inference from disorder


I don’t know if this is a real category, but between conversations with Jonas Peters, Aurora Delaigle and Zdravko Botev, I’ve seen a few references to the idea that we can draw inference from the lack of structure, in some sense, of the world.

Janzing and Peters and so forth do this with inferring the arrow of time or causality. Delaigle and Hall do very blind statistical deconvolution. I’m sure other uses could be made of the idea. Perhaps the Chatterjee correlation does this also? Azadkia and Chatterjee (2019); Chatterjee (2020)

Connection: algorithmic statistics, independence.



Azadkia, Mona, and Sourav Chatterjee. 2019. “A Simple Measure of Conditional Dependence.” December 20, 2019.
Chatterjee, Sourav. 2020. “A New Coefficient of Correlation.” January 18, 2020.
Delaigle, Aurore, and Peter Hall. 2015. “Methodology for Non-Parametric Deconvolution When the Error Distribution Is Unknown.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), February, n/a–.
Hoyer, Patrik O., Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. 2009. “Nonlinear Causal Discovery with Additive Noise Models.” In Advances in Neural Information Processing Systems 21, edited by D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, 689–96. Curran Associates, Inc.
Janzing, Dominik. 2007. “On Causally Asymmetric Versions of Occam’s Razor and Their Relation to Thermodynamics.” August 24, 2007.
Janzing, Dominik, Joris Mooij, Kun Zhang, Jan Lemeire, Jakob Zscheischler, Povilas Daniušis, Bastian Steudel, and Bernhard Schölkopf. 2012. “Information-Geometric Approach to Inferring Causal Directions.” Artificial Intelligence 182-183 (May): 1–31.
Janzing, Dominik, and Bernhard Schölkopf. 2010. “Causal Inference Using the Algorithmic Markov Condition.” IEEE Transactions on Information Theory 56 (10): 5168–94.
Janzing, Dominik, Xiaohai Sun, and Bernhard Schoelkopf. 2009. “Distinguishing Cause and Effect via Second Order Exponential Models.” October 29, 2009.
Mooij, Joris M., Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bernhard Schölkopf. 2014. “Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks.” December 11, 2014.
———. 2016. “Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks.” Journal of Machine Learning Research 17 (32): 1–102.
Peters, Jonas, Dominik Janzing, Arthur Gretton, and Bernhard Schölkopf. 2009. “Detecting the Direction of Causal Time Series.” In Proceedings of the 26th Annual International Conference on Machine Learning, 801–8. ICML ’09. New York, NY, USA: ACM.
Peters, Jonas, Joris M. Mooij, Dominik Janzing, and Bernhard Schölkopf. 2014. “Causal Discovery with Continuous Additive Noise Models.” The Journal of Machine Learning Research 15 (1): 2009–53.
Reiersol, Olav. 1950. “Identifiability of a Linear Relation Between Variables Which Are Subject to Error.” Econometrica 18 (4): 375.
Schölkopf, Bernhard, Bernhard, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. 2012. “On Causal and Anticausal Learning.” In ICML 2012.
Zhang, Kun, Jiji Zhang, and Bernhard Schölkopf. 2015. “Distinguishing Cause from Effect Based on Exogeneity.” April 22, 2015.

Warning! Experimental comments system! If is does not work for you, let me know via the contact form.

No comments yet!

GitHub-flavored Markdown & a sane subset of HTML is supported.