graphical_models on Dan MacKinlay
https://danmackinlay.name/tags/graphical_models.html
Recent content in graphical_models on Dan MacKinlayHugo -- gohugo.ioen-usMon, 15 Mar 2021 16:44:16 +1100Diagramming and visualising graphical models
https://danmackinlay.name/notebook/diagrams_graphical_models.html
Mon, 15 Mar 2021 16:44:16 +1100https://danmackinlay.name/notebook/diagrams_graphical_models.htmlDaggity dagR yEd diagrammeR flowchart.fun Mermaid TETRAD Matplotlib Graphviz tikz Misc References On the art and science of algorithmic line drawings for representing graphical models, which is a important part of statistics. The diagrams we need here are nearly flowchart-like, so I can sketch them with a flowchart if need be; but they are closely integrated with the equations of a particular statistical model, so I would like to incorporate them into the same system to avoid tedious and error-prone manual sync.Variational inference by message-passing in graphical models
https://danmackinlay.name/notebook/message_passing.html
Wed, 25 Nov 2020 17:42:32 +1100https://danmackinlay.name/notebook/message_passing.htmlReferences Variational inference where the model factorizes over some graphical independence structure, which means we get cheap and distributed inference. I am currently particularly interested in this for latent GP models. Many things can be expressed as message passing algorithms. The grandparent idea in this unification seems to be “Belief propagation”, a.k.a. “sum-product message-passing”, credited to (Pearl, 1982) for DAGs and then generalised to MRFs, PGMs, factor graphs etc.External validity
https://danmackinlay.name/notebook/external_validity.html
Mon, 09 Nov 2020 15:58:56 +1100https://danmackinlay.name/notebook/external_validity.htmlStandard graphical models Tools Salad Meta References TBD.
This Maori gentleman from the 1800s demonstrates an artful transfer learning from the western fashion domain
One could read Sebastian Ruder’s NN-style introduction to “transfer learning”. NN people like to think about this in particular way which I like because of the diversity of out-of-the-box ideas it invites and which I dislike because it is sloppy.Causal inference on DAGs
https://danmackinlay.name/notebook/causal_inference.html
Wed, 04 Nov 2020 12:36:13 +1100https://danmackinlay.name/notebook/causal_inference.htmlLearning materials do-calculus Counterfactuals Continuously indexed fields External validity Propensity scores Causal Graph inference from data Causal time series DAGS Drawing graphical models Tools References Inferring the optimal intervention requires accounting for which arrows are independent of which
Inferring cause and effect from nature. Graphical models and related techniques for doing it. Avoiding the danger of folk statistics. Observational studies, confounding, adjustment criteria, d-separation, identifiability, interventions, moral equivalence…Efficient factoring of GP likelihoods
https://danmackinlay.name/notebook/gp_factoring.html
Mon, 26 Oct 2020 12:46:34 +1100https://danmackinlay.name/notebook/gp_factoring.htmlBasic sparsity via inducing variables SVI for Gaussian processes Latent Gaussian Process models References There are many ways to cleverly slice up GP likelihoods so that inference is cheap.
This page is about some of them, especially the union of sparse and variational tricks. Scalable Gaussian process regressions choose cunning factorisations such that the model collapses down to a lower-dimensional thing than it might have seemed to need, at least approximately.Causal Bayesian networks
https://danmackinlay.name/notebook/causal_bayesian_networks.html
Tue, 01 Sep 2020 08:29:10 +1000https://danmackinlay.name/notebook/causal_bayesian_networks.htmlReferences Some kind of alternative graphical formalism for causal independence graphs 🤷?
discrete probability trees, sometimes also called staged tree models. A probability tree is one of the simplest models for representing the causal generative process of a random experiment or stochastic process The semantics are self-explanatory: each node in the tree corresponds to a potential state of the process, and the arrows indicate both the probabilistic transitions and the causal dependencies between them.Directed graphical models
https://danmackinlay.name/notebook/graphical_models_directed.html
Wed, 13 May 2020 13:40:29 +1000https://danmackinlay.name/notebook/graphical_models_directed.htmlReferences I found James F Fixx’s puzzle book on the shelf when writing this post
Graphs of conditional, directed independence are a convenient formalism for many models. These are also called Bayes nets (not to be confused with Bayesian inference.)
Once you have the graph, you can infer more detailed relations than mere conditional dependence or otherwise; this is precisely that hierarchical models emphasise.Learning graphical models from data
https://danmackinlay.name/notebook/graphical_models_learning.html
Sat, 11 Apr 2020 13:19:00 +1000https://danmackinlay.name/notebook/graphical_models_learning.htmlReferences Learning the independence graph structure from data in a graphical model.. A particular sparse model selection problem where the model is hierarchical.
Learning these models turns out to need a conditional independence test, an awareness of multiple testing and graph theory.
skggm (python) does the Gaussian thing but also has a nice sparsification and good explanation. Xun Zheng, Bryon Aragam and Chen Dan blog Learning DAGs with Continuous Optimization.Factor graphs
https://danmackinlay.name/notebook/factor_graphs.html
Mon, 16 Dec 2019 14:36:45 +1100https://danmackinlay.name/notebook/factor_graphs.htmlReferences A unifying formalism for the directed and undirected graphical models. How does that work then?
Wikipedia
A factor graph is a bipartite graph representing the factorization of a function. In probability theory and its applications, factor graphs are used to represent factorization of a probability distribution function, enabling efficient computations, such as the computation of marginal distributions through the sum-product algorithm.
To discuss: relation to message passing via factor graph decompositions?Inference on graphical models
https://danmackinlay.name/notebook/graphical_models_inference.html
Mon, 28 Oct 2019 08:19:36 +1100https://danmackinlay.name/notebook/graphical_models_inference.htmlIntroductory reading Doing variational inference References Given a graphical model and some observations of some nodes, what can I say about other nodes?
Introductory reading (Barber 2012; Steffen L. Lauritzen 1996) are rigorous introductions. (Murphy 2012) has a minimal introduction intermixed with some related models, with a more ML, more Bayesian formalism. For use in causality, (Pearl 2009; Spirtes, Glymour, and Scheines 2001) are readable.Undirected graphical models
https://danmackinlay.name/notebook/graphical_models_undirected.html
Mon, 28 Oct 2019 08:19:36 +1100https://danmackinlay.name/notebook/graphical_models_undirected.htmlReferences a.k.a Markov random fields, Markov random networks. (other types?)
I would like to know about spatial Poisson random fields, Markov random fields, Bernoulli random fields, especially for discrete multivariate sequences. Gibbs and Boltzman distribution inference.
Wasserman’s explanation of the use case here is good: Estimating Undirected Graphs Under Weak Assumptions
References Altun, Yasemin, Alex J. Smola, and Thomas Hofmann. 2004. “Exponential Families for Conditional Random Fields.Probabilistic graphical models over continuous index sets
https://danmackinlay.name/notebook/graphical_models_continuous.html
Wed, 25 Sep 2019 08:18:04 +1000https://danmackinlay.name/notebook/graphical_models_continuous.htmlReferences Placeholder for my notes on probabilistic graphical models over a continuum, i.e. with possibly-uncountably many nodes in the graph; or put another way, where the random field has an uncountable index set (but some kind of structure — a metric space, say.)
There is much formalising to be done here, which I do not propose to attempt right now. Here’s a concrete example. Consider Gaussian process whose covariance kerne \(K\) is continuous and of bounded support.Probabilistic graphical models
https://danmackinlay.name/notebook/graphical_models.html
Mon, 11 Sep 2017 17:20:44 +1000https://danmackinlay.name/notebook/graphical_models.htmlDirected graphs Undirected, a.k.a. Markov graphs Factor graphs Implementations References (Barber 2012):
Taxonomy of graphical models
Placeholder for my notes on probabilistic graphical models. In general graphical models are a particular type of way of handling multivariate data based on working out what is conditionally independent of what else.
Thematically, this is scattered across graphical models in inference, learning graphs from data, learning causation from data plus graphs, quantum graphical models because it all looks a bit different with noncommutative probability.Quantum-probabilistic graphical models
https://danmackinlay.name/notebook/graphical_models_quantum.html
Mon, 07 Aug 2017 11:46:35 +1000https://danmackinlay.name/notebook/graphical_models_quantum.htmlReferences When you have quantum probability, directed graphical models look different, especially for causality. So I am told.
Jacques Pienaar, Causality in the quantum world:
In the case of two entangled particles, Reichenbach’s principle would suggest that the correlations between the particles could be explained by a common cause. However, we also know that quantum statistics can violate Bell’s inequalities, which means that variables serving as common causes that could make the correlation disappear cannot exist.