Expectation Propagation - The Essential Minimum

Taken from Vehtari et al. (2014). Assume that you can factorize your target distribution f(θ)k=0Kfk(θ) Approximate target distribution using a distribution with a similar factorization, called the global approximation g(θ)k=0Kgk(θ) At each iteration of the algorithm, and for every factor k=1,,K form the cavity distribution by removing kth term from the global approximation gkg(θ)gk(θ) also form the tilted distribution by including fk(θ) to the cavity distribution (in place of gk(θ)) gk(θ)fk(θ)gk(θ) The algorithm updates the site approximation gk(θ) so that the new global approximation approximates the tilted distribution gk(θ)newgk(θ)fk(θ)gk(θ)

Bibliography

Vehtari, Aki, Andrew Gelman, Tuomas Sivula, Pasi Jylänki, Dustin Tran, Swupnil Sahai, Paul Blomstedt, John P. Cunningham, David Schiminovich, and Christian Robert. 2014. “Expectation Propagation as a Way of Life: A Framework for Bayesian Inference on Partitioned Data.”

Avatar
Mauro Camara Escudero
Machine Learning Engineer

My research interests include approximate manifold sampling and generative models.

Related