# Expectation Propagation - The Essential Minimum

Taken from Vehtari et al. (2014). Assume that you can factorize your **target distribution**
\[
f(\theta) \propto \prod_{k=0}^K f_k(\theta)
\]
Approximate target distribution using a distribution with a similar factorization, called the **global approximation**
\[
g(\theta) \propto \prod_{k=0}^K g_k(\theta)
\]
At each iteration of the algorithm, and for every factor \(k=1, \ldots, K\) form the **cavity distribution** by removing \(k^{\text{th}}\) term from the global approximation
\[
g_{-k} \propto \frac{g(\theta)}{g_k(\theta)}
\]
also form the **tilted distribution** by including \(f_k(\theta)\) to the cavity distribution (in place of \(g_k(\theta)\))
\[
g_{\backslash k}(\theta) \propto f_k(\theta) g_{-k}(\theta)
\]
The algorithm updates the site approximation \(g_k(\theta)\) so that the new global approximation approximates the tilted distribution
\[
g_k(\theta)^{\text{new}}g_{-k}(\theta) \approx f_k(\theta)g_{-k}(\theta)
\]

# Bibliography

Vehtari, Aki, Andrew Gelman, Tuomas Sivula, Pasi Jylänki, Dustin Tran, Swupnil Sahai, Paul Blomstedt, John P. Cunningham, David Schiminovich, and Christian Robert. 2014. “Expectation Propagation as a Way of Life: A Framework for Bayesian Inference on Partitioned Data.”