.ipynb .pdf

Binder

Bayesian approach

• Suppose $\mathbf{z}=[{\mathbf{x}}^{\prime },{\mathbf{y}}^{\prime }{]}^{\prime }$ is a random vector with joint density $p(\mathbf{z})$

• Suppose we observe a sample realization of $\mathbf{x}$. How does that change our knowledge of $\mathbf{y}$?

• before observing the sample $\mathbf{x}$, our knowledge of $\mathbf{y}$ is given by

$$p(\mathbf{y})={\int }_{\mathbf{x}}p(\mathbf{z})\text{d}\mathbf{x}$$

• after observing $\mathbf{x}$, our knowledge of $\mathbf{y}$ is given by

$$p(\mathbf{y}|\mathbf{x})=\frac{p(\mathbf{z})}{p(\mathbf{x})}=\frac{p(\mathbf{x}|\mathbf{y})p(\mathbf{y})}{p(\mathbf{x})}$$

• for exmple, if $\mathbf{z}$ is Gaussian:

$$\begin{array}{r}p(\mathbf{y}|\mathbf{x})=\mathcal{N}({\mathit{\mu }}_y+{\mathbf{\Sigma }}_{yx}{\mathbf{\Sigma }}_x^{-1}(\mathbf{x}-{\mathit{\mu }}_x),{\mathbf{\Sigma }}_y-{\mathbf{\Sigma }}_{yx}{\mathbf{\Sigma }}_x^{-1}{\mathbf{\Sigma }}_{xy})\end{array}$$

$$p(\mathbf{y}|\mathbf{x})=\frac{p(\mathbf{z})}{p(\mathbf{x})}=\frac{p(\mathbf{x}|\mathbf{y})p(\mathbf{y})}{p(\mathbf{x})}$$

• in Bayesian parlance

  • $p(\mathbf{y})$ is prior distribution

  • $p(\mathbf{y}|\mathbf{x})$ is posterior distribution

  • $p(\mathbf{x}|\mathbf{y})$ is likelihood

• typically, we would think of $\mathbf{y}$ as parameters, and use instead $\theta$

• also, the prior and the likelihood are specified separately

• as a result, the posterior is not availalbe in closed form

• simulation method are used instead, to sample from it

since $p(\mathbf{x})$ is independent from $\theta$

$$p(\theta |\mathbf{x})=\frac{p(\mathbf{x}|\theta )p(\theta )}{p(\mathbf{x})}\propto p(\mathbf{x}|\theta )p(\theta )$$

Goal of Bayesian inference: characterize the distribution of $\theta$, given

• the data $\mathbf{x}$

• the prior knowledge about $\theta$

• the model, embodied by the likelihood function

characterize here means estimate the moments of the posterior, based on sample draws from it

previous

Granger causality

next

Homework 0