Analytic solutions

The range of problems for which an analytic solution is possible is limited and depends on the choice of the model (leading to a tractable likelihood function) and prior.

A simple model assumes that the data ${{\mbox{\boldmath$x$}}=(x_1,\ldots,n_n)}$ is a random sample, of size $n$, from some distribution having density function $p({\mbox{\boldmath$x$}}\vert {\mbox{\boldmath$\theta$}})$ with

$$p({\mbox{\boldmath$x$}}\vert {\mbox{\boldmath$\theta$}}) = \prod_{i=1}^n p(x_i \vert {\mbox{\boldmath$\theta$}}) .$$ (11)

If a prior $p({\mbox{\boldmath$\theta$}})$ is chosen from some family $\cal F$, say $f({\mbox{\boldmath$\theta$}} \vert {\mbox{\boldmath$\alpha$}} )$, the choice of ${\mbox{\boldmath$\alpha$}}$ being regarded as part of the model specification, the posterior is given by

$$p({\mbox{\boldmath$\theta$}} \vert {\mbox{\boldmath$x$}}) =... ...rt {\mbox{\boldmath$\alpha$}}) d{\mbox{\boldmath$\theta$}}}.$$ (12)

If $p({\mbox{\boldmath$\theta$}}\vert{\mbox{\boldmath$x$}})$ is also a member of $\cal F$, say

$$p({\mbox{\boldmath$\theta$}} \vert {\mbox{\boldmath$x$}}) = f({\mbox{\boldmath$\theta$}} \vert {\mbox{\boldmath$\beta$}})$$ (13)

where the parameter ${\mbox{\boldmath$\beta$}}$ is a function of only ${\mbox{\boldmath$\alpha$}}$ and ${\mbox{\boldmath$x$}}$ then the family $\cal F$ is said to be closed under sampling, with respect to the density $p(x \vert{\mbox{\boldmath$\theta$}})$, (Barnard, 1949). The prior $p({\mbox{\boldmath$\theta$}} \vert {\mbox{\boldmath$\alpha$}})$ is called a conjugate prior for $p(x \vert{\mbox{\boldmath$\theta$}})$ (see Smith and Bernardo, 1994, p. 265).