As an illustration, consider inference about the mean $\theta$ of an exponential distribution with density

p(x\vert\theta) = \theta^{-1} e^{-x/\theta}      (x > 0, \theta > 0).
\end{displaymath} (14)

The likelihood of a random sample ${\mbox{\boldmath$x$}} = (x_1, x_2, \ldots ,
x_n)^T$ is

p({\mbox{\boldmath$x$}} \vert \theta) = \theta^{-n} e^{-n \bar x/\theta}
\end{displaymath} (15)

in terms of the sufficient statistic3

\bar x = \sum_{i=1}^n \frac{x_i}{n}.
\end{displaymath} (16)

Take as prior a density of the form

$\displaystyle p(\theta)$ $\textstyle =$ $\displaystyle f(\theta \vert \alpha_1, \alpha_2)$ (17)
  $\textstyle =$ $\displaystyle \frac{\alpha_2^{\alpha_1-1}}{\Gamma(\alpha_1-1)} \theta^{-\alpha_1}
e^{-\alpha_2/\theta}$ (18)
  $\textstyle \propto$ $\displaystyle \theta^{-\alpha_1} e^{-\alpha_2/\theta}$ (19)

then the corresponding posterior is

$\displaystyle p(\theta\vert{\mbox{\boldmath$x$}})$ $\textstyle \propto$ $\displaystyle \theta^{-\alpha_1}e^{-\alpha_2/\theta}
\theta^{-n} e^{-n\bar x / \theta}$ (20)
  $\textstyle =$ $\displaystyle \theta^{-(\alpha_1 + n)} e^{-(\alpha_2 +n\bar x)/\theta}$ (21)

which is clearly also of the form (17) and so may be written as

$\displaystyle p(\theta\vert{\mbox{\boldmath$x$}})$ $\textstyle =$ $\displaystyle f(\theta \vert
\alpha_2^\prime)$ (22)

    $\displaystyle \alpha_1^\prime = \alpha_1 + n\mbox{ , and } \alpha_2^\prime =
\alpha_2 + n\bar x$ (23)

Hence (14) is closed under sampling, with (17) as a convenient natural conjugate prior family for inference about the mean $\theta$. In such a case a complete analytic solution is available.

danny 2009-07-23