2.1 Wait, Jeffreys?

Sir Harold Jeffreys sought a solution to this problem. He developed a prior invariant to choice of parametrization. In practice, what is most important is not the Jeffreys prior but the “Jeffreys” posterior generated from combining the prior with the likelihood. With a Jeffreys posterior of one variable, a change-of-variables will yield the Jeffreys posterior of the new variable. This way, the Jeffreys prior gives a posterior that best represents the data, no matter how it is expressed. In scientific communication, this means different information about the same data can be expressed through change-of-variables rather than by starting over and finding a new posterior.

 

Non-informative priors are appealing because they are flat priors in a meaningful parametrization, which allow us to conduct Bayesian inference without much influence from prior knowledge.

A classic prior distribution commonly mistaken as non-informative is \(Beta(1,1)\). Beta distributions are widely used to measure the probability of a certain event \(X\) on a scale of \((0,1)\), whereas \(Beta(1,1)\) indicates that the probability is equally distributed. The flat prior distribution is \(\pi(\theta) = 1\). Since \(\theta\) lies between \(0\) and \(1\), we can reparametrize \(\theta\) using the log-odds ratio: \(\rho = log\frac{\theta}{1-\theta}\). Under this change of variables, the prior distribution \(\pi(\rho)\) shows transformation variance. This highlights the importance of finding a prior that delivers a principle of invariance. (Jordan 2010)