Bayesian Robustness: A New Look from Geometry

Abstract

The geometric concept of the Lie derivative is introduced as the natural way of quantifying the intrinsic robustness of a hypothesis space. Prior and posterior probability measures are interpreted as differential forms defined invariantly on the hypothesis space. Rates of change with respect to local deformations of the model are computed by means of Lie derivatives of tensors defined on the model (like the information metric, prior, posterior, etc.). In this way a field theory of inference is obtained. The class of deformations preserving the state of total ignorance is introduced and characterized by a partial differential equation. For location models this equation is the familiar ∇ · ξ= 0. A simple condition for the robustness of prior (or posterior) distributions is found: There is robustness when the deformation is along level surfaces of the prior (or posterior) density. These results are then applied to the class of entropic priors. It is shown that the hyper parameter controls the sensitivity with respect to local deformations. It is also shown that entropic priors are only sensitive to deformations that change the intrinsic form of the model around the initial guess.