Scale Invariant Markov Models for Bayesian Inversion of Linear Inverse Problems

  • Stéphane Brette
  • Jérôme Idier
  • Ali Mohammad-Djafari
Conference paper
Part of the Fundamental Theories of Physics book series (FTPH, volume 70)


In a Bayesian approach for solving linear inverse problems one needs to specify the prior laws for calculation of the posterior law. A cost function can also be defined in order to have a common tool for various Bayesian estimators which depend on the data and the hyperparameters. The Gaussian case excepted, these estimators are not linear and so depend on the scale of the measurements. In this paper a weaker property than linearity is imposed on the Bayesian estimator, namely the scale invariance property (SIP).

First, we state some results on linear estimation and then we introduce and justify a scale invariance axiom. We show that arbitrary choice of scale measurement can be avoided if the estimator has this SIP. Some examples of classical regularization procedures are shown to be scale invariant. Then we investigate general conditions on classes of Bayesian estimators which satisfy this SIP, as well as their consequences on the cost function and prior laws. We also show that classical methods for hyperparameters estimation (i.e., Maximum Likelihood and Generalized Maximum Likelihood) can be introduced for hyperparameters estimation, and we verify the SIP property for them.

Finally we discuss how to choose the prior laws to obtain scale invariant Bayesian estimators. For this, we consider two cases of prior laws: entropic prior laws and first-order Markov models. In related preceding works [1, 2], the SIP constraints have been studied for the case of entropic prior laws. In this paper extension to the case of first-order Markov models is provided.

Key Words

Bayesian estimation Scale invariance Markov modelling Inverse Problems Image reconstruction Prior model selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Mohammad-Djafari and J. Idier, “Maximum entropy prior laws of images and estimation of their parameters,” in Maximum Entropy and Bayesian Methods in Science and Engineering ( T. Grandy, ed.), ( Dordrecht, The Netherlands), MaxEnt Workshops, Kluwer Academic Publishers, 1990.Google Scholar
  2. [2]
    A. Mohammad-Djafari and J. Idier, “Scale invariant Bayesian estimators for linear inverse problems,” in Proc. of the First ISBA meeting, (San Fransisco, USA ), Aug. 1993.Google Scholar
  3. [3]
    G. Demoment, “Image reconstruction and restoration: Overview of common estimation structure and problems,” IEEE Transactions on Acoustics Speech and Signal Processing, vol. 37, pp. 2024 – 2036, Dec. 1989.CrossRefGoogle Scholar
  4. [4]
    A. Mohammad-Djafari and G. Demoment, “Estimating priors in maximum entropy image processing,” in Proceedings of IEEE ICASSP, pp. 2069–2072, IEEE, 1990.Google Scholar
  5. [5]
    G. Le Besnerais, J. Navaza, and G. Demoment, “Aperture synthesis in astronomical radio-interferometry using maximum entropy on the mean,” in SPIE Conf., Stochastic and Neural Methods in Signal Processing, Image Processing and Computer Vision (S. Chen, ed.), ( San Diego ), p. 11, July 1991.Google Scholar
  6. [6]
    G. Le Besnerais, J. Navaza, and G. Demoment, “Synthèse d’ouverture en radioastronomie par maximum d’entropie sur la moyenne,” in Actes du 13ème colloque GRETSI, (Juan-les-Pins, France), pp. 217–220, Sept. 1991.Google Scholar
  7. [7]
    E. Jaynes, “Prior probabilities,” IEEE Transactions on Systems Science and Cybernetics, vol. SSC-4, pp. 227 – 241, Sept. 1968.Google Scholar
  8. [8]
    G. Box and T. G.C., Bayesian inference in statistical analysis. Addison-Wesley publishing, 1972.Google Scholar
  9. C. Bouman and K. Sauer, “A generalized Gaussian image model for edge-preserving MAP estimation,” IEEE Transactions on Medical Imaging, vol. MI-2, no. 3, pp. 296–310, 1993.Google Scholar
  10. [10]
    J. Besag, “Digital image processing: Towards Bayesian image analysis,” Journal of Applied Statistics, vol. 16, no. 3, pp. 395–407,1989.CrossRefGoogle Scholar
  11. [11]
    Glatter, O.: 1982, In Small Angle X-ray Scattering edited by O. Glatter and O. Kratky.Academic Press, London.Google Scholar
  12. [12]
    Gull, S.F.: 1989, ‘Developments in Maximum Entropy Data Analysis’, in Maximum-Entropy and Bayesian Methods, edited by J. Skilling, pp. 53–71. Dordrecht: Kluwer Academic Publishers.Google Scholar
  13. [13]
    MacKay, D.J.C.: 1992, ‘Bayesian Interpolation’, in Maximum Entropy and Bayesian Methods, Seattle, 1991edited by C.R.Smith et al, pp. 39–66. Kluwer Academic Publishers, Netherlands.Google Scholar
  14. [14]
    May, R. P. & Nowotny, V. (1989). ‘Distance Information Derived from Neutron Low-Q Scattering’J. Appi. Cryst 22,231–237CrossRefGoogle Scholar
  15. [15]
    Moore, P. B. (1980). ‘Small-Angle Scattering. Information Content and Error Analysis’J. Appl. Cryst. 13, 168–175CrossRefGoogle Scholar
  16. [16]
    Morrison, J. D., Corcoran, J. D. & Lewis, K. E.: 1992, ‘The Determination of Particle Size Distributions in Small-Angle Scattering Using the Maximum-Entropy Method’,J. Appi. Cryst. 25, 504–513.CrossRefGoogle Scholar
  17. [17]
    Müller, J.J., Zalkova, T.N., Zirwer, D., Misselwitz, R., Gast K., Serdyuk, I.N., Welfle H., Damschun, G.: 1986, ‘Comparison of the structure ribosomal 5S RNA from E.coli and from rat liver using X-ray scattering and dynamic light scattering’, Eur. Biophys. J13 301 –307CrossRefGoogle Scholar
  18. [18]
    Müller, J.J., & Hansen, S.: 1994, ‘A Study of High-Resulution X-ray Scattering Data Evaluation by the Maximum-Entrophy Method’, J. Appl. Cryst. 27 257–270CrossRefGoogle Scholar
  19. [19]
    Skilling, J.: 1988, “The Axioms Of Maximum Entropy”, in Maximum-Entropy and Bayesian Methods in Science and Engineering (Vol 1) edited by G. J. Erickson and C. Ray Smith, pp.173–187. Kluwer Academic publichers. Dordrecht.Google Scholar
  20. [20]
    Skilling, J.: 1989, ‘Classical Maximum Entropy’, in Maximum-Entropy and Bayesian Methods edited by J. Skilling, pp. 42–52. Kluwer Academic Publisher, Dordrecht.Google Scholar
  21. [21]
    Skilling, J.: 1991, ‘On parameter estimation and quantified MaxEnt’ in Maximum-Entropy and Bayesian Methods, edited by Grandy and Schick, pp. 267–273. Kluwer Academic Publishers, Dordrecht.CrossRefGoogle Scholar
  22. [22]
    Steenstrup, S. & Hansen S.: 1994, ‘The Maximum-Entropy Method without the Positivity Constraint —Applications to the Determination of the Distance-Distribution Function in Small-Angle Scattering’, J. Appl. Cryst.27, 574–580CrossRefGoogle Scholar
  23. [23]
    Svergun, D.I.: 1992, ‘Determination of the Regularization Parameter in Indirect-Transform methods using perceptual criteria’, J. Appl. Cryst.25 495–503.CrossRefGoogle Scholar
  24. [24]
    Svergun, D.I., Semenyuk, A.V. & Feigin, L.A.: 1988, ‘Small-Angle-Scattering-Data Treatment by the Regularization Method’,Acta Cryst. A44, 244–250.Google Scholar
  25. [25]
    Tikhonov, A.N. & Arsenin, V. Ya.: 1977 in Solution of Ill-Posed Problems,New York:Wiley.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • Stéphane Brette
    • 1
  • Jérôme Idier
    • 1
  • Ali Mohammad-Djafari
    • 1
  1. 1.Laboratoire des Signaux et Systèmes (CNRS-ESE-UPS)École Supérieure d’ÉlectricitéGif-sur-Yvette CedexFrance

Personalised recommendations