Spectral Dichromatic Parameter Recovery from Two Views via Total Variation Hyper-priors

  • Filippo Bergamasco
  • Andrea Torsello
  • Antonio Robles-KellyEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)


In this paper, we propose an approach for the recovery of the dichromatic model from two hyperspectral or multispectral images, i.e., the joint estimation of illuminant, reflectance, and shading of each pixel, as well as the optical flow between the two views. The approach is based on the minimization of an energy functional linking the dichromatic model to the image appearances and the flow between the images to the factorized reflectance component. In order to minimize the resulting under-constrained problem, we apply vectorial total variation regularizers both to the scene reflectance, and to the flow hyper-parameters. We do this by enforcing the physical priors for the reflectance of the materials in the scene and assuming the flow varies smoothly within rigid objects in the image. We show the effectiveness of the approach compared with single view model recovery both in terms of model constancy and of closeness to the ground truth.


Optical Flow Gradient Magnitude Colour Constancy Bidirectional Reflectance Distribution Function Total Variation Regularization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Albarelli, A., Rodolà, E., Torsello, A.: Imposing semi-local geometric constraints for accurate correspondences selection in structure from motion: a game-theoretic perspective. IJCV 97(1), 36–53 (2012)CrossRefGoogle Scholar
  2. 2.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)CrossRefGoogle Scholar
  3. 3.
    Brainard, D.H., Delahunt, P.B., Freeman, W.T., Kraft, J.M., Xiao, B.: Bayesian model of human color constancy. J. Vis. 6(11), 1267–1281 (2006)CrossRefGoogle Scholar
  4. 4.
    Brelstaff, G., Blake, A.: Detecting specular reflection using Lambertian constraints. In: International Conference on Computer Vision, pp. 297–302 (1988)Google Scholar
  5. 5.
    Bresson, X., Chan, T.F.: Fast dual minimization of the vectorial total variation norm and applications to color image processing (2008)Google Scholar
  6. 6.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE TPAMI 33(3), 500–513 (2011)CrossRefGoogle Scholar
  7. 7.
    Drulea, M., Nedevschi, S.: Total variation regularization of local-global optical flow. In: 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 318–323, October 2011Google Scholar
  8. 8.
    Finlayson, G.D., Schaefer, G.: Convex and non-convex illuminant constraints for dichromatic colour constancy. In: IEEE CVPR, vol. 1, pp. 598–604 (2001)Google Scholar
  9. 9.
    Finlayson, G.D., Schaefer, G.: Solving for colour constancy using a constrained dichromatic reflection model. IJCV 42(3), 127–144 (2001)CrossRefzbMATHGoogle Scholar
  10. 10.
    Forsyth, A.: Calculus of Variations. Dover Books on Advanced Mathematics. Dover Publications, New York (1960)zbMATHGoogle Scholar
  11. 11.
    Foster, D.H., Amano, K., Nascimento, S.M.C., Foster, M.J.: Frequency of metamerism in natural scenes. J. Opt. Soc. Am. A 23(10), 2359–2372 (2006)CrossRefGoogle Scholar
  12. 12.
    Huynh, C.P., Robles-Kelly, A.: A solution of the dichromatic model for multispectral photometric invariance. IJCV 90(1), 1–27 (2010)CrossRefGoogle Scholar
  13. 13.
    Huynh, C.P., Robles-Kelly, A., Hancock, E.R.: Shape and refractive index recovery from single-view polarisation images. In: IEEE CVPR (2010)Google Scholar
  14. 14.
    Klinker, G., Shafer, S., Kanade, T.: A physical approach to color image understanding. Int. J. Comput. Vis. 4(1), 7–38 (1990)CrossRefGoogle Scholar
  15. 15.
    Kong, N., Tai, Y., Shin, J.S.: A physically-based approach to reflection separation: from physical modeling to constrained optimization. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014)CrossRefGoogle Scholar
  16. 16.
    Land, E.H., Mccann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11 (1971)CrossRefGoogle Scholar
  17. 17.
    Leordeanu, M., Zanfir, A., Sminchisescu, C.: Locally affine sparse-to-dense matching for motion and occlusion estimation. In: IEEE ICCV, December 2013Google Scholar
  18. 18.
    Lin, S., Shum, H.: Separation of diffuse and specular reflection in color images. In: International Conference on Computer Vision and Pattern Recognition (2001)Google Scholar
  19. 19.
    Marr, D., Poggio, T.: A computational theory of human stereo vision. Proc. R. Soc. Lond. Ser. B Biol. Sci. 204, 301–328 (1979)Google Scholar
  20. 20.
    Nagel, H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8, 565–593 (1986)CrossRefGoogle Scholar
  21. 21.
    Narasimhan, S.G., Nayar, S.K.: Contrast restoration of weather degraded images. IEEE TPAMI 25, 713–724 (2003)CrossRefGoogle Scholar
  22. 22.
    Nayar, S., Bolle, R.: Reflectance based object recognition. Int. J. Comput. Vis. 17(3), 219–240 (1996)CrossRefGoogle Scholar
  23. 23.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: ICCV, pp. 1133–1140. IEEE (2009)Google Scholar
  24. 24.
    Robles-Kelly, A., Huynh, C.P.: Imaging Spectroscopy for Scene Analysis. Springer, London (2013)CrossRefGoogle Scholar
  25. 25.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10(4), 210–218 (1985)CrossRefGoogle Scholar
  27. 27.
    Stiles, W.S., Burch, J.M.: Interim report to the Commission Internationale de l’Éclairage Zurich, 1955, on the National Physical Laboratory’s investigation of colour-matching. Optica Acta 2, 168–181 (1955)CrossRefGoogle Scholar
  28. 28.
    Tan, R.T., Nishino, K., Ikeuchi, K.: Separating reflection components based on chromaticity and noise analysis. IEEE TPAMI 26(10), 1373–1379 (2004)CrossRefGoogle Scholar
  29. 29.
    Terzopoulos, D.: Multilevel computational processes for visual surface reconstruction. Comput. Vis. Graph. Image Underst. 24, 52–96 (1983)CrossRefGoogle Scholar
  30. 30.
    Tominanga, S., Wandell, B.A.: Standard surface-reflectance model and illuminant estimation. J. Opt. Soc. Am. A 6, 576–584 (1989)CrossRefGoogle Scholar
  31. 31.
    Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, pp. 2464–2471. IEEE (2010)Google Scholar
  32. 32.
    Zickler, T., Mallick, S.P., Kriegman, D.J., Belhumeur, P.N.: Color subspaces as photometric invariants. IJCVs 79(1), 13–30 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Filippo Bergamasco
    • 1
  • Andrea Torsello
    • 1
  • Antonio Robles-Kelly
    • 2
    • 3
    Email author
  1. 1.Dipart. di Sci. Ambientali, Informatica e StatisticaUniversità Ca’ Foscari VeneziaVeniceItaly
  2. 2.CSIRODATA61CanberraAustralia
  3. 3.College of Engineering and Computer ScienceAustralian National UniversityCanberraAustralia

Personalised recommendations