Abstract
In this paper, we propose an approach for the recovery of the dichromatic model from two hyperspectral or multispectral images, i.e., the joint estimation of illuminant, reflectance, and shading of each pixel, as well as the optical flow between the two views. The approach is based on the minimization of an energy functional linking the dichromatic model to the image appearances and the flow between the images to the factorized reflectance component. In order to minimize the resulting under-constrained problem, we apply vectorial total variation regularizers both to the scene reflectance, and to the flow hyper-parameters. We do this by enforcing the physical priors for the reflectance of the materials in the scene and assuming the flow varies smoothly within rigid objects in the image. We show the effectiveness of the approach compared with single view model recovery both in terms of model constancy and of closeness to the ground truth.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Albarelli, A., Rodolà, E., Torsello, A.: Imposing semi-local geometric constraints for accurate correspondences selection in structure from motion: a game-theoretic perspective. IJCV 97(1), 36–53 (2012)
Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)
Brainard, D.H., Delahunt, P.B., Freeman, W.T., Kraft, J.M., Xiao, B.: Bayesian model of human color constancy. J. Vis. 6(11), 1267–1281 (2006)
Brelstaff, G., Blake, A.: Detecting specular reflection using Lambertian constraints. In: International Conference on Computer Vision, pp. 297–302 (1988)
Bresson, X., Chan, T.F.: Fast dual minimization of the vectorial total variation norm and applications to color image processing (2008)
Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE TPAMI 33(3), 500–513 (2011)
Drulea, M., Nedevschi, S.: Total variation regularization of local-global optical flow. In: 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 318–323, October 2011
Finlayson, G.D., Schaefer, G.: Convex and non-convex illuminant constraints for dichromatic colour constancy. In: IEEE CVPR, vol. 1, pp. 598–604 (2001)
Finlayson, G.D., Schaefer, G.: Solving for colour constancy using a constrained dichromatic reflection model. IJCV 42(3), 127–144 (2001)
Forsyth, A.: Calculus of Variations. Dover Books on Advanced Mathematics. Dover Publications, New York (1960)
Foster, D.H., Amano, K., Nascimento, S.M.C., Foster, M.J.: Frequency of metamerism in natural scenes. J. Opt. Soc. Am. A 23(10), 2359–2372 (2006)
Huynh, C.P., Robles-Kelly, A.: A solution of the dichromatic model for multispectral photometric invariance. IJCV 90(1), 1–27 (2010)
Huynh, C.P., Robles-Kelly, A., Hancock, E.R.: Shape and refractive index recovery from single-view polarisation images. In: IEEE CVPR (2010)
Klinker, G., Shafer, S., Kanade, T.: A physical approach to color image understanding. Int. J. Comput. Vis. 4(1), 7–38 (1990)
Kong, N., Tai, Y., Shin, J.S.: A physically-based approach to reflection separation: from physical modeling to constrained optimization. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014)
Land, E.H., Mccann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11 (1971)
Leordeanu, M., Zanfir, A., Sminchisescu, C.: Locally affine sparse-to-dense matching for motion and occlusion estimation. In: IEEE ICCV, December 2013
Lin, S., Shum, H.: Separation of diffuse and specular reflection in color images. In: International Conference on Computer Vision and Pattern Recognition (2001)
Marr, D., Poggio, T.: A computational theory of human stereo vision. Proc. R. Soc. Lond. Ser. B Biol. Sci. 204, 301–328 (1979)
Nagel, H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8, 565–593 (1986)
Narasimhan, S.G., Nayar, S.K.: Contrast restoration of weather degraded images. IEEE TPAMI 25, 713–724 (2003)
Nayar, S., Bolle, R.: Reflectance based object recognition. Int. J. Comput. Vis. 17(3), 219–240 (1996)
Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the Mumford-Shah functional. In: ICCV, pp. 1133–1140. IEEE (2009)
Robles-Kelly, A., Huynh, C.P.: Imaging Spectroscopy for Scene Analysis. Springer, London (2013)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)
Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10(4), 210–218 (1985)
Stiles, W.S., Burch, J.M.: Interim report to the Commission Internationale de l’Éclairage Zurich, 1955, on the National Physical Laboratory’s investigation of colour-matching. Optica Acta 2, 168–181 (1955)
Tan, R.T., Nishino, K., Ikeuchi, K.: Separating reflection components based on chromaticity and noise analysis. IEEE TPAMI 26(10), 1373–1379 (2004)
Terzopoulos, D.: Multilevel computational processes for visual surface reconstruction. Comput. Vis. Graph. Image Underst. 24, 52–96 (1983)
Tominanga, S., Wandell, B.A.: Standard surface-reflectance model and illuminant estimation. J. Opt. Soc. Am. A 6, 576–584 (1989)
Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, pp. 2464–2471. IEEE (2010)
Zickler, T., Mallick, S.P., Kriegman, D.J., Belhumeur, P.N.: Color subspaces as photometric invariants. IJCVs 79(1), 13–30 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bergamasco, F., Torsello, A., Robles-Kelly, A. (2017). Spectral Dichromatic Parameter Recovery from Two Views via Total Variation Hyper-priors. In: Chen, CS., Lu, J., Ma, KK. (eds) Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science(), vol 10116. Springer, Cham. https://doi.org/10.1007/978-3-319-54407-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-54407-6_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54406-9
Online ISBN: 978-3-319-54407-6
eBook Packages: Computer ScienceComputer Science (R0)