Multimedia Tools and Applications

, Volume 75, Issue 11, pp 6431–6443 | Cite as

Interactive image recoloring by combining global and local optimization

  • Xujie Li
  • Hanli Zhao
  • Hui Huang
  • Zhongyi Hu
  • Lei Xiao


We propose a novel interactive image recoloring method by combining global and local optimization. Our approach assumes that each pixel is a linear transform of its neighbors which can be in spatial or feature space. Corresponding, a new framework for combining global and local energy optimization is designed and derived. By taking advantage of global and local color propagation, our approach requires only a few user scribbles to produce the high-quality results. We show various experimental results and comparisons on image recoloring. Compared with the state-of-the-art methods, our approach produces higher-quality results with only a small amount of user interaction than those only consider local propagation or global propagation approaches.


Image recoloring Global optimization Local optimization Color propagation 



This work was supported by the National Natural Science Foundation of China (Grant No. 61100146), the Zhejiang Provincial Natural Science Foundation of China (Grant Nos. LQ14F020006, LQ12F02010, LY12F02015 and LY12F02014), and the Science and Technology Plan Program of Wenzhou, China (Grant Nos. G20130017 and No. S20100053).


  1. 1.
    An X, Pellacini F (2008) Appprop: All-pairs appearance-space edit propagation. ACM Trans Graph 27(3):40:1–40:9CrossRefGoogle Scholar
  2. 2.
    Beigpour S, van de Weijer J (2011) Object recoloring based on intrinsic image estimation. In: Proceedings of the 2011 International Conference on Computer Vision, ICCV ’11, pp 327–334, Washington, DC, USA. IEEE Computer SocietyGoogle Scholar
  3. 3.
    Bhat P, Lawrence Zitnick C, Cohen M, Curless B (2010) Gradientshop: A gradient-domain optimization framework for image and video filtering. ACM Trans Graph 29(2):10:1–10:14CrossRefGoogle Scholar
  4. 4.
    Chen Q, Li D, Tang C-K (2013) Knn matting. IEEE Trans Pattern Anal Mach Intell 35(9):2175–2188CrossRefGoogle Scholar
  5. 5.
    Chen X, Zou D, Zhao Q, Tan P (2012) Manifold preserving edit propagation. ACM Trans Graph 31(6):132:1–132:7Google Scholar
  6. 6.
    Chen X, Zou D, Zhou SZ, Zhao Q, Tan P (2013) Image matting with local and nonlocal smooth priors. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp 1902–1907, Washington, DC, USA. IEEE Computer SocietyGoogle Scholar
  7. 7.
    Cohen-Or D, Sorkine O, Gal R, Leyvand T, Xu Y-Q (2006) Color harmonization. ACM Trans Graph 25(3):624–630CrossRefGoogle Scholar
  8. 8.
    Criminisi A, Sharp T, Rother C, P’erez P (2010) Geodesic image and video editing. ACM Trans Graph 29(5):134:1–134:15CrossRefGoogle Scholar
  9. 9.
    Fattal R (2009) Edge-avoiding wavelets and their applications. ACM Trans Graph 28(3):22:1–22:10CrossRefGoogle Scholar
  10. 10.
    Farbman Z, Fattal R, Lischinski D (2010) Diffusion maps for edge-aware image editing. ACM Trans Graph 29(6):145:1–145:10CrossRefGoogle Scholar
  11. 11.
    He K, Sun J, Tang X (2010) Guided image filtering. In: Proceedings of the 11th European Conference on Computer Vision: Part I, ECCV’10, pp 1–14. Springer-Verlag, Berlin, HeidelbergGoogle Scholar
  12. 12.
    Hsu E, Mertens T, Paris S, Avidan S, Durand F (2008) Light mixture estimation for spatially varying white balance. ACM Trans Graph 27(3):70:1–70:7CrossRefGoogle Scholar
  13. 13.
    Huang H, Li X, Zhao H, Nie G, Zhongyi H, Xiao L (2014) Manifold-preserving image colorization with nonlocal estimation. Multimedia Tools and Applications:1–14Google Scholar
  14. 14.
    Krishnan D, Szeliski R (2011) Multigrid and multilevel preconditioners for computational photography. ACM Trans Graph 30(6):177:1–177:10CrossRefGoogle Scholar
  15. 15.
    Lee P, Wu Y (2011) Nonlocal matting. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’11, pp 2193–2200, Washington, DC, USA. IEEE Computer SocietyGoogle Scholar
  16. 16.
    Levin A, Lischinski D, Weiss Y (2004) Colorization using optimization. ACM Trans Graph 23(3):689–694CrossRefGoogle Scholar
  17. 17.
    Levin A, Rav-Acha A, Lischinski D (2008) Spectral matting. IEEE Trans Pattern Anal Mach Intell 30(10):1699–1712CrossRefGoogle Scholar
  18. 18.
    Lin S, Ritchie D, Fisher M, Hanrahan P (2013) Probabilistic color-by-numbers: Suggesting pattern colorizations using factor graphs. ACM Trans Graph 32(4):37:1–37:12MATHGoogle Scholar
  19. 19.
    Liu X, Wan L, Yingge Q, Wong T-T, Lin S, Leung C-S, Heng P-A (2008) Intrinsic colorization. ACM Trans Graph 27(5):152:1–152:9CrossRefGoogle Scholar
  20. 20.
    Musialski P, Cui M, Ye J, Razdan A, Wonka P (2013) A framework for interactive image color editing. Vis Comput 29(11):1173–1186CrossRefGoogle Scholar
  21. 21.
    Olonetsky I, Avidan S (2012) Treecann - k-d tree coherence approximate nearest neighbor algorithm. In: Proceedings of the 12th European Conference on Computer Vision - Volume Part IV, ECCV’12, pp 602–615. Springer-Verlag, Berlin, HeidelbergGoogle Scholar
  22. 22.
    Sheng B, Sun H, Magnor M, Li P (2014) Video colorization using parallel optimization in feature space. IEEE Transactions on Circuits and Systems for Video Technology 24(3):407–417CrossRefGoogle Scholar
  23. 23.
    Sheng B, Sun H, Chen S, Liu X, Enhua W (2011) Colorization using the rotation-invariant feature space. IEEE Comput Graph Appl 31(2):24–35CrossRefGoogle Scholar
  24. 24.
    Seo S, Park Y, Ostromoukhov V (2013) Image recoloring using linear template mapping. Multimedia Tools Appl 64(2):293–308CrossRefGoogle Scholar
  25. 25.
    Wang B, Yizhou Y, Wong T-T, Chen C, Xu Y-Q (2010) Data-driven image color theme enhancement. ACM Trans Graph 29(6):146:1–146:10Google Scholar
  26. 26.
    Yatziv L, Sapiro G (2006) Fast image and video colorization using chrominance blending. Trans Img Proc 15(5):1120–1129CrossRefGoogle Scholar
  27. 27.
    Yingge Q, Wong T-T, Heng P-A (2006) Manga colorization. ACM Trans Graph 25(3):1214–1220CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Xujie Li
    • 1
  • Hanli Zhao
    • 1
  • Hui Huang
    • 1
  • Zhongyi Hu
    • 1
  • Lei Xiao
    • 1
  1. 1.Intelligent Information Systems InstituteWenzhou UniversityWenzhouChina

Personalised recommendations