Image Colorization with an Affective Word

  • Xiaohui Wang
  • Jia Jia
  • Hanyu Liao
  • Lianhong Cai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7633)


An important role of image color is the conveyer of emotions (through color themes). The colorization is less useful with an undesired color theme, even semantically correct, which has been rarely considered previously. In this paper, we propose a complete system for the image colorization with an affective word. We only need users to assist object segmentation along with text labels and give an affective word. First, the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images. Second, we select a set of color themes according to the affective word based on art theories. With these themes, a generic algorithm is adopted to select the best reference for each object. Finally, we propose a hybrid texture synthesis approach to colorize each object. Our experiments show that the results of our system have both the correct semantics and the desired emotions.


Image colorization affective word color theme 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Transactions on Graphics 23, 689 (2004)CrossRefGoogle Scholar
  2. 2.
    Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to greyscale images. ACM Transactions on Graphics 21(3), 277–280 (2002)CrossRefGoogle Scholar
  3. 3.
    Ironi, R., Cohen-Or, D., Lischinski, D.: Colorization by example. In: Rendering Techniques, pp. 201–210 (2005)Google Scholar
  4. 4.
    Charpiat, G., Hofmann, M., Schölkopf, B.: Automatic Image Colorization Via Multimodal Predictions. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 126–139. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Tai, Y.W., Jia, J., Tang, C.K.: Local color transfer via probabilistic segmentation by expectation-maximization. In: Computer Vision and Pattern Recognition, vol. 1, pp. 747–754. IEEE (2005)Google Scholar
  6. 6.
    Chia, A.Y.S., Zhuo, S., Gupta, R.K., Tai, Y.W., Cho, S.Y., Tan, P., Lin, S.: Semantic colorization with internet images. ACM Transactions on Graphics 30(6), 1–8 (2011)CrossRefGoogle Scholar
  7. 7.
    Arnheim, R.: Art and visual perception: A psychology of the creative eye. University of California Press (1954)Google Scholar
  8. 8.
    Kobayashi, S.: Color image scale. Kosdansha International (1991)Google Scholar
  9. 9.
    Kobayashi, S.: Art of Color Combinations. Kosdansha International (1995)Google Scholar
  10. 10.
    Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Computer Graphics and Applications 21(5), 34–41 (2001)CrossRefGoogle Scholar
  11. 11.
    Chen, T., Cheng, M.M., Tan, P., Shamir, A., Hu, S.M.: Sketch2photo: internet image montage. ACM Transactions on Graphics 28(5), 124:1–124:10 (2009)Google Scholar
  12. 12.
    Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., Xu, Y.Q.: Color harmonization. ACM Transactions on Graphics 25(3), 624–630 (2006)CrossRefGoogle Scholar
  13. 13.
    O’Donovan, P., Agarwala, A., Hertzmann, A.: Color compatibility from large datasets. ACM Transactions on Graphics 30(4), 63:1–63:12 (2011)CrossRefGoogle Scholar
  14. 14.
    Rother, C., Kolmogorov, V., Blake, A.: ”GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23(3), 309–314 (2004)CrossRefGoogle Scholar
  15. 15.
    Cheng, M.M., Zhang, F.L., Mitra, N.J., Huang, X., Hu, S.M.: Repfinder: finding approximately repeated scene elements for image editing. ACM Transactions on Graphics 29, 83:1–83:8 (2010)CrossRefGoogle Scholar
  16. 16.
    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 409–416. IEEE (June 2011)Google Scholar
  17. 17.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(4), 509–522 (2002)CrossRefGoogle Scholar
  18. 18.
    Wang, X.H., Jia, J., Cai, L.H.: Affective image adjustment with a single word. To apper in The Visual ComputerGoogle Scholar
  19. 19.
    Dong, Z., Dong, Q.: HowNet and the Computation of Meaning. World Scientific (2006)Google Scholar
  20. 20.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics 28(3), 24:1–24:11 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Xiaohui Wang
    • 1
  • Jia Jia
    • 1
  • Hanyu Liao
    • 1
  • Lianhong Cai
    • 1
  1. 1.Key Laboratory of Pervasive Computing, Ministry of Education, Tsinghua National Laboratory for Information Science and Technology (TNList), Department of Computer Science and TechnologyTsinghua UniversityChina

Personalised recommendations