Abstract
Recent progress in computational photography has shown that we can acquire physical information beyond visible (RGB) image representations. In particular, we can acquire near-infrared (NIR) cues with only slight modification to any standard digital camera. In this paper, we study whether this extra channel can improve semantic image segmentation. Based on a state-of-the-art segmentation framework and a novel manually segmented image database that contains 4-channel images (RGB+NIR), we study how to best incorporate the specific characteristics of the NIR response. We show that it leads to improved performances for 7 classes out of 10 in the proposed dataset and discuss the results with respect to the physical properties of the NIR response.
This work was supported by the Swiss National Science Foundation under grant number 200021-124796/1 and Xerox foundation.
Chapter PDF
Similar content being viewed by others
Keywords
- Conditional Random Field
- Regularization Part
- Fisher Vector
- Pairwise Potential
- Conditional Random Field Model
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)
Csurka, G., Perronnin, F.: An efficient approach to semantic segmentation. IJCV 95 (2011)
Verbeek, J., Triggs, B.: Region classification with markov field aspects models. In: CVPR (2007)
Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Graph Cut Based Inference with Co-occurrence Statistics. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 239–253. Springer, Heidelberg (2010)
Finlayson, G.D., Drew, M.S., Funt, B.V.: Color constancy: generalized diagonal transforms suffice. Journal of the Optical Society of America 11, 3011–3019 (1994)
Fredembach, C., Süsstrunk, S.: Colouring the Near-infrared. In: CIC (2008)
Schaul, L., Fredembach, C., Süsstrunk, S.: Color image dehazing using the Near-infrared. In: ICIP (2009)
Krishnan, D., Fergus, F.: Dark flash photography. In: SIGGRAPH (2009)
Brown, M., Süsstrunk, S.: Multispectral SIFT for scene category recognition. In: CVPR (2011)
Salamati, N., Larlus, D., Csurka, G.: Combining visible and Near-infrared cues for image categorisation. In: BMVC (2011)
Salamati, N., Fredembach, C., Süsstrunk, S.: Material classification using color and NIR images. In: CIC (2009)
Zhou, W., Huang, G., Troy, A., Cadenasso, M.L.: Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: A comparison study. Remote Sensing of Environment 113, 1769–1777 (2009)
Zhang, X., Sim, T., Miao, X.: Enhancing photographs with NIR images. In: CVPR (2008)
Kermani, Z., Lu, Y., Süsstrunk, S.: Correlation-based joint acquisition and demosaicing of visible and Near-infrared images. In: ICIP (2011)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: ICML (2001)
Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV (2004)
Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: ECCV SLCV Workshop (2004)
Perronnin, F., Dance, C.: Fisher kernels on visual vocabularies for image categorization. In: CVPR (2007)
Salamati, N., Süsstrunk, S.: Material-based object segmentation using Near-infrared Information. In: CIC (2010)
Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. In: SIGGRAPH (2004)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE TPAMI 26, 1124–1137 (2004)
Kolmogorov, V., Zabih, R.: What energy functions can be minimized via Graph Cuts? IEEE TPAMI 26, 147–159 (2004)
van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. IEEE TPAMI 32, 1582–1596 (2010)
Kohli, P., Ladický, L., Torr, P.H.S.: Robust higher order potentials for enforcing label consistency. IJCV 82, 302–324 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Salamati, N., Larlus, D., Csurka, G., Süsstrunk, S. (2012). Semantic Image Segmentation Using Visible and Near-Infrared Channels. In: Fusiello, A., Murino, V., Cucchiara, R. (eds) Computer Vision – ECCV 2012. Workshops and Demonstrations. ECCV 2012. Lecture Notes in Computer Science, vol 7584. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33868-7_46
Download citation
DOI: https://doi.org/10.1007/978-3-642-33868-7_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33867-0
Online ISBN: 978-3-642-33868-7
eBook Packages: Computer ScienceComputer Science (R0)