Abstract
In this paper, we reconsider the clustering problem for image over-segmentation from a new perspective. We propose a novel search algorithm called “active search” which explicitly considers neighbor continuity. Based on this search method, we design a back-and-forth traversal strategy and a joint assignment and update step to speed up the algorithm. Compared to earlier methods, such as simple linear iterative clustering (SLIC) and its variants, which use fixed search regions and perform the assignment and the update steps separately, our novel scheme reduces the number of iterations required for convergence, and also provides better boundaries in the over-segmentation results. Extensive evaluation using the Berkeley segmentation benchmark verifies that our method outperforms competing methods under various evaluation metrics. In particular, our method is fastest, achieving approximately 30 fps for a 481 × 321 image on a single CPU core. To facilitate further research, our code is made publicly available.
Article PDF
Similar content being viewed by others
References
Cheng, M.-M.; Liu, Y.; Hou, Q.; Bian, J.; Torr, P.; Hu, S.-M.; Tu, Z. HFS: Hierarchical feature selection for efficientimage segmentation. In: Computer Vision–ECCV 2016. Lecture Notes in Computer Science, Vol. 9907. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 867–882, 2016.
Wang, Z.; Feng, J.; Yan, S.; Xi, H. Image classificationvia object-aware holistic superpixel selection. IEEE Transactions on Image Processing Vol. 22, No. 11, 4341–4352, 2013.
Hoiem, D.; Efros, A. A.; Hebert, M. Automatic photopop-up. ACM Transactions on Graphics Vol. 24, No. 3, 577–584, 2005.
Wang, S.; Lu, H.; Yang, F.; Yang, M.-H. Superpixeltracking. In: Proceedings of the IEEE International Conference on ComputerVision, 1323–1330, 2011.
Felzenszwalb, P. F.; Huttenlocher, D. P. Efficient-graph-based image segmentation. International Journal of Computer Vision Vol. 59, No. 2, 167–181, 2004.
Comaniciu, D.; Meer, P. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 24, No. 5, 603–619, 2002.
Vincent, L.; Soille, P. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 13, No. 6, 583–598, 1991.
Shi, J.; Malik, J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 22, No. 8, 888–905, 2000.
Boykov, Y.; Veksler, O.; Zabih, R. Fast approxi-mate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 23, No. 11, 1222–1239, 2001.
Veksler, O.; Boykov, Y.; Mehrani, P. Superpixels and supervoxels in an energy optimization framework. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6315. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 211–224, 2010.
Boykov, Y.; Kolmogorov, V. An experimentalcomparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 26, No. 9, 1124–1137, 2004.
Kolmogorov, V.; Zabin, R. What energy functions can be minimized via graph cuts? IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 26, No. 2, 147–159, 2004.
Levinshtein, A.; Stere, A.; Kutulakos, K. N.; Fleet, D. J.; Dickinson, S. J.; Siddiqi, K. TurboPixels: Fast superpixels using geometric flows. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 31, No. 12, 2290–2297, 2009.
Osher, S.; Sethian, J. A. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics Vol. 79, No. 1, 12–49, 1988.
Van den Bergh, M.; Boix, X.; Roig, G.; de Capitani, B.; Van Gool, L. SEEDS: Superpixels extracted via energy-drivensampling. In: Computer Vision–ECCV 2012. Lecture Notes in Computer Science, Vol. 7578. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 13–26, 2012.
Liu, M.-Y.; Tuzel, O.; Ramalingam, S.; Chellappa, R. Entropy rate superpixel segmentation. In: Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition, 2097–2104, 2011.
Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; S¨usstrunk, S. SLIC superpixels compared to stateof- the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274–2282, 2012.
Lloyd, S. P. Least squares quantization in PCM. IEEE Transactions on Information Theory Vol. 28, No. 2, 129–137, 1982.
Wang, P.; Zeng, G.; Gan, R.; Wang, J.; Zha, H. Structure-sensitive superpixels via geodesic distance. International Journal of Computer Vision Vol. 103, No. 1, 1–21, 2013.
Peyr´e, G.; P´echaud, M.; Keriven, R.; Cohen, L. D. Geodesic methods in computer vision and graphics. Foundations and Trends in Computer Graphics and Vision Vol. 5, Nos. 3–4, 197–397, 2010.
Liu, Y.-J.; Yu, C.-C.; Yu, M.-J.; He, Y. Manifold SLIC: A fast method to compute content-sensitive superpixels. In: Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition, 651–659, 2016.
Du, Q.; Faber, V.; Gunzburger, M. Centroidal Voronoi tessellations: Applications and algorithms. SIAM Review Vol. 41, No. 4, 637–676, 1999.
Barnes, C.; Shechtman, E.; Finkelstein, A.; Goldman, D. B. PatchMatch: A randomized correspondence algorithm forstructural image editing. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 24, 2009.
Arbelaez, P.; Maire, M.; Fowlkes, C.; Malik, J. Contourdetection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 5, 898–916, 2011.
Stutz, D. Superpixelsegmentation: An evaluation. In: Pattern Recognition. Lecture Notes in Computer Science, Vol. 9358. Gall, J.; Gehler, P.; Leibe, B. Eds. Springer Cham, 555–562, 2015.
Mottaghi, R.; Chen, X.; Liu, X.; Cho, N.-G.; Lee, S.-W.; Fidler, S.; Urtasun, R.; Yuille, A. The role of contextfor object detection and semantic segmentation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 891–898, 2014.
Everingham, M.; Van Gool, L.; Williams, C. K. I.; Winn, J.; Zisserman, A. The PASCAL visual object classes challenge 2010 (VOC2010) results. 2010. Available at https://doi.org/www.pascalnetwork.org/challenges/VOC/voc2010/workshop/index.html.
Neubert, P.; Protzel, P. Superpixel benchmark and comparison. In: Proceedings of the Forum Bildverarbeitung, 1–12, 2012.
Acknowledgements
This research was sponsored by National Natural Science Foundation of China (Nos. 61620106008 and 61572264), Huawei Innovation Research Program (HIRP), and IBM Global SUR Award.
Author information
Authors and Affiliations
Corresponding author
Additional information
Jiaxing Zhao is a master student at CCCE&CS, Nankai University (Tianjin, China). He received his bachelor degree from Nankai University in 2017. His research interest includes computer vision and machine learning (especially deep learning).
Bo Ren is a lecturer at CCCE&CS, Nankai University (Tianjin, China). He received his B.S. and Ph.D. degrees from Tsinghua University (Beijing, China) in 2010 and 2015, respectively. His main research interest is in physically-based simulation and rendering, and geometry for computer graphics.
Qibin Hou is at present a first-year Ph.D. student at CCCE&CS, Nankai University (Tianjin, China). Before joining in the media group at Nankai University, he was a machine learning engineer in Baidu. His research interests include low-level vision, deep learning, and multimedia applications.
Ming-Ming Cheng is a professor with College of Computer Science, Nankai University, leading the Media Computing Lab. He received his Ph.D. degree from Tsinghua University in 2012. Then he worked as a research fellow for 2 years, working with Prof. Philip Torr in Oxford. Dr. Cheng’s research primarily centers on algorithmic issues in image understanding and processing, including image segmentation, editing, retrieval, etc. He has published over 30 papers in leading journals and conferences, such as IEEE TPAMI, ACM TOG, ACM SIGGRAPH, IEEE CVPR, and IEEE ICCV. He has designed a series of popular methods and novel systems, indicated by 9000+ paper citations (2000+ citations to his first author paper on salient object detection). He received several research awards, including ACM China Rising Star Award, the IBM Global SUR award, and the CCF-Intel Young Faculty Award. His work has been reported by several famous international media, such as BBC, UK Telegraph, Der Spiegel, and Huffington Post.
Paul Rosin gained his B.S. degree in computer science and microprocessor systems in 1984 from Strathclyde University, Glasgow, and Ph.D. degree in information engineering from City University, London, in 1988. He was a research fellow at City University, developing a prototype system for the home office to detect and classify intruders in image sequences. He worked on the Alvey project “Model-Based Interpretation of Radiological Images” at Guy’s Hospital, London, before becoming a lecturer at the Department of Computer Science, Curtin University of Technology, Perth, Australia. He moved to Italy to work at the Institute for Remote Sensing Applications at the Joint Research Centre, followed by a return to the UK, becoming lecturer at the Department of Information Systems and Computing, Brunel University, London. In 2000, he moved to the School of Computer Science & Informatics, Cardiff University.
Rights and permissions
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (https://doi.org/creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from https://doi.org/www.springer.com/journal/41095. To submit a manuscript, please go to https://doi.org/www.editorialmanager.com/cvmj.
About this article
Cite this article
Zhao, J., Bo, R., Hou, Q. et al. FLIC: Fast linear iterative clustering with active search. Comp. Visual Media 4, 333–348 (2018). https://doi.org/10.1007/s41095-018-0123-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-018-0123-y