Skip to main content
SpringerLink
Log in

A parallel LEGION algorithm and cell-based architecture for real time split and merge video segmentation

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Split and merge segmentation is a popular region-based segmentation scheme for its robustness and computational efficiency. But it is hard to realize for larger size images or video frames in real time due to its iterative sequential data flow pattern. A quad-tree data structure is quite popular for software implementation of the algorithm, where a local parallelism is difficult to establish due to inherent data dependency between processes. In this paper, we have proposed a parallel algorithm of splitting and merging which depends only on local operations. The algorithm is mapped onto a hierarchical cell network, which is a parallel version of Locally Excitory Globally Inhibitory Oscillatory Network (LEGION). Simulation results show that the proposed design is faster than any of the standard split and merge algorithmic implementations, without compromising segmentation quality. The timing performance enhancement is manifested in its Finite State Machine based VLSI implementation in VIRTEX series FPGA platforms. We have also shown that, though segmentation qualitywise split-and-merge algorithm is little bit behind the state-of-the-art algorithms, computational speedwise it over performs those sophisticated and complex algorithms. Good segmentation performance with minimal computational cost enables the proposed design to tackle real time segmentation problem in live video streams. In this paper, we have demonstrated live PAL video segmentation using VIRTEX 5 series FPGA. Moreover, we have extended our design to HD resolution for which the time taken is less than 5 ms rendering a processing throughput of 200 frames per second.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Associative hierarchical CRFs for object class image segmentation. In: IEEE 12 th International Conference on Computer Vision (ICCV 2009), pp. 739–746 (2009)

  2. Lecumberry, F., Pardo, A., Sapiro, G.: Simultaneous object classification and segmentation with high-order multiple shape models. IEEE Trans. Image Process. 19(3), 625–635 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  3. Peng, B., Zhang, D., Zhang, D.: Automatic image segmentation by dynamic region merging. IEEE Trans. Image Process. 20(12), 3592–3605 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Kimmel, R., Bruckstein, A.M.: Regularized Laplacian zero crossings and other optimal edge integrators. Int. J. Comput. Vis. 53(3), 225–243 (2003)

    Article  Google Scholar 

  5. Senthilkumaran, N., Rajesh, R.: Edge detection techniques for image segmentation—a survey of soft computing approaches. Int. J. Recent Trends Eng. 1(2), (2009)

  6. Gonzalez, R.C., Woods, R.E.: Digital image processing, 3rd edn. Prentice Hall, NJ, USA (2008)

    Google Scholar 

  7. Pratt, W.K.: Digital Image Processing, 4th edn. John Wiley & Sons Inc, Los Altos (2007)

    Book  MATH  Google Scholar 

  8. Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–598 (1991)

    Article  Google Scholar 

  9. Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994)

    Article  Google Scholar 

  10. Hunter, G.M., Steiglitz, K.S.: Operations on images using quad- tree. IEEE Trans. Pattern Anal. Mach. Intell. 1(2), 145–153 (1979)

    Article  Google Scholar 

  11. Haris, K., Efstratiadis, S.N., Maglaveras, N., Katsaggelos, A.K.: Hybrid image segmentation using watersheds and fast region merging. IEEE Trans. Image Process. 7(12), 1684–1699 (1998)

    Article  Google Scholar 

  12. Chen, S., Lin, W., Chen, C.: Split-and-merge image segmentation based on localized feature analysis and statistical tests. J CVGIP Gr Models Image Process. 53(5), 457–475 (1991)

    Article  Google Scholar 

  13. Serratosa, F., Millan, P.,Montseny, E.: Systolic processors applied to computer vision systems. In: Proceedings of the IEEE Computer Architectures for Machine Perception, (CAMP’95), pp 178–183 (1995)

  14. Jonker, P.P.: An SIMD-MIMD architecture for image processing and pattern recognition. In: Proceedings of the IEEE Computer Architectures for Machine Perception, (CAMP’93), pp 222–230 (1993)

  15. Masaki, I.: Parallel/pipelined processor dedicated to visual recognition. In Proceed. IEEE Int. Conf. Robot. Autom. 2, 100–107 (1985)

    Google Scholar 

  16. Kim, J., Chen, T.: A VLSI architecture for video-object segmentation. IEEE Trans. Circuits Syst. Video Technol. 13(1), 83–96 (2003)

    Article  Google Scholar 

  17. Arena, P., Basile, A., Bucolo, M., Fortuna, L.: An object oriented segmentation on analog CNN chip. IEEE Trans. Circuits Syst. I 50(7), 837–846 (2003)

    Article  Google Scholar 

  18. Chen, T., Sun, C., Su, H., Chien, S., Deguchi, D., Ide, I., Murase, H.: Power-efficient hardware architecture of K-means clustering with Bayesian-information-criterion processor for multimedia processing applications. IEEE J. Emerg. Sel. Topics Circuits Syst. 1(3), 357–368 (2011)

    Article  Google Scholar 

  19. Sirotkovic, J., Dujmic, H., Papic, V.: K-means image segmentation on massively parallel GPU architecture. In: Proceedings of the IEEE 35th International Convention MIPRO, 2012, pp. 489–484 (2012)

  20. Horowitz, S., Pavlidis, T.: Picture segmentation by a tree traversal algorithm. J. ACM. 23, 368–388 (1976)

    Article  MATH  Google Scholar 

  21. Kunlin, T., Lacassagne, L., Merigot, A.: A fast image segmentation scheme. In: Proceedings of the International Conference on Information and Communication Technologies: From Theory to Applications, (ICTTA2004), pp. 351–352 (2004)

  22. Zhang, Z., Chen, C., Sun, J., Chan, K.L.: EM algorithms for Gaussian mixtures with split-and-merge operation. Pattern Recognit. 36(9), 1973–1983 (2003)

    Article  MATH  Google Scholar 

  23. Panagiotakis, C., Grinias, I., Tziritas, G.: Natural image segmentation based on tree equipartition, Bayesian flooding and region merging. IEEE Trans. Image Process. 20(8), 2276–2287 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  24. Willebeek-LeMair, M., Reeves, A.P.: Solving nonuniform problems on SIMD computers: case study on region growing. J. Parallel Distrib. Comput. 8(2), 135–149 (1990)

    Article  Google Scholar 

  25. Montoya, M.D.G., Gil, C., Garcia, I.: The load unbalancing problem for region growing image segmentation algorithms. J. Parallel Distrib. Comput. 63(4), 387–395 (2003)

    Article  MATH  Google Scholar 

  26. Xuejing, G., Linlin, C., Kangze, Y.: Two parallel strategies of split-merge algorithm for image segmentation. IEEE Int. Conf. Wavelet Anal Pattern Recognit. 2, 840–845 (2007)

    Article  Google Scholar 

  27. Roy, P., Das, D., Biswas, P.K.: Real time VLSI implementation of a fast split and merge segmentation algorithm. In: Proceedings of the IEEE International Conference on Computational Intelligence and Computing Research (ICCIC 2012), pp. 114–121 (2012)

  28. Faruquzzaman, A. B. M., Paiker, N. R., Arafat, J., Ali, M. A., Sorwar, G.: Literature on image segmentation based on split and merge techniques. In: Proceedings of the 5th International Conference on Information Technology and Applications (ICITA 2008), pp. 120–125 (2008)

  29. Prabhashankar, L.: Parallel implementation of the split and merge algorithm on the hypercube machine. MS Thesis, Ohio University (1989)

  30. Tyagi, A., Bayoumi, M.A.: Image segmentation on a 2D array by a directed split and merge procedure. IEEE Trans. Signal Process. 40(11), 2804–2813 (1992)

    Article  Google Scholar 

  31. Copty, N., Ranka, S., Fox, G., Shankar, R.: Solving the region growing problem on the connection machine. In Proceed. Int. Conf. Parallel Process. 3, 102–105 (1993)

    Google Scholar 

  32. Reyes, A, Martinez, V., Ramirez, A.: Parallel design the algorithm of digital images segmentation split and merge. In: Conferencia Latinoamericana de Computación de Alto Rendimiento, vol. 1, pp. 85–89 (2009)

  33. Wang, D., Terman, D.: Image segmentation based on oscillatory correlation. J. Neural Comput. 9(4), 805–836 (1997)

    Article  Google Scholar 

  34. Ando, H., Miyake, M., Morie, T., Nagata, M., Iwata, A.: A nonlinear oscillator network circuit for image segmentation with double- threshold phase detection. IEEE Ninth Int. Conf. Artif Neural Netw. 2, 655–660 (1999)

    Article  Google Scholar 

  35. Morimoto, T., Harada, Y., Koide, T., Mattausch, H. J.: Pixel-parallel digital CMOS implementation of image segmentation by region growing. IEE Proceed. Circuits Devices Syst. 152(6), 579–589 (2005)

    Article  Google Scholar 

  36. Roy P., Biswas P. K.: VLSI implementation of fast connected component labeling using finite state machine based cell network. In: Proceedings of the Sixth Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP’ 08), pp. 238–243 (2008)

  37. Fernandes, D., Stedile, J., Navaux, P.: Architecture of oscillatory neural network for image segmentation. In: IEEE 14th Symposium on Computer Architecture and High Performance Computing (CAHPC 02), pp. 29–36 (2002).

  38. http://www.xilinx.com

  39. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the eighth IEEE International Conference on Computer Vision (ICCV01), vol. 2, pp. 416–423 (2001)

  40. Kranthi Kumar, J. D., Srinivasan, S.: A novel VLSI architecture to implement region merging algorithm for image segmentation. In: IEEE Euromicro Symposium on Digital System Design, pp. 620–623 (2004)

  41. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  42. Khan, S.K., Ahmad, A.: Cluster center initialization algorithm for k-means clustering. Pattern Recogn. Lett. 25, 1293–1302 (2004)

    Article  Google Scholar 

  43. Pham, D.L., Prince, J.L.: An adaptive fuzzy C-means algorithm for image segmentation in the presence of intensity inhomogeneities. Pattern Recognit. Lett. 20(1), 57–68 (1999)

    Article  MATH  Google Scholar 

  44. Wanga, X., Wanga, T., Bu, J.: Color image segmentation using pixel wise support vector machine classification. Pattern Recognit. 44(4), 777–787 (2011)

    Article  MATH  Google Scholar 

  45. Deng, H., Clausi, D.A.: Unsupervised image segmentation using a simple MRF model with a new implementation scheme. In proceed. 17th Int. Conf Pattern Recognit. 2, 691–694 (2004)

    Article  Google Scholar 

  46. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)

    Article  Google Scholar 

  47. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Article  MATH  Google Scholar 

  48. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  49. Li, K., Song X.: A fast large size image segmentation algorithm based on spectral clustering. In: IEEE Fourth International Conference on Computational and Information Sciences (ICCIS 2012), pp. 345–348 (2012)

  50. Feng, W., Jia, J., Liu, Z.: Self-validated labeling of Markov random fields for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1871–1887 (2010)

    Article  Google Scholar 

  51. Garrett, Z., Saito, H.: Live video object tracking and segmentation using graph cuts. In: 15th IEEE International Conference on Image Processing (ICIP 2008), pp. 1576–1579 (2008)

  52. Liu, J., Sun J.: Parallel graph-cuts by adaptive bottom-up merging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2010), pp. 2181–2188 (2010)

  53. Kobori, D., Maruyama, T.: An acceleration of a graph cut segmentation with FPGA. In: 22nd International Conference on Field Programmable Logic and Applications (FPL 2012), pp. 407–413 (2012)

  54. http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

  55. Yang, A.Y., Wright, J., Ma, Y., Sastry, S.S.: Unsupervised segmentation of natural images via lossy data compression. Comput. Vis. Image Underst. 110(2), 212–225 (2008)

    Article  Google Scholar 

  56. Vineet, V., Narayanan, P.J.: CUDA cuts: fast graph cuts on the GPU. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 08), pp. 1–8 (2008)

Download references

Author information

Authors and Affiliations

  1. Optronics Centre, Integrated Test Range, Chandipur, 756025, India

    Pradipta Roy

  2. Department of E and ECE, Indian Institute of Technology, Kharagpur, India

    Prabir Kumar Biswas

Authors
  1. Pradipta Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prabir Kumar Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pradipta Roy.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roy, P., Biswas, P.K. A parallel LEGION algorithm and cell-based architecture for real time split and merge video segmentation. J Real-Time Image Proc 15, 363–387 (2018). https://doi.org/10.1007/s11554-015-0488-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-015-0488-4

Keywords

Search

Navigation