Skip to main content
SpringerLink
Log in

Multivariate bounded support Laplace mixture model

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, bounded Laplace mixture model (BLMM) is proposed. The parameters of proposed model are estimated by maximum likelihood approach via expectation maximization and Newton–Raphson algorithm. The model is proposed for data modeling to perform clustering using synthetic data for univariate and multivariate examples and real datasets of different medical experiments. BLMM is validated through correctness of estimated parameters for synthetic data and clustering accuracy of medical datasets. A new modeling scheme is also introduced for wavelet coefficients which is based on BLMM. It is applied to image clustering and content-based image retrieval (CBIR) for feature extraction in wavelet domain. For feature extraction in this application, each image is decomposed into a set of wavelet subspaces and BLMM with two components is adopted to model the statistical characteristics of the wavelet coefficients for each wavelet subspace. The model parameters adapted from BLMM represent the image features in wavelet domain for each subspace and selected to formulate the feature space which is further used in clustering and CBIR. In the framework for clustering and image retrieval, features extracted in wavelet domain are further modeled through BLMM to categorize images into different groups and trained model is adopted for CBIR. In order to perform image retrieval with trained model via BLMM, city block distance, posterior probability and Kullback–Leibler divergence are introduced. We also propose a novel solution to compute Kullback–Leibler divergence which is very effective for image retrieval due to its low computational complexity and high retrieval rate. The effectiveness and viability of BLMM in texture image clustering and CBIR are demonstrated through UIUC, KTH-TIPS, DTD, STex and Kylberg databases. Different experiments are performed in the chosen applications, and from the results, BLMM has demonstrated its effectiveness in modeling synthetic data, real datasets from medical experiments, feature extraction in wavelet domain, image clustering and CBIR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  • Allili MS, Bouguila N, Ziou D (2008) Finite general Gaussian mixture modeling and application to image and video foreground segmentation. J Electron Imaging 17(1):013005

    Article  Google Scholar 

  • Amin T, Zeytinoglu M, Guan L (2007) Application of Laplacian mixture model to image and video retrieval. IEEE Trans Multimed 9(7):1416–1429

    Article  Google Scholar 

  • Andrearczyk V, Whelan PF (2017) Deep learning in texture analysis and its application to tissue image classification. In: Depeursinge A, Al-Kadi OS, Mitchell JR (eds) Biomedical texture analysis. Elsevier, Amsterdam, pp 95–129

    Chapter  Google Scholar 

  • Azam M, Bouguila N (2018) Bounded Laplace Mixture Model with Applications to Image Clustering and Content Based Image Retrieval. In: 2018 17th IEEE international conference on machine learning and applications (ICMLA), pp 558–563, https://doi.org/10.1109/ICMLA.2018.00090

  • Azam M, Bouguila N (2019) Bounded generalized gaussian mixture model with ICA. Neural Process Lett 49(3):1299–1320. https://doi.org/10.1007/s11063-018-9868-7

    Article  Google Scholar 

  • Bai S, Bai X, Tian Q, Latecki LJ (2018) Regularized diffusion process on bidirectional context for object retrieval. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2018.2828815

    Article  Google Scholar 

  • Banerjee I, Kurtz C, Devorah AE, Do B, Rubin DL, Beaulieu CF (2018) Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: application to bone tumor radiographs. J Biomed Inform 84:123–135. https://doi.org/10.1016/j.jbi.2018.07.002

    Article  Google Scholar 

  • Banfield JD, Raftery AE (1993) Model-based Gaussian and non-Gaussian clustering. Biometrics 49:803–821

    Article  MathSciNet  MATH  Google Scholar 

  • Bhowmick D, Davison A, Goldstein DR, Ruffieux Y (2006) A Laplace mixture model for identification of differential expression in microarray experiments. Biostatistics 7(4):630–641

    Article  MATH  Google Scholar 

  • Bradley PS, Mangasarian OL, Street WN (1997) Clustering via concave minimization. In: Advances in neural information processing systems, pp 368–374

  • Cheeseman PC, Stutz JC et al (1996) Bayesian classification (autoclass): theory and results. Adv Knowl Discov Data Min 180:153–180

    MATH  Google Scholar 

  • Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5(Aug):913–939

    MathSciNet  Google Scholar 

  • Chipman HA, Kolaczyk ED, McCulloch RE (1997) Adaptive Bayesian wavelet shrinkage. J Am Stat Assoc 92(440):1413–1421

    Article  MATH  Google Scholar 

  • Choy S, Tong C (2010) Statistical wavelet subband characterization based on generalized gamma density and its application in texture retrieval. IEEE Trans Image Process 19(2):281–289. https://doi.org/10.1109/TIP.2009.2033400

    Article  MathSciNet  MATH  Google Scholar 

  • Cimpoi M, Maji S, Kokkinos I, Mohamed S, , Vedaldi A (2014) Describing textures in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  • Cord A, Ambroise C, Cocquerez JP (2006) Feature selection in robust clustering based on Laplace mixture. Pattern Recognit Lett 27(6):627–635

    Article  Google Scholar 

  • Dang VM (1998) Classification de données spatiales: modèles probabilistes et critères de partitionnement. PhD thesis, Compiègne

  • Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  • Elguebaly T, Bouguila N (2014) Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection. Mach Vis Appl 25(5):1145–1162

    Article  Google Scholar 

  • Elter M, Schulz-Wendtland R, Wittenberg T (2007) The prediction of breast cancer biopsy outcomes using two cad approaches that both emphasize an intelligible decision process. Med Phys 34(11):4164–4172

    Article  Google Scholar 

  • Eltoft T, Kim T, Lee TW (2006) On the multivariate Laplace distribution. IEEE Signal Process Lett 13(5):300–303

    Article  Google Scholar 

  • Ernst MD (1998) A multivariate generalized Laplace distribution. Comput Stat 13:227–232

    MathSciNet  MATH  Google Scholar 

  • Espindola R, Ebecken N (2005) On extending f-measure and g-mean metrics to multi-class problems. WIT Transactions on Information and Communication Technologies 35

  • Farag A, El-Baz A, Gimel’farb G (2006) Precise segmentation of multimodal images. IEEE Trans Image Process 15(4):952–968. https://doi.org/10.1109/TIP.2005.863949

    Article  Google Scholar 

  • Figueiredo MA, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396

    Article  Google Scholar 

  • Franczak BC, Browne RP, McNicholas PD (2014) Mixtures of shifted asymmetric Laplace distributions. IEEE Trans Pattern Anal Mach Intell 36(6):1149–1157. https://doi.org/10.1109/TPAMI.2013.216

    Article  Google Scholar 

  • Fritz M, Hayman E, Caputo B, olof Eklundh J (2004) The kth-tips database

  • Gai S, Zhang B, Yang C, Yu L (2018) Speckle noise reduction in medical ultrasound image using monogenic wavelet and Laplace mixture distribution. Digital Signal Process 72:192–207. https://doi.org/10.1016/j.dsp.2017.10.006

    Article  Google Scholar 

  • Garg L, McClean S, Meenan B, El-Darzi E, Millard P (2009) Clustering patient length of stay using mixtures of Gaussian models and phase type distributions. In: 2009 22nd IEEE international symposium on computer-based medical systems, pp 1–7, https://doi.org/10.1109/CBMS.2009.5255245

  • Gil D, Girela JL, De Juan J, Gomez-Torres MJ, Johnsson M (2012) Predicting seminal quality with artificial intelligence methods. Expert Syst Appl 39(16):12564–12573

    Article  Google Scholar 

  • Gorodkin J (2004) Comparing two k-category assignments by a k-category correlation coefficient. Comput Biol Chem 28(5–6):367–374

    Article  MATH  Google Scholar 

  • Hedelin P, Skoglund J (2000) Vector quantization based on gaussian mixture models. IEEE Trans Speech Audio Process 8(4):385–401. https://doi.org/10.1109/89.848220

    Article  Google Scholar 

  • Hedelin P, Skoglund J, Samuelsson J (1999) Performance bounds for lpc spectrum quantization. In: 1999 IEEE international conference on acoustics, speech, and signal processing. proceedings. ICASSP99 (Cat. No.99CH36258), vol 2, pp 677–680, https://doi.org/10.1109/ICASSP.1999.759757

  • Jurman G, Furlanello C (2010) A unifying view for performance measures in multi-class prediction. arXiv preprint arXiv:1008.2908

  • Jurman G, Riccadonna S, Furlanello C (2012) A comparison of MCC and CEN error measures in multi-class prediction. PLoS ONE 7(8):e41882

    Article  Google Scholar 

  • Khozeimeh F, Alizadehsani R, Roshanzamir M, Khosravi A, Layegh P, Nahavandi S (2017a) An expert system for selecting wart treatment method. Comput Biol Med 81:167–175

    Article  Google Scholar 

  • Khozeimeh F, Jabbari Azad F, Mahboubi Oskouei Y, Jafari M, Tehranian S, Alizadehsani R, Layegh P (2017b) Intralesional immunotherapy compared to cryotherapy in the treatment of warts. Int J Dermatol 56(4):474–478

    Article  Google Scholar 

  • Kurgan LA, Cios KJ, Tadeusiewicz R, Ogiela M, Goodenday LS (2001) Knowledge discovery approach to automated cardiac spect diagnosis. Artif Intell Med 23(2):149–169

    Article  Google Scholar 

  • Kwitt R, Meerwald P (2019) Salzburg texture image database (STex). http://www.wavelab.at/sources/STex/. Accessed 15 Jan 2019

  • Kylberg G (2011) The Kylberg texture dataset v. 1.0. External report (Blue series) 35, Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University, Uppsala, Sweden. http://www.cb.uu.se/~gustaf/texture/

  • Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166. https://doi.org/10.1109/TPAMI.2004.71

    Article  Google Scholar 

  • Lazebnik S, Schmid C, Ponce J (2005) A sparse texture representation using local affine regions. IEEE Trans Pattern Anal Mach Intell 27(8):1265–1278. https://doi.org/10.1109/TPAMI.2005.151

    Article  Google Scholar 

  • Lindblom J, Samuelsson J (2003) Bounded support Gaussian mixture modeling of speech spectra. IEEE Trans Speech Audio Process 11(1):88–99. https://doi.org/10.1109/TSA.2002.805639

    Article  Google Scholar 

  • Little MA, McSharry PE, Roberts SJ, Costello DA, Moroz IM (2007) Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. Biomed Eng Online 6(1):23

    Article  Google Scholar 

  • Malik F, Baharudin B (2013) Analysis of distance metrics in content-based image retrieval using statistical quantized histogram texture features in the DCT domain. J King Saud Univ Comput Inf Sci 25(2):207–218. https://doi.org/10.1016/j.jksuci.2012.11.004

    Article  Google Scholar 

  • Mayhew MB, Petersen BK, Sales AP, Greene JD, Liu VX, Wasson TS (2018) Flexible, cluster-based analysis of the electronic medical record of sepsis with composite mixture models. J Biomed Inform 78:33–42. https://doi.org/10.1016/j.jbi.2017.11.015

    Article  Google Scholar 

  • Mclachlan G, Basford K (1988) Mixture models: inference and applications to clustering. J R Stat Soc Ser C Appl Stat 38:384–385. https://doi.org/10.2307/2348072

    Article  MATH  Google Scholar 

  • McLachlan G, Krishnan T (2007) The EM algorithm and extensions, vol 382. Wiley, Hoboken

    MATH  Google Scholar 

  • McLachlan G, Peel D (2004) Finite mixture models. Wiley, Hoboken

    MATH  Google Scholar 

  • Medasani S, Krishnapuram R (1998) Categorization of image databases for efficient retrieval using robust mixture decomposition. In: Proceedings. IEEE workshop on content-based access of image and video libraries (Cat. No.98EX173), pp 50–54, https://doi.org/10.1109/IVL.1998.694495

  • Medasani S, Krishnapuram R (2001) Categorization of image databases for efficient retrieval using robust mixture decomposition. Comput Vis Image Underst 83(3):216–235. https://doi.org/10.1006/cviu.2001.0926

    Article  MATH  Google Scholar 

  • Mitianoudis N, Stathaki T (2005) Overcomplete source separation using Laplacian mixture models. IEEE Signal Process Lett 12(4):277–280. https://doi.org/10.1109/LSP.2005.843759

    Article  Google Scholar 

  • Najjar A, Gagne C, Reinharz D (2015) Two-step heterogeneous finite mixture model clustering for mining healthcare databases. In: 2015 IEEE international conference on data mining, pp 931–936, https://doi.org/10.1109/ICDM.2015.70

  • Neal RM, Hinton GE (1998) A view of the Em algorithm that justifies incremental, sparse, and other variants. In: Jordan MI (ed) Learning in graphical models. Springer, Berlin, pp 355–368

    Chapter  Google Scholar 

  • Nguyen H, Azam M, Bouguila N (2019a) Data clustering using variational learning of finite scaled dirichlet mixture models. In: 2019 IEEE 28th international symposium on industrial electronics (ISIE), pp 1391–1396, https://doi.org/10.1109/ISIE.2019.8781334

  • Nguyen H, Kalra M, Azam M, Bouguila N (2019b) Data clustering using online variational learning of finite scaled dirichlet mixture models. In: 2019 IEEE 20th international conference on information reuse and integration for data science (IRI), pp 267–274, https://doi.org/10.1109/IRI.2019.00050

  • Nguyen TM, Wu QJ, Zhang H (2014) Bounded generalized Gaussian mixture model. Pattern Recognit 47(9):3132–3142

    Article  MATH  Google Scholar 

  • Patrício M, Pereira J, Crisóstomo J, Matafome P, Gomes M, Seiça R, Caramelo F (2018) Using resistin, glucose, age and bmi to predict the presence of breast cancer. BMC Cancer 18(1):29

    Article  Google Scholar 

  • Peel D, McLachlan G (2000) Robust mixture modelling using the t distribution. Stat Comput 10(4):339–348

    Article  Google Scholar 

  • Rish I (2001) An empirical study of the naive Bayes classifier. Technical report

  • Romberg JK, Choi H, Baraniuk RG (2001) Bayesian tree-structured image modeling using wavelet-domain hidden Markov models. IEEE Trans Image Process 10(7):1056–1068. https://doi.org/10.1109/83.931100

    Article  Google Scholar 

  • Rui Y, Huang TS, Ortega M, Mehrotra S (1998) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655. https://doi.org/10.1109/76.718510

    Article  Google Scholar 

  • Scricciolo C (2018) Bayes and maximum likelihood for L-1-Wasserstein deconvolution of Laplace mixtures. Stat Methods Appl 27(2):333–362

    Article  MathSciNet  MATH  Google Scholar 

  • Shi F, Selesnick IW (2006) Multivariate quasi-Laplacian mixture models forwavelet-based image denoising. In: 2006 International Conference on Image Processing, pp 2625–2628, https://doi.org/10.1109/ICIP.2006.313048

  • Simoncelli EP (1997) Statistical models for images: Compression, restoration and synthesis. In: Conference record of the thirty-first Asilomar conference on signals, systems & computers, vol 1, pp 673–678. IEEE

  • Song Y, Li Q, Feng D, Zou JJ, Cai W (2016) Texture image classification with discriminative neural networks. Comput Vis Media 2(4):367–377

    Article  Google Scholar 

  • Subramaniam AD, Rao BD (2000) PDF optimized parametric vector quantization of speech line spectral frequencies. In: 2000 IEEE workshop on speech coding. proceedings. Meeting the challenges of the new millennium (Cat. No.00EX421), pp 87–89, https://doi.org/10.1109/SCFT.2000.878407

  • Sun J, Zhou A, Keates S, Liao S (2018) Simultaneous bayesian clustering and feature selection through student’s \({t}\) mixtures model. IEEE Trans Neural Netw Learn Syst 29(4):1187–1199. https://doi.org/10.1109/TNNLS.2016.2619061

    Google Scholar 

  • Titterington D, Smith A, Makov U (1985) Statistical analysis of finite mixture distributions. Wiley, New York

    MATH  Google Scholar 

  • Usevitch BE (2001) A tutorial on modern lossy wavelet image compression: foundations of jpeg 2000. IEEE Signal Process Mag 18(5):22–35. https://doi.org/10.1109/79.952803

    Article  Google Scholar 

  • Usevitch BE, Orchard MT (1995) Smooth wavelets, transform coding, and Markov-1 processes. IEEE Trans Signal Process 43(11):2561–2569. https://doi.org/10.1109/78.482107

    Article  Google Scholar 

  • Varma M, Zisserman A (2005) A statistical approach to texture classification from single images. Int J Comput Vis 62(1–2):61–81

    Article  Google Scholar 

  • Xu L, Jordan MI (1996) On convergence properties of the em algorithm for gaussian mixtures. Neural Comput 8(1):129–151

    Article  Google Scholar 

  • Yeh IC, Yang KJ, Ting TM (2009) Knowledge discovery on RFM model using Bernoulli sequence. Expert Syst Appl 36(3):5866–5871

    Article  Google Scholar 

  • Yuan H, Zhang XP (2004) Texture image retrieval based on a Gaussian mixture model and similarity measure using a Kullback divergence. In: 2004 IEEE international conference on multimedia and expo (ICME) (IEEE Cat. No.04TH8763), vol 3, pp 1867–1870, https://doi.org/10.1109/ICME.2004.1394622

  • Yuan H, Zhang XP (2010) Statistical modeling in the wavelet domain for compact feature extraction and similarity measure of images. IEEE Trans Circuits Syst Video Technol 20(3):439–445. https://doi.org/10.1109/TCSVT.2009.2031396

    Article  Google Scholar 

  • Yuan H, Zhang XP, Guan L (2003) Content-based image retrieval using a Gaussian mixture model in the wavelet domain. In: Proceedings of SPIE, vol 5150, pp 422–429, https://doi.org/10.1117/12.503262

  • Zheng L, Yang Y, Tian Q (2018) Sift meets CNN: a decade survey of instance retrieval. IEEE Trans Pattern Anal Mach Intell 40(5):1224–1244. https://doi.org/10.1109/TPAMI.2017.2709749

    Article  Google Scholar 

Download references

Acknowledgements

The completion of this research was made possible thanks to the Natural Sciences and Engineering Research Council of Canada (NSERC).

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada

    Muhammad Azam

  2. Concordia Institute for Information Systems Engineering, Concordia University, Montreal, QC, Canada

    Nizar Bouguila

Authors
  1. Muhammad Azam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nizar Bouguila
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nizar Bouguila.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azam, M., Bouguila, N. Multivariate bounded support Laplace mixture model. Soft Comput 24, 13239–13268 (2020). https://doi.org/10.1007/s00500-020-04737-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-04737-7

Keywords

Search

Navigation