Skip to main content
SpringerLink
Log in

Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the area of machine learning and pattern recognition, object classification is getting an attraction due to its range of applications such as visual surveillance. In recent times, numerous deep learning-based methods are presented for object classification but still, set of problems/concerns exists which reduce the overall classification accuracy. Complex background, congest situtaions, and similarity among different objects are few challenging issues. To tackle such problems, we propose a technique by using deep convolutional neural network (DCNN) and scale invariant features transform (SIFT). First, an improved saliency method is implemented, and the point features are extracted. Then, DCNN features are extracted from two deep CNN models like VGG and AlexNet. Thereafter, Reyni entropy-controlled method is implemented on DCNN pooling and the SIFT point matrix to select the robust features. Finally, the selected robust features are fused in a matrix by a serial approach, which is later fed to ensemble classifier for recognition. The proposed method is evaluated on three publically available datasets including Caltech101, Barkley 3D, and Pascal 3D and obtained classification accuracy of 93.8%, 99%, and 88.6% - clearly showing the exceptional performance compared to existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta R et al (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  2. Agrawal S et al (2018) A comparative study of fuzzy PSO and fuzzy SVD-based RBF neural network for multi-label classification. Neural Comput & Applic 29(1):245–256

    Article  Google Scholar 

  3. Akcay S, et al (2018) Using deep convolutional neural network architectures for object classification and detection within X-ray baggage security imagery. IEEE Trans Inf Forensics Secur

  4. Akram T, et al (2018) Skin lesion segmentation and recognition using multichannel saliency estimation and M-SVM on selected serially fused features. J Ambient Intell Humaniz Comput:1–20

  5. Arel I, Rose DC, Karnowski TP (2010) Deep machine learning-a new frontier in artificial intelligence research [research frontier]. IEEE Comput Intell Mag 5(4):13–18

    Article  Google Scholar 

  6. Chen S et al (2018) Local patch vectors encoded by fisher vectors for image classification. Information 9(2):38

    Article  MathSciNet  Google Scholar 

  7. Cheng G et al (2016) Study on planetary gear fault diagnosis based on entropy feature fusion of ensemble empirical mode decomposition. Measurement 91:140–154

    Article  Google Scholar 

  8. Dong H, et al (2018) A novel hybrid genetic algorithm with granular information for feature selection and optimization. Appl Soft Comput

  9. Ejbali R, Zaied M (2018) A dyadic multi-resolution deep convolutional neural wavelet network for image classification. Multimed Tools Appl 77(5):6149–6163

    Article  Google Scholar 

  10. Esteva A et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115

    Article  Google Scholar 

  11. Everingham, M., et al., The pascal visual object classes challenge: a retrospective. Int J Comput Vis, 2015. 111(1): p. 98–136

  12. F.a.F. (2018) https://melanoma.canceraustralia.gov.au/statistics

  13. Fang X, et al (2018) Approximate low-rank projection learning for feature extraction. IEEE Transactions on Neural Networks and Learning Systems

  14. Fei-Fei L, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4):594–611

    Article  Google Scholar 

  15. Fondón I, et al (2018) Automatic classification of tissue malignancy for breast carcinoma diagnosis. J Comput Biol Med

  16. Ghose U, Mehta R (2018) Attribute reduction method using the combination of entropy and fuzzy entropy. In: Networking Communication and Data Knowledge Engineering. Springer, p 169–177

  17. Gomathi D, Seetharaman K Object classification techniques using tree based classifiers

  18. Grabner A, Roth PM, Lepetit V (2018) 3d pose estimation and 3d model retrieval for objects in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  19. He K, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  20. Hu F et al (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens 7(11):14680–14707

    Article  Google Scholar 

  21. Janoch A, et al (2013) A category-level 3d object dataset: putting the kinect to work. In: Consumer depth cameras for computer vision. Springer, p 141–165

  22. Juuti M, Corona F, Karhunen J (2018) Stochastic discriminant analysis for linear supervised dimension reduction. Neurocomputing

  23. Khan MA et al (2017) License number plate recognition system using entropy-based features selection approach with SVM. IET Image Process 12(2):200–209

    Article  Google Scholar 

  24. Khan MA, et al (2018) An implementation of optimized framework for action classification using multilayers neural network on selected fused features. Pattern Anal Applic:1–21

  25. Khan MA et al (2018) CCDF: automatic system for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features. Comput Electron Agric 155:220–236

    Article  Google Scholar 

  26. Khan MA et al (2018) An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification. BMC Cancer 18(1):638

    Article  Google Scholar 

  27. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems

  28. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Computer vision and pattern recognition, 2006 IEEE computer society conference on. IEEE

  29. Leng L, et al (2010) Dynamic weighted discrimination power analysis in DCT domain for face and palmprint recognition. In: Information and Communication Technology Convergence (ICTC), 2010 International Conference on. IEEE

  30. Leng L, Zhang J, Khan MK, Chen X, Alghathbar K (2010) Dynamic weighted discrimination power analysis: a novel approach for face and palmprint recognition in DCT domain. International Journal of Physical Sciences 5(17):2543–2554

  31. Leng L, et al (2011) Two dimensional PalmPhasor enhanced by multi-orientation score level fusion. In: FTRA International Conference on Secure and Trust Computing, Data Management, and Application. Springer

  32. Leng L, et al (2011) Two-directional two-dimensional random projection and its variations for face and palmprint recognition. In: International Conference on Computational Science and Its Applications. Springer

  33. Leng L, et al (2012) Two-dimensional cancelable biometric scheme. In: Wavelet Analysis and Pattern Recognition (ICWAPR), 2012 International Conference on. IEEE

  34. Leng L, Li M, Teoh ABJ (2013) Conjugate 2dpalmhash code for secure palm-print-vein verification. In: Image and Signal Processing (CISP), 2013 6th International Congress on. IEEE

  35. Leng L et al (2017) Dual-source discrimination power analysis for multi-instance contactless palmprint recognition. Multimed Tools Appl 76(1):333–354

    Article  Google Scholar 

  36. Li B, et al (2017) Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN. In: Multimedia & Expo Workshops (ICMEW), 2017 IEEE International Conference on. IEEE

  37. Li K et al (2018) Multi-modal feature fusion for geographic image annotation. Pattern Recogn 73:1–14

    Article  Google Scholar 

  38. Li Q, et al (2018) Improving image classification accuracy with ELM and CSIFT. Comput Sci Eng

  39. Liaqat A, et al (2018) Automated ulcer and bleeding classification from wce images using multiple features fusion and selection. Journal of Mechanics in Medicine and Biology:1850038

  40. Liu L, Wang L, Liu X (2011) In defense of soft-assignment coding. In: Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE

  41. Liu W et al (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26

    Article  Google Scholar 

  42. Liu W, Yang X, Tao D, Cheng J, Tang Y (2018) Multiview dimension reduction via hessian multiset canonical correlations. Information Fusion 41:119–128

  43. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  44. Naeini AA et al (2018) Particle swarm optimization for object-based feature selection of VHSR satellite images. IEEE Geosci Remote Sens Lett 15(3):379–383

    Article  Google Scholar 

  45. Pan Y, et al Locality constrained encoding of frequency and spatial information for image classification. Multimed Tools Appl:1–17

  46. Qin C, Sun M, Chang C-C (2018) Perceptual hashing for color images based on hybrid extraction of structural features. Signal Process 142:194–205

    Article  Google Scholar 

  47. Rastegari M, et al (2016) Xnor-net: imagenet classification using binary convolutional neural networks. In: European Conference on Computer Vision. Springer

  48. Raza M et al (2018) Appearance based pedestrians’ gender recognition by employing stacked auto encoders in deep learning. Futur Gener Comput Syst 88:28–39

    Article  Google Scholar 

  49. Roy PK, Om H (2018) Suspicious and violent activity detection of humans using HOG features and SVM classifier in surveillance videos. In: Advances in soft computing and machine learning in image processing. Springer, p 277–294

  50. Sankar AS, Nair SS, Dharan VS, Sankaran P (2015) Wavelet sub band entropy based feature extraction method for BCI. Procedia Computer Science 46:1476–1482

  51. Sharif M, Khan MA, Akram T, Javed MY, Saba T, Rehman A (2017) A framework of human detection and action recognition based on uniform segmentation and combination of Euclidean distance and joint entropy-based features selection. EURASIP Journal on Image and Video Processing 2017(1):89

  52. Sharif M, et al. (2018) A framework for offline signature verification system: best features selection approach. Pattern Recogn Lett

  53. Sharif M et al (2018) Detection and classification of citrus diseases in agriculture based on optimized weighted segmentation and feature selection. Comput Electron Agric 150:220–234

    Article  Google Scholar 

  54. Siddiqui S, Khan MA, Bashir K, Sharif M, Azam F, Javed MY (2018) Human action recognition: a construction of codebook by discriminative features selection approach. International Journal of Applied Pattern Recognition 5(3):206–228

  55. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  56. Singh C, Walia E, Kaur KP (2018) Enhancing color image retrieval performance with feature fusion and non-linear support vector machine classifier. Optik 158:127–141

    Article  Google Scholar 

  57. Song J, et al (2018) Structure preserving dimensionality reduction for visual object recognition. Multimed Tools Appl:1–17

  58. Szegedy C, et al (2015) Going deeper with convolutions. Cvpr

  59. Wei G et al (2018) Content-based image retrieval for lung nodule classification using texture features and learned distance metric. J Med Syst 42(1):13

    Article  Google Scholar 

  60. Yu W, et al (2018) Hierarchical semantic image matching using CNN feature pyramid. Comput Vis Image Underst

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, COMSATS University Islamabad, Wah Campus, Islamabad, Pakistan

    Muhammad Rashid, Muhammad Sharif, Mudassar Raza & Farhat Afza

  2. Department of Computer Science and Engineering, HITEC University, Museum Road, Taxila, Pakistan

    Muhammad Attique Khan

  3. Department of Electrical Engineering, COMSATS University Islamabad, Wah Campus, Islamabad, Pakistan

    Muhammad Masood Sarfraz

Authors
  1. Muhammad Rashid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Attique Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mudassar Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Muhammad Masood Sarfraz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Farhat Afza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Attique Khan.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rashid, M., Khan, M.A., Sharif, M. et al. Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features. Multimed Tools Appl 78, 15751–15777 (2019). https://doi.org/10.1007/s11042-018-7031-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-7031-0

Keywords

Search

Navigation