Face sketch recognition using a hybrid optimization model

  • Hussein Samma
  • Shahrel Azmin Suandi
  • Junita Mohamad-Saleh
Original Article


In this work, a hybrid optimization-based model is introduced to handle the problem of face sketch recognition. The proposed model comprises a total of three layers that are global search layer, control layer, and fine-tuning layer. The global layer contains a set of search operations from particle swarm optimization (PSO) algorithm to perform the task of global search. However, the control layer is responsible about controlling the execution of the implemented search operations at run time. Finally, the fine-tuning layer is aimed at performing search refinement to enhance the search ability. For sketch recognition, the proposed hybrid model is applied on the input face sketch to locate the internal sketch facial components. Three types of texture features extraction techniques are adopted in this study including Histogram Of Gradient (HOG), Local Binary Pattern (LBP), and Gabor wavelet. To assess the performances of the proposed model, a total of three face sketch databases have been used which are LFW, AR, and CUHK. The reported results indicate that the proposed hybrid model was able to achieve a competitive performance with 96% on AR, 87.68% on CUHK, and 50.00% on LFW. Additionally, the outcomes reveal that the proposed model statistically outperforms others PSO-based models as well as the state-of-the-art meta-heuristic optimization models.


Particle swarm optimization Hybrid optimization model Local search face sketch recognition Meta-heuristic algorithm 



This paper was fully supported by Universiti Sains Malaysia (USM) Research University Individual (RUI) Grant Scheme under Grant Nos. 1001/PELECT/814208.

Compliance with ethical standards

Conflict of interest

Hussein Samma, Shahrel Azmin Suandi, and Junita Mohamad-Saleh declare that they have no conflict of interest.


  1. 1.
    Peng C, Gao X, Wang N, Li J (2017) Superpixel-based face sketch–photo synthesis. IEEE Trans Circuits Syst Video Technol 27(2):288–299CrossRefGoogle Scholar
  2. 2.
    Gao X, Zhong J, Tao D, Li X (2008) Local face sketch synthesis learning. Neurocomputing 71(10–12):1921–1930CrossRefGoogle Scholar
  3. 3.
    Zhang S, Gao X, Wang N, Li J (2017) Face sketch synthesis from a single photo–sketch pair. IEEE Trans Circuits Syst Video Technol 27(2):275–287CrossRefGoogle Scholar
  4. 4.
    Tu C-T, Chan Y-H, Chen Y-C (2016) Facial sketch synthesis using 2D direct combined model-based face-specific markov network. IEEE Trans Image Process 25:3546–3561MathSciNetCrossRefGoogle Scholar
  5. 5.
    Wang N, Li J, Tao D, Li X, Gao X (2013) Heterogeneous image transformation. Pattern Recognit Lett 34:77–84CrossRefGoogle Scholar
  6. 6.
    Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2012) Memetically optimized MCWLD for matching sketches with digital face images. IEEE Trans Inf Forensics Secur 7:1522–1535CrossRefGoogle Scholar
  7. 7.
    Hu H, Klare BF, Bonnen K, Jain AK (2013) Matching composite sketches to face photos: a component-based approach. IEEE Trans Inf Forensics Secur 8:191–204CrossRefGoogle Scholar
  8. 8.
    Klare BF, Zhifeng L, Jain AK (2011) Matching forensic sketches to mug shot photos. IEEE Trans Pattern Anal Mach Intell 33:639–646CrossRefGoogle Scholar
  9. 9.
    Xiaoou T, Xiaogang W (2004) Face sketch recognition. IEEE Trans Circuits Syst Video Technol 14:50–57CrossRefGoogle Scholar
  10. 10.
    Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2010) On matching sketches with digital face images, biometrics: theory applications and systems (BTAS). In: 2010 Fourth international conference on IEEE, pp 1–7Google Scholar
  11. 11.
    Samma H, Lim CP, Saleh JM (2016) A new reinforcement learning-based memetic particle swarm optimizer. Appl Soft Comput 43:276–297CrossRefGoogle Scholar
  12. 12.
    Wu Z, Wu Z, Zhang J (2016) An improved FCM algorithm with adaptive weights based on SA-PSO. Neural Comput Appl 28:1–6CrossRefGoogle Scholar
  13. 13.
    Samma H, Lim CP, Saleh JM, Suandi SA (2016) A memetic-based fuzzy support vector machine model and its application to license plate recognition. Memetic Comput 8:1–17CrossRefGoogle Scholar
  14. 14.
    Abd-Elazim S, Ali E (2013) A hybrid particle swarm optimization and bacterial foraging for optimal power system stabilizers design. Int J Electr Power Energy Syst 46:334–341CrossRefGoogle Scholar
  15. 15.
    Abd-Elazim S, Ali E (2013) Synergy of particle swarm optimization and bacterial foraging for TCSC damping controller design. Int J WSEAS Trans Power Syst 8:74–84Google Scholar
  16. 16.
    Abd-Elazim S, Ali E (2013) Power system stability enhancement via bacteria foraging optimization algorithm, Arabian Journal for Science & Engineering, vol 38. Springer, BerlinGoogle Scholar
  17. 17.
    Kamboj VK (2016) A novel hybrid PSO–GWO approach for unit commitment problem. Neural Comput Appl 27:1643–1655CrossRefGoogle Scholar
  18. 18.
    Subasi A, Kevric J, Canbaz MA (2017) Epileptic seizure detection using hybrid machine learning methods. Neural Comput Appl 1–9.
  19. 19.
    Sun S, Li J (2014) A two-swarm cooperative particle swarms optimization. Swarm Evolut Comput 15:1–18CrossRefGoogle Scholar
  20. 20.
    Lim WH, Mat Isa NA (2013) Two-layer particle swarm optimization with intelligent division of labor. Eng Appl Artif Intel 26:2327–2348CrossRefGoogle Scholar
  21. 21.
    Lim WH, Mat Isa NA (2014) Teaching and peer-learning particle swarm optimization. Appl Soft Comput 18:39–58CrossRefGoogle Scholar
  22. 22.
    Lim WH, Mat Isa NA (2014) Particle swarm optimization with increasing topology connectivity. Eng Appl Artif Intel 27:80–102CrossRefGoogle Scholar
  23. 23.
    Li X (2010) Niching without niching parameters: particle swarm optimization using a ring topology. IEEE Trans Evolut Comput 14:150–169CrossRefGoogle Scholar
  24. 24.
    Mendes R, Kennedy J, Neves J (2004) The fully informed particle swarm: simpler, maybe better. IEEE Trans Evolut Comput 8:204–210CrossRefGoogle Scholar
  25. 25.
    de Oca MAM, Stutzle T, Birattari M, Dorigo M (2009) Frankenstein’s PSO: a composite particle swarm optimization algorithm. IEEE Trans Evolut Comput 13:1120–1132CrossRefGoogle Scholar
  26. 26.
    Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evolut Comput 10:281–295CrossRefGoogle Scholar
  27. 27.
    Zhi-Hui Z, Jun Z, Yun L, Chung HSH (2009) Adaptive particle swarm optimization. IEEE Trans Syst Man Cybern Part B Cybern 39:1362–1381CrossRefGoogle Scholar
  28. 28.
    Hu M, Wu T, Weir JD (2013) An adaptive particle swarm optimization with multiple adaptive methods. IEEE Trans Evolut Comput 17:705–720CrossRefGoogle Scholar
  29. 29.
    Zhang W, Ma D, Wei J-J, Liang H-F (2014) A parameter selection strategy for particle swarm optimization based on particle positions. Expert Syst Appl 41:3576–3584CrossRefGoogle Scholar
  30. 30.
    Çomak E (2016) A modified particle swarm optimization algorithm using Renyi entropy-based clustering. Neural Comput Appl 5:1381–1390CrossRefGoogle Scholar
  31. 31.
    Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31:1955–1967CrossRefGoogle Scholar
  32. 32.
    Zhou H, Kuang Z, Wong K.-Y.K (2012) Markov weight fields for face sketch synthesis, computer vision and pattern recognition (CVPR). In: 2012 IEEE Conference on IEEE, pp 1091–1097Google Scholar
  33. 33.
    Wang N, Tao D, Gao X, Li X, Li J (2013) Transductive face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 24:1364–1376CrossRefGoogle Scholar
  34. 34.
    Peng C, Gao X, Wang N, Li J (2017) Superpixel-based face sketch-photo synthesis. IEEE Trans Circuits Syst Video Technol 27:288–299CrossRefGoogle Scholar
  35. 35.
    Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34:2274–2282CrossRefGoogle Scholar
  36. 36.
    Wang N, Gao X, Sun L, Li J (2017) Bayesian face sketch synthesis. IEEE Trans Image Process 26:1264–1274MathSciNetCrossRefGoogle Scholar
  37. 37.
    Zhang Y, McCullough C, Sullins JR, Ross CR (2010) Hand-drawn face sketch recognition by humans and a pca-based algorithm for forensic applications. IEEE Trans Syst Man Cyber Part A Syst Humans 40:475–485CrossRefGoogle Scholar
  38. 38.
    Klare BF, Jain AK (2013) Heterogeneous face recognition using kernel prototype similarities. IEEE Trans Pattern Anal Mach Intell 35:1410–1422CrossRefGoogle Scholar
  39. 39.
    Lahasan BM, Venkat I, Al-Betar MA, Lutfi SL, De Wilde P (2016) Recognizing faces prone to occlusions and common variations using optimal face subgraphs. Appl Math Comput 283:316–332MathSciNetGoogle Scholar
  40. 40.
    Lahasan BM, Venkat I, Lutfi SL (2014) Recognition of occluded faces using an enhanced EBGM algorithm, computer and information sciences (ICCOINS). In: 2014 international conference on IEEE, pp 1–5Google Scholar
  41. 41.
    Connolly J-F, Granger E, Sabourin R (2012) Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition. Pattern Recognit 45:2460–2477CrossRefGoogle Scholar
  42. 42.
    Naik MK, Panda R (2016) A novel adaptive cuckoo search algorithm for intrinsic discriminant analysis based face recognition. Appl Soft Comput 38:661–675CrossRefGoogle Scholar
  43. 43.
    Al-Ayyoub M, AlZu’bi S, Jararweh Y, Shehab MA, Gupta BB (2016) Accelerating 3D medical volume segmentation using GPUs. Multim Tools Appl 77:1–20Google Scholar
  44. 44.
    Jararweh Y, Al-Ayyoub M, Fakirah M, Alawneh L, Gupta BB (2017) Improving the performance of the needleman–wunsch algorithm using parallelization and vectorization techniques. Multim Tools Appl 1–17.
  45. 45.
    Alsmirat MA, Jararweh Y, Al-Ayyoub M, Shehab MA, Gupta BB (2017) Accelerating compute intensive medical imaging segmentation algorithms using hybrid CPU–GPU implementations. Multim Tools Appl 76:3537–3555CrossRefGoogle Scholar
  46. 46.
    Kennedy J, Eberhart R (1995) Particle swarm optimization, Neural Networks. IEEE Int Conf Proc 1944:1942–1948Google Scholar
  47. 47.
    Ratnaweera A, Halgamuge S, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evolut Comput 8:240–255CrossRefGoogle Scholar
  48. 48.
    Van Laarhoven PJ, Aarts EH (1987) Simulated annealing, simulated annealing: theory and applications. Springer, Berlin, pp 7–15CrossRefzbMATHGoogle Scholar
  49. 49.
    Déniz O, Bueno G, Salido J, De la Torre F (2011) Face recognition using histograms of oriented gradients. Pattern Recognit Lett 32:1598–1603CrossRefGoogle Scholar
  50. 50.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28:2037–2041CrossRefzbMATHGoogle Scholar
  51. 51.
    Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans Image Process 11:467–476CrossRefGoogle Scholar
  52. 52.
    Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07-49, University of Massachusetts, AmherstGoogle Scholar
  53. 53.
    Martinez A, Benavente R (2007) The AR face database, 1998. Computer vision center, technical report, p 3Google Scholar
  54. 54.
    Weiping C, Yongsheng G (2013) Face recognition using ensemble string matching. IEEE Trans Image Process 22:4798–4808MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Gao Y, Qi Y (2005) Robust visual similarity retrieval in single model face databases. Pattern Recognit 38:1009–1020CrossRefGoogle Scholar
  56. 56.
    Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3:71–86CrossRefGoogle Scholar
  57. 57.
    Efron B (1979) Bootstrap methods: another look at the Jackknife. Ann Stat 7:1–26MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Efrorn B, Tibshirani RJ (1993) An Introduction to the bootstrap. Chapman and Hall, New YorkCrossRefGoogle Scholar
  59. 59.
    Van den Bergh F, Engelbrecht AP (2004) A Cooperative approach to particle swarm optimization. IEEE Trans Evolut Comput 8:225–239CrossRefGoogle Scholar
  60. 60.
    Pham D, Ghanbarzadeh A, Koc E, Otri S, Rahim S, Zaidi M (2005) The bees algorithm. Technical note, Manufacturing Engineering Centre, Cardiff University, UK, pp 1–57Google Scholar
  61. 61.
    Zhao SZ, Suganthan PN, Pan Q-K, Fatih M (2011) Tasgetiren, Dynamic multi-swarm particle swarm optimizer with harmony search. Exp Syst Appl 38:3735–3742CrossRefGoogle Scholar
  62. 62.
    Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61CrossRefGoogle Scholar
  63. 63.
    Mirjalili S, Mirjalili S, Hatamlou A (2015) Multi-verse optimizer: a nature-inspired algorithm for global optimization. Neural Comput Appl, pp 1–19Google Scholar
  64. 64.
    Sahu A, Panigrahi SK, Pattnaik S (2012) Fast convergence particle swarm optimization for functions optimization. Proc Technol 4:319–324CrossRefGoogle Scholar
  65. 65.
    Liu B, Wang L, Jin Y-H (2007) An effective PSO-based memetic algorithm for flow shop scheduling. IEEE Trans Syst Man Cyber Part B Cyber 37:18–27CrossRefGoogle Scholar
  66. 66.
    Rakshit P, Konar A, Bhowmik P, Goswami I, Das S, Jain LC, Nagar AK (2013) Realization of an adaptive memetic algorithm using differential evolution and Q-learning: a case study in multirobot path planning. IEEE Trans Syst Man Cyber Part B Cyber 43:814–831CrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2018

Authors and Affiliations

  • Hussein Samma
    • 1
    • 2
  • Shahrel Azmin Suandi
    • 1
  • Junita Mohamad-Saleh
    • 1
  1. 1.Intelligent Biometric Group, School of Electrical and Electronic Engineering, Engineering CampusUniversiti Sains MalaysiaNibong TebalMalaysia
  2. 2.Department of Computer Programming, Faculty of Education – ShabwaUniversity of AdenAdenRepublic of Yemen

Personalised recommendations