Skip to main content
SpringerLink
Log in

Super-resolution for Selfie Biometrics: Introduction and Application to Face and Iris

  • Chapter
  • First Online:
Selfie Biometrics

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Biometrics research is heading towards enabling more relaxed acquisition conditions. This has effects on the quality and resolution of acquired images, severely affecting the accuracy of recognition systems if not tackled appropriately. In this chapter, we give an overview of recent research in super-resolution reconstruction applied to biometrics, with a focus on face and iris images in the visible spectrum, two prevalent modalities in selfie biometrics. After an introduction to the generic topic of super-resolution, we investigate methods adapted to cater for the particularities of these two modalities. By experiments, we show the benefits of incorporating super-resolution to improve the quality of biometric images prior to recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041. https://doi.org/10.1109/TPAMI.2006.244

    Article  MATH  Google Scholar 

  2. Aljadaany R, Luu K, Venugopalan S, Savvides M (2015) Iris super-resolution via nonparametric over-complete dictionary learning. In: Proceedings of the IEEE international conference on image processing, ICIP, pp 3856–3860. https://doi.org/10.1109/ICIP.2015.7351527

  3. Alonso-Fernandez F, Bigun J (2013) Quality factors affecting iris segmentation and matching. In: Proceedings of the international conference on biometrics, ICB, pp 1–6. https://doi.org/10.1109/ICB.2013.6613016

  4. Alonso-Fernandez F, Farrugia RA, Bigun J (2015) Eigen-patch iris super-resolution for iris recognition improvement. In: Proceedings of the European signal processing conference, EUSIPCO

    Google Scholar 

  5. Alonso-Fernandez F, Farrugia RA, Bigun J (2017) Iris super-resolution using iterative neighbor embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, CVPRW, pp 655–663. https://doi.org/10.1109/CVPRW.2017.94

  6. Alonso-Fernandez F, Fierrez J, Ortega-Garcia J (2012) Quality measures in biometric systems. IEEE Secur Privacy 10(6):52–62

    Google Scholar 

  7. Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition (Cat. No. PR00580), pp 83–88 . https://doi.org/10.1109/AFGR.2000.840616

  8. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183

    Article  Google Scholar 

  9. Barnard R, Pauca VP, Torgersen TC, Plemmons RJ, Prasad S, van der Gracht J, Nagy J, Chung J, Behrmann G, Mathews S, Mirotznik M (2006) High-resolution iris image reconstruction from low-resolution imagery. https://doi.org/10.1117/12.681930

  10. Cao Q, Lin L, Shi Y, Liang X, Li G (2017) Attention-aware face hallucination via deep reinforcement learning. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 1656–1664. https://doi.org/10.1109/CVPR.2017.180

  11. Carrato S, Ramponi G, Marsi S (1996) A simple edge-sensitive image interpolation filter. In: Proceedings of 3rd IEEE international conference on image processing, vol 3, pp 711–714

    Google Scholar 

  12. Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel pca-based prior. IEEE Trans Multimedia 9(4):888–892. https://doi.org/10.1109/TMM.2007.893346

    Article  Google Scholar 

  13. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol 1, p I

    Google Scholar 

  14. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R (1996) Super-resolved surface reconstruction from multiple images. Springer Netherlands, Dordrecht, pp 293–308

    Chapter  Google Scholar 

  15. Chen HY, Chien SY (2014) Eigen-patch: position-patch based face hallucination using eigen transformation. In: Proceedings of the IEEE international conference on multimedia and expo, ICME, pp 1–6

    Google Scholar 

  16. Deshpande A, Patavardhan P (2017) Multi-frame super-resolution for long range captured iris polar image. IET Biomet 6(2):108–116. https://doi.org/10.1049/iet-bmt.2016.0076

    Article  Google Scholar 

  17. Deshpande A, Patavardhan PP (2017) Super resolution and recognition of long range captured multi-frame iris images. IET Biomet 6(5):360–368. https://doi.org/10.1049/iet-bmt.2016.0075

    Article  Google Scholar 

  18. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307. https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  19. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630. https://doi.org/10.1109/TIP.2012.2235847

    Article  MathSciNet  MATH  Google Scholar 

  20. Fahmy G (2007) Super-resolution construction of iris images from a visual low resolution face video. In: Proceedigns of the national radio science conference, NRSC

    Google Scholar 

  21. Farrugia RA, Guillemot C (2016) Robust face hallucination using quantization-adaptive dictionaries. In: 2016 IEEE international conference on image processing (ICIP), pp 414–418 . https://doi.org/10.1109/ICIP.2016.7532390

  22. Farrugia RA, Guillemot C (2017) Face hallucination using linear models of coupled sparse support. IEEE Trans Image Process 26(9):4562–4577. https://doi.org/10.1109/TIP.2017.2717181

    Article  MathSciNet  Google Scholar 

  23. Farsiu S, Robinson D, Elad M, Milanfar P (2003) Fast and robust super-resolution. In: Proceedings 2003 international conference on image processing (Cat. No.03CH37429), vols 2 and 3, pp II–291–4. https://doi.org/10.1109/ICIP.2003.1246674

  24. Farsiu S, Robinson D, Elad M, Milanfar P (2003) Robust shift and add approach to superresolution. https://doi.org/10.1117/12.507194

  25. Farsiu S, Robinson MD, Elad M, Milanfar P (2004) Fast and robust multiframe super resolution. IEEE Trans Image Process 13(10):1327–1344. https://doi.org/10.1109/TIP.2004.834669

    Article  Google Scholar 

  26. Fierrez J, Morales A, Vera-Rodriguez R, Camacho D (2018) Multiple classifiers in biometrics. Part 1: fundamentals and review. Inf Fusion 44:57–64 . https://doi.org/10.1016/j.inffus.2017.12.003

    Article  Google Scholar 

  27. Fierrez J, Morales A, Vera-Rodriguez R, Camacho D (2018) Multiple classifiers in biometrics. Part 2: trends and challenges. Inf Fusion 44:103–112 . https://doi.org/10.1016/j.inffus.2017.12.005

    Article  Google Scholar 

  28. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747

    Article  Google Scholar 

  29. Galbally J, Marcel S, Fierrez J (2014) Image quality assessment for fake biometric detection: application to iris, fingerprint, and face recognition. IEEE Trans Image Process 23(2):710–724

    Article  MathSciNet  Google Scholar 

  30. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: The IEEE international conference on computer vision (ICCV)

    Google Scholar 

  31. Gonzalez RC, Woods RE (2006) Digital image processing, 3rd edn. Prentice-Hall Inc., Upper Saddle River, NJ, USA

    Google Scholar 

  32. Hardie RC, Barnard KJ, Armstrong EE (1997) Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans Image Process 6(12):1621–1633. https://doi.org/10.1109/83.650116

    Article  Google Scholar 

  33. Hollingsworth K, Peters T, Bowyer K, Flynn P (2009) Iris recognition using signal-level fusion of frames from video. IEEE Trans Inf Forensic Secur 4(4):837–848

    Article  Google Scholar 

  34. Hsieh SH, Li YH, Tien CH, Chang CC (2016) Extending the capture volume of an iris recognition system using wavefront coding and super-resolution. IEEE Trans Cybern 46(12):3342–3350. https://doi.org/10.1109/TCYB.2015.2504388

    Article  Google Scholar 

  35. Hu Y, Lam KM, Qiu G, Shen T (2011) From local pixel structure to global image super-resolution: a new face hallucination framework. IEEE Trans Image Process 20(2):433–445. https://doi.org/10.1109/TIP.2010.2063437

    Article  MathSciNet  MATH  Google Scholar 

  36. Huang H, He H, Fan X, Zhang J (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recogn 43(7):2532–2543. https://doi.org/10.1016/j.patcog.2010.02.007

    Article  Google Scholar 

  37. Huang J, Ma L, Tan T, Wang Y (2003) Learning based resolution enhancement of iris images. In: Proceedings of the BMVC

    Google Scholar 

  38. Irani M, Peleg S (1990) Super resolution from image sequences. In: [1990] Proceedings. 10th international conference on pattern recognition, vol 2, pp 115–120. https://doi.org/10.1109/ICPR.1990.119340

  39. Irani M, Peleg S (1993) Motion analysis for image enhancement: resolution, occlusion, and transparency. J Vis Commun Image Represent 4(4):324–335. https://doi.org/10.1006/jvci.1993.1030. URL http://www.sciencedirect.com/science/article/pii/S1047320383710308

    Article  Google Scholar 

  40. Jain A, Nandakumar K, Ross A (2016) 50 years of biometric research: accomplishments, challenges, and opportunities. Pattern Recogn Lett 79:80–105

    Article  Google Scholar 

  41. Jain AK, Kumar A (2011) Second generation biometrics, chapter. An overview. Springer, Biometrics of next generation

    Google Scholar 

  42. Jiang J, Hu R, Wang Z, Han Z (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Proces 23(10):4220–4231. https://doi.org/10.1109/TIP.2014.2347201

    Article  MathSciNet  MATH  Google Scholar 

  43. Jillela R, Ross A, Flynn P (2011) Information fusion in low-resolution iris videos using principal components transform. In: Proceedings of the IEEEwWorkshop on applications of computer vision, WACV, pp 262–269. https://doi.org/10.1109/WACV.2011.5711512

  44. Jung C, Jiao L, Liu B, Gong M (2011) Position-patch based face hallucination using convex optimization. IEEE Sig Process Lett 18(6):367–370. https://doi.org/10.1109/LSP.2011.2140370

    Article  Google Scholar 

  45. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: The IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  46. Li B, Chang H, Shan S, Chen X (2009) Aligning coupled manifolds for face hallucination. IEEE Sig Process Lett 16(11):957–960. https://doi.org/10.1109/LSP.2009.2027657

    Article  Google Scholar 

  47. Li Y, Cai C, Qiu G, Lam KM (2014) Face hallucination based on sparse local-pixel structure. Pattern Recogn 47(3), 1261–1270. https://doi.org/10.1016/j.patcog.2013.09.012. URL http://www.sciencedirect.com/science/article/pii/S0031320313003841 (Handwriting recognition and other PR applications)

    Article  Google Scholar 

  48. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1132–1140. https://doi.org/10.1109/CVPRW.2017.151

  49. Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans Pattern Anal Mach Intell 26(1):83–97

    Article  Google Scholar 

  50. Liu J, Sun Z, Tan T (2013) Code-level information fusion of low-resolution iris image sequences for personal identification at a distance. In: Proceedings of the international conference on biometrics: theory, applications and systems, BTAS, pp 1–6. https://doi.org/10.1109/BTAS.2013.6712692

  51. Ma X, Zhang J, Qi C (2009) Position-based face hallucination method. In: 2009 IEEE international conference on multimedia and expo, pp 290–293. https://doi.org/10.1109/ICME.2009.5202492

  52. Masek L (2003) Recognition of human iris patterns for biometric identification. Master’s thesis. School of Computer Science and Software Engineering, University of Western Australia

    Google Scholar 

  53. Nasir H, Stankovic V, Marshall S (2011) Singular value decomposition based fusion for super-resolution image reconstruction. In: 2011 IEEE international conference on signal and image processing applications (ICSIPA), pp 393–398. https://doi.org/10.1109/ICSIPA.2011.6144138

  54. Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468. https://doi.org/10.1007/s00138-014-0623-4

    Article  Google Scholar 

  55. Nguyen K, Fookes C, Sridharan S (2010) Robust mean super-resolution for less cooperative nir iris recognition at a distance and on the move. In: Proceedings of the symposium on information and communication technology, SoICT, pp 122–127 . https://doi.org/10.1145/1852611.1852635

  56. Nguyen K, Fookes C, Sridharan S., Denman S (2010) Focus-score weighted super-resolution for uncooperative iris recognition at a distance and on the move. In: Proceedings of the 25th international conference of image and vision computing New Zealand, IVCNZ, pp 1–8. https://doi.org/10.1109/IVCNZ.2010.6148792

  57. Nguyen K, Fookes C, Sridharan S, Denman S (2011) Feature-domain super-resolution for iris recognition. In: Proceedings of the IEEE international conference on image processing, ICIP, pp 3197–3200. https://doi.org/10.1109/ICIP.2011.6116348

  58. Nguyen K, Fookes C, Sridharan S, Denman S (2011) Quality-driven super-resolution for less constrained iris recognition at a distance and on the move. IEEE Trans Inf For Secur 6(4):1248–1258

    Article  Google Scholar 

  59. Nguyen K, Fookes C, Sridharan S, Denman S (2013) Feature-domain super-resolution for iris recognition. Comput Vis Image Underst 117(10):1526–1535. https://doi.org/10.1016/j.cviu.2013.06.010. URL http://www.sciencedirect.com/science/article/pii/S1077314213001306

    Article  Google Scholar 

  60. Nguyen K, Fookes C, Sridharan S, Tistarelli M, Nixon M (2018) Super-resolution for biometrics: a comprehensive survey. Pattern Recogn 78:23–42. https://doi.org/10.1016/j.patcog.2018.01.002. URL http://www.sciencedirect.com/science/article/pii/S0031320318300049

    Article  Google Scholar 

  61. Nguyen K, Sridharan S, Denman S, Fookes C (2012) Feature-domain super-resolution framework for gabor-based face and iris recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, CVPR, pp 2642–2649

    Google Scholar 

  62. Othman N, Dorizzi B (2015) Impact of quality-based fusion techniques for video-based iris recognition at a distance. IEEE Trans Inf For Secur 10(8):1590–1602. https://doi.org/10.1109/TIFS.2015.2421314

    Article  Google Scholar 

  63. Park JS, Lee SW (2008) An example-based face hallucination method for single-frame, low-resolution facial images. IEEE Trans Image Process 17(10):1806–1816. https://doi.org/10.1109/TIP.2008.2001394

    Article  MathSciNet  MATH  Google Scholar 

  64. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: Proceedings of the British machine vision conference, BMVC

    Google Scholar 

  65. Pham TQ, van Vliet LJ, Schutte K (2006) Robust fusion of irregularly sampled data using adaptive normalized convolution. EURASIP J Adv Signal Process 2006(1):083,268. https://doi.org/10.1155/ASP/2006/83268

  66. Raja KB, Raghavendra R, Vemuri VK, Busch C (2015) Smartphone based visible iris recognition using deep sparse filtering. Pattern Recogn Lett 57:33–42

    Article  Google Scholar 

  67. Ribeiro E, Uhl A, Alonso-Fernandez F, Farrugia RA (2017) Exploring deep learning image super-resolution for iris recognition. In: Proceedings of the 25th European signal processing conference, EUSIPCO, pp 2176–2180. https://doi.org/10.23919/EUSIPCO.2017.8081595

  68. Schultz RR, Stevenson RL (1996) Extraction of high-resolution frames from video sequences. IEEE Trans Image Process 5(6):996–1011. https://doi.org/10.1109/83.503915

    Article  Google Scholar 

  69. Shin KY, Park KR, Kang BJ, Park SJ (2009) Super-resolution method based on multiple multi-layer perceptrons for iris recognition. In: International conference ubiquitous information technologies applications, ICUT, pp 1–5

    Google Scholar 

  70. Simonyan K, Grishin S, Vatolin D, Popov D (2008) Fast video super-resolution via classification. In: 2008 15th IEEE international conference on image processing, pp 349–352. https://doi.org/10.1109/ICIP.2008.4711763

  71. Su D, Willis P (2004) Image interpolation by pixel-level data-dependent triangulation. Comput Graph Forum. https://doi.org/10.1111/j.1467-8659.2004.00752.x

    Article  Google Scholar 

  72. Su K, Tian Q, Xue Q, Sebe N, Ma J (2005) Neighborhood issue in single-frame image super-resolution. In: 2005 IEEE international conference on multimedia and expo, p 4. https://doi.org/10.1109/ICME.2005.1521623

  73. Thapa D, Raahemifar K, Bobier WR, Lakshminarayanan V (2016) A performance comparison among different super-resolution techniques. Comput Electr Eng 54:313–329. https://doi.org/10.1016/j.compeleceng.2015.09.011. URL http://www.sciencedirect.com/science/article/pii/S0045790615003183

    Article  Google Scholar 

  74. Thévenaz P, Blu T, Unser M (2000) Handbook of medical imaging. Chapter. Image interpolation and resampling. Academic Press, Inc., Orlando, pp 393–420. URL http://dl.acm.org/citation.cfm?id=374166.374424

    Chapter  Google Scholar 

  75. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  76. Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30

    Article  Google Scholar 

  77. Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Trans Syst Man Cybern Part C Appl Rev 35(3):425–434. https://doi.org/10.1109/TSMCC.2005.848171

    Article  Google Scholar 

  78. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  79. Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: 2008 IEEE conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2008.4587647

  80. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  MATH  Google Scholar 

  81. Zhang Q, Li H, He Z, Sun Z (2016) Image super-resolution for mobile iris recognition. In: Proceedings of the 11th Chinese conference on biometric recognition, CCBR, pp 399–406

    Google Scholar 

  82. Zomet A, Peleg S (2000) Efficient super-resolution and applications to mosaics. In: Proceedings 15th international conference on pattern recognition. ICPR-2000, vol 1, pp 579–583. https://doi.org/10.1109/ICPR.2000.905404

Download references

Acknowledgements

Authors F. A.-F. and J. B thank the Swedish Research Council (VR), the Sweden’s innovation agency (VINNOVA), and the Swedish Knowledge Foundation (CAISR programme and SIDUS-AIR project). Author J. F. thanks Accenture and the project CogniMetrics (TEC2015-70627-R) from MINECO/FEDER

Author information

Authors and Affiliations

  1. School of Information Technology (ITE), Halmstad University, Box 823, 30118, Halmstad, Sweden

    Fernando Alonso-Fernandez & Josef Bigun

  2. Department of Communications and Computer Engineering (CCE), University of Malta, Msida, Malta

    Reuben A. Farrugia

  3. Escuela Politecnica Superior, Universidad Autonoma de Madrid, 28049, Madrid, Spain

    Julian Fierrez

Authors
  1. Fernando Alonso-Fernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reuben A. Farrugia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Julian Fierrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Josef Bigun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Alonso-Fernandez .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering and Computer Science, Wichita State University, Wichita, KS, USA

    Ajita Rattani

  2. Department of Computer Science and Electrical Engineering, University of Missouri-Kansas City, Kansas City, MO, USA

    Reza Derakhshani

  3. Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA

    Arun Ross

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Alonso-Fernandez, F., Farrugia, R.A., Fierrez, J., Bigun, J. (2019). Super-resolution for Selfie Biometrics: Introduction and Application to Face and Iris. In: Rattani, A., Derakhshani, R., Ross, A. (eds) Selfie Biometrics. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-030-26972-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-26972-2_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-26971-5

  • Online ISBN: 978-3-030-26972-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation