Privacy Protection and Face Recognition

  • Andrew W. SeniorEmail author
  • Sharathchandra Pankanti


In this chapter, we describe the privacy issues surrounding the proliferation of digital imagery, particularly of faces, in surveillance video, online photo-sharing, medical records and online navigable street imagery. We highlight the growing capacity for computer systems to process, recognize, and index face images and outline some of the techniques that have been used to protect privacy while supporting ongoing innovation and growth in the applications of digital imagery.


False Alarm Face Recognition Face Image Video Surveillance Privacy Protection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Acquisti, A.: Privacy and security of personal information: Economic incentives and technological solutions. In: Camp, J., Lewis, R. (eds.) The Economics of Information Security. Kluwer, Dordrecht (2004) Google Scholar
  2. 2.
    Associated Press. Swiss official demands shutdown of Google Street View. New York Times (2009) Google Scholar
  3. 3.
    Avidan, S., Butman, M.: Blind vision. In: European Conference on Computer Vision, vol. 3953, pp. 1–13 (2006) Google Scholar
  4. 4.
    Avidan, S., Butman, M.: Efficient methods for privacy preserving face detection. In: NIPS, pp. 57–64 (2006) Google Scholar
  5. 5.
    Bentham, J.: Panopticon Letters. London (1787).
  6. 6.
    Black, J., Ellis, T.: Multi camera image tracking. In: International Workshop on Performance Evaluation of Tracking and Surveillance (2001) Google Scholar
  7. 7.
    Bolle, R.M., Connell, J.H., Pankanti, S., Ratha, N.K., Senior, A.W.: Guide to Biometrics: Selection and Use. Springer, New York (2003) Google Scholar
  8. 8.
    Boult, T., Micheals, R.J., Gao, X., Eckmann, M.: Into the woods: Visual surveillance of non-cooperative and camouflaged targets in complex outdoor settings. Proc. IEEE 89(10), 1382–1402 (2001) CrossRefGoogle Scholar
  9. 9.
    Brassil, J.: Using mobile communications to assert privacy from video surveillance. In: 19th IEEE International Parallel and Distributed Processing Symposium, p. 290a (2005) CrossRefGoogle Scholar
  10. 10.
    Brin, D.: The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom. Perseus, Cambridge (1999) Google Scholar
  11. 11.
    Caloyannides, M.: Society cannot function without privacy. IEEE Secur. Priv. Mag., May/June 2003 Google Scholar
  12. 12.
    Carrillo, P., Kalva, H., Magliveras, S.: Compression independent reversible encryption for privacy in video surveillance. EURASIP J. Inf. Secur. Google Scholar
  13. 13.
    Congestion charging: Enforcement technology. BBC LDN (2003).
  14. 14.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 492–7 (2001) CrossRefGoogle Scholar
  15. 15.
    Coutaz, J., Bérard, F., Carraux, E., Astier, W., Crowley, J.L.: CoMedi: Using computer vision to support awareness and privacy in mediaspaces. In: CHI, pp. 13–14. ACM Press, New York (1999) Google Scholar
  16. 16.
    Danielson, P.: Video surveillance for the rest of us: Proliferation, privacy, and ethics education. In: International Symposium on Technology and Society, 6–8 June 2002, pp. 162–167 (2002) Google Scholar
  17. 17.
    Davies, S.: The loose cannon: An overview of campaigns of opposition to national identity card proposals. In: e-ID: Securing the Mobility of Citizens and Commerce in a Greater Europe. Unisys, February 2004 Google Scholar
  18. 18.
    Dufaux, F., Ebrahimi, T.: Scrambling for video surveillance with privacy. In: Proceedings of Computer Vision and Pattern Recognition, June 2006, p. 160 (2006) Google Scholar
  19. 19.
    Erkin, Z., Franz, M., Guajardo, J., Katzenbeisser, S., Lagendijk, I., Toft, T.: Privacy-preserving face recognition. In: Privacy Enhancing Technologies Symposium (2009) Google Scholar
  20. 20.
    Facebook press room, 21 April 2010 Google Scholar
  21. 21.
    Flores, A., Belongie, S.: Removing pedestrians from Google Street View images. In: International Workshop on Mobile Vision, June 2010 Google Scholar
  22. 22.
    Frome, A., Cheung, G., Abdulkader, A., Zennaro, M., Wu, B., Bissacco, A., Adam, H., Neven, H., Vincent, L.: Large-scale privacy protection in Google Street View. In: Proceedings of Computer Vision and Pattern Recognition (2009) Google Scholar
  23. 23.
    Gross, R., Airoldi, E., Malin, B., Sweeney, L.: Integrating utility into face de-identification. In: Workshop on Privacy Enhancing Technologies. CMU (2005) Google Scholar
  24. 24.
    Gross, R., Sweeney, L., de la Torre, F., Baker, S.: Model-based face de-identification. In: Workshop on Privacy Research in Vision. IEEE, New York (2006) Google Scholar
  25. 25.
    Gross, R., Sweeney, L., Cohn, J., de la Torre, F., Baker, S.: Face de-identification. In: Senior, A.W. (ed.) Protecting Privacy in Video Surveillance. Springer, Berlin (2009) Google Scholar
  26. 26.
    Hampapur, A., Brown, L., Connell, J., Ekin, A., Lu, M., Merkl, H., Pankanti, S., Senior, A., Tian, Y.L.: Multi-scale tracking for smart video surveillance. IEEE Trans. Signal Process. (2005) Google Scholar
  27. 27.
    Hudson, S., Smith, I.: Techniques for addressing fundamental privacy and distribution tradeoffs in awareness support systems. In: CSCW, pp. 248–257 (1996) Google Scholar
  28. 28.
    Ito, I., Kiya, H.: One-time key based phase scrambling for phase-only correlation between visually protected images. EURASIP J. Inf. Secur. Google Scholar
  29. 29.
    Khan, S., Shah, M.: Tracking people in presence of occlusion. In: Asian Conference on Computer Vision (2000) Google Scholar
  30. 30.
    Koshimizu, T., Toriyama, T., Babaguchi, N.: Factors on the sense of privacy in video surveillance. In: Proceedings of the 3rd ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, pp. 35–44. ACM, New York (2006) CrossRefGoogle Scholar
  31. 31.
    Koshimizu, T., Umata, I., Toriyama, T., Babaguchi, N.: Psychological study for designing privacy protected video surveillance system: PriSurv. In: Senior, A.W. (ed.) Protecting Privacy in Video Surveillance. Springer, Berlin (2009) Google Scholar
  32. 32.
    Lipton, A.J., Clark, J.I.W., Thompson, B., Myers, G., Zhang, Z., Titus, S., Venetianer, P.: The intelligent vision sensor: Turning video into information. In: Advanced Video and Signal-based Surveillance. IEEE, New York (2007) Google Scholar
  33. 33.
    McCahill, M., Norris, C.: CCTV. Perpetuity Press, Leicester (2003) Google Scholar
  34. 34.
    McKenna, S., Jabri, J.S., Duran, Z., Wechsler, H.: Tracking interacting people. In: International Conference on Face and Gesture Recognition, March 2000, pp. 348–53 (2000) CrossRefGoogle Scholar
  35. 35.
    Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput. Hum. Interact. (2006) Google Scholar
  36. 36.
    Newton, E., Sweeney, L., Malin, B.: Preserving privacy by de-identifying facial images. Technical Report CMU-CS-03-119, Carnegie Mellon University, School of Computer Science, Pittsburgh (2003) Google Scholar
  37. 37.
    Newton, E., Sweeney, L., Malin, B.: Preserving privacy by de-identifying facial images. IEEE Trans. Knowl. Data Eng. 2(17), 232–243 (2005) CrossRefGoogle Scholar
  38. 38.
    New York city police department releases draft of public security privacy guidelines for public comment. NYPD Press Release, February 2009 Google Scholar
  39. 39.
    Palmer, M.: Google debates face recognition technology. Financial Times, 19 May 2010 Google Scholar
  40. 40.
    Paruchuri, J.K., Cheung, S.S., Hail, M.W.: Video data hiding for managing privacy information in surveillance systems. EURASIP J. Inf. Secur. Google Scholar
  41. 41.
    Patel, S.N., Summet, J.W., Truong, K.N.: Blindspot: Creating capture-resistant spaces. In: Senior, A.W. (ed.) Protecting Privacy in Video Surveillance, pp. 185–201. Springer, Berlin (2009) CrossRefGoogle Scholar
  42. 42.
    Phillips, P.J., Scruggs, W.T., O’Toole, A.J., Flynn, P.J., Bowyer, K.W., Schott, C.L., Sharpe, M.: FRVT 2006 and ICE 2006 large-scale results. Technical Report NISTIR 7408, NIST, Gaithersburg, MD 20899, March 2006 Google Scholar
  43. 43.
    Poindexter, J.: Overview of the information awareness office, August 2002.
  44. 44.
  45. 45.
    Potamianos, G., Neti, C., Gravier, G., Garg, A., Senior, A.W.: Recent advances in the automatic recognition of audiovisual speech. Proc. IEEE (2003) Google Scholar
  46. 46.
    Privacy fears force search giant to block facial recognition application on Google goggles. The Daily Mail Online, December 2009 Google Scholar
  47. 47.
    Sadeghi, A.R., Schneider, T., Wehrenberg, I.: Efficient privacy-preserving face recognition. In: 12th International Conference on Information Security and Cryptology (2010) Google Scholar
  48. 48.
    Sargin, M.E., Aradhye, H., Moreno, P., Zhao, M.: Audiovisual celebrity recognition in unconstrained web videos. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, April 2009 Google Scholar
  49. 49.
    Schiff, J., Meingast, M., Mulligan, D.K., Sastry, S., Goldberg, K.: Respectful cameras: Detecting visual markers in real-time to address privacy concerns. In: Senior, A.W. (ed.) Protecting Privacy in Video Surveillance. Springer, Berlin (2009) Google Scholar
  50. 50.
    Senior, A.W.: Privacy protection in a video surveillance system. In: Senior, A.W. (ed.) Protecting Privacy in Video Surveillance. Springer, Berlin (2009) CrossRefGoogle Scholar
  51. 51.
    Senior, A.W., Pankanti, S., Hampapur, A., Brown, L., Tian, Y.-L., Ekin, A.: Blinkering surveillance: Enabling video privacy through computer vision. Technical Report RC22886, IBM T.J.Watson Research Center, NY 10598, August 2003 Google Scholar
  52. 52.
    Senior, A.W., Pankanti, S., Hampapur, A., Brown, L., Tian, Y.-L., Ekin, A.: Enabling video privacy through computer vision. IEEE Secur. Priv. 3(5), 50–57 (2004) Google Scholar
  53. 53.
    Spiekermann, S., Grossklags, J., Berendt, B.: E-Privacy in 2nd Generation E-Commerce: Privacy Preferences Versus Actual Behavior, pp. 38–47. ACM Press, New York (2001) Google Scholar
  54. 54.
    Stana, R.: Video surveillance. Technical Report GAO–03–748, United States General Accounting Office, June 2003 Google Scholar
  55. 55.
    Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000) CrossRefGoogle Scholar
  56. 56.
    Sweeney, L., Gross, R.: Mining images in publicly-available cameras for homeland security. In: AAAI Spring Symposium on AI technologies for Homeland Security (2005) Google Scholar
  57. 57.
    Swiss Federal Data Protection Commissioner. Leaflet on video surveillance by private individuals. 3003 Bern, January 2003 Google Scholar
  58. 58.
    The health insurance portability and accountability act (HIPAA) privacy and security rules (1996) Google Scholar
  59. 59.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991) CrossRefGoogle Scholar
  60. 60.
    Venetianer, P., Zhang, Z., Scanlon, A., Hu, Y., Lipton, A.: Video verification of point of sale transactions. In: AVSS (2007) Google Scholar
  61. 61.
    Weaver, M.: Canoe mystery man arrested for fraud. The Guardian, December 2007 Google Scholar
  62. 62.
    Wickramasuriya, J., Alhazzazi, M., Datt, M., Mehrotra, S., Venkatasubramanian, N.: Privacy-protecting video surveillance. In: SPIE International Symposium on Electronic Imaging (2005) Google Scholar
  63. 63.
    Yu, T., Lim, S.-N., Patwardhan, K., Krahnstoever, N.: Monitoring, recognizing and discovering social networks. In: Proceedings of Computer Vision and Pattern Recognition (2009) Google Scholar
  64. 64.
    Zhang, W., Cheung, S.-C., Chen, M.: Hiding privacy information in video surveillance system. In: Proc. International Conference on Image Processing (2005) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Google ResearchNew YorkUSA
  2. 2.IBM ResearchYorktown HeightsUSA

Personalised recommendations