Skip to main content
SpringerLink
Log in

SSIM Based Signature of Facial Micro-Expressions

  • Conference paper
  • First Online:
Image Analysis and Recognition (ICIAR 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12131))

Included in the following conference series:

Abstract

Facial microexpressions (MEs) play a crucial role in the non verbal communication. Their automatic detection and recognition on a real video is a topic of great interest in different fields. However, the main difficulty in automatically capturing this kind of feature consists in its rapid temporal evolution, i.e. MEs occur in very few frames of video acquired by a conventional camera. In this paper a first study concerning the perceptual characteristics of ME is performed. The study is based on the observation that MEs are visible by a human observer, even though they are very rapid, and almost independently of the context. The Structural SIMilarity index (SSIM), which is a common perception-based metric, has been then used to detect a sort of fingerprint of MEs, that will be indicated as PES (Perceptual Expression Signature). The latter is able to efficiently guide the preprocessing step for MEs recognition procedure, as it allows for a fast video segmentation by providing only those frames where a ME occurs with high probability. Preliminary empirical studies on MEs in the wild have confirmed the feasibility of such an approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brunet, D., Vrscay, E.R., Wang, Z.: On the mathematical properties of the structural similarity index. IEEE Trans. Image Process. 21(4), 1488–1499 (2012)

    Article  MathSciNet  Google Scholar 

  2. Duque, C., Alata, O., Emonet, R., Legrand, A.-C., Konik, H.: Micro-expression spotting using the Riesz pyramid. In: WACV Proceedings, Lake Tahoe (2018)

    Google Scholar 

  3. Ekman, P., Friesen, W.V.: Nonverbal leakage and clues to deception. Psychiatry 32, 88–106 (1969)

    Article  Google Scholar 

  4. Ekman, P.: Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage (Revised Edition). WW Norton & Company, New York (2009)

    Google Scholar 

  5. Ekman, P.: Lie catching and microexpressions. In: Martin, C. (ed.) The Philosophy of Deception, pp. 118–133. Oxford University Press (2009)

    Google Scholar 

  6. Haggard, E.A., Isaacs, K.S.: Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy. In: GottschalkArthur, L.A., Auerbach, H.A. (eds.) Methods of Research in Psychotherapy. TCPS, pp. 154–165. Springer, Boston (1966). https://doi.org/10.1007/978-1-4684-6045-2_14

    Chapter  Google Scholar 

  7. Kendall, M., Stuart, A.: The Advanced Theory of Statistics. Chareles Griffinn & Company Limited, London (1976)

    MATH  Google Scholar 

  8. Ma, H., An, G., Wu, S., Yang, F.: A region histogram of oriented optical flow (RHOOF) feature for apex frame spotting in micro-expression. In: Proceedings of the International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), Xiamen, pp. 281–286 (2017)

    Google Scholar 

  9. MEVIEW homepage. http://cmp.felk.cvut.cz/~cechj/ME/

  10. Moilanen, A., Zhao, G., Pietikainen, M.: Spotting rapid facial movements from videos using appearance-based feature difference analysis. In: Proceedings of the 22nd International Conference on Pattern Recognition (ICPR), Stockholm, pp. 1722–1727 (2014)

    Google Scholar 

  11. Oh, Y.-H., See, J., Le Ngo, A.C., Phan, R.C.-W., Baskaran, V.M.: A survey of automatic facial micro-expression analysis: databases, methods, and challenges. Front. Psychol. 9, article no. 1128 (2018)

    Google Scholar 

  12. Polikovsky, S., Kameda, Y.: Facial micro-expression detection in hi-speed video based on facial action coding system (FACS). IEICE Trans. Inf. Syst. 96, 81–92 (2013)

    Article  Google Scholar 

  13. Porter, S., Ten Brinke, L.: Reading between the lies identifying concealed and falsified emotions in universal facial expressions. Psychol. Sci. 19, 508–514 (2008)

    Article  Google Scholar 

  14. Shreve, M., Godavarthy, S., Goldgof, D., Sarkar, S.: Macro-and micro-expression spotting in long videos using spatio-temporal strain. In: Proceedings of the 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), Santa Barbara, pp. 51–56 (2011)

    Google Scholar 

  15. Valstar, M.F., Pantic, M.: Fully automatic recognition of the temporal phases of facial actions. IEEE Trans. Syst. Man Cybern. Part B 42, 28–43 (2012)

    Article  Google Scholar 

  16. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  17. Wang, S.-J., Wu, S., Qian, X., Li, J., Fu, X.: A main directional maximal difference analysis for spotting facial movements from long-term videos. Neurocomputing 230, 382–389 (2016)

    Article  Google Scholar 

  18. Weinberger, S.: Airport security: intent to deceive? Nature 465, 412–415 (2010)

    Article  Google Scholar 

  19. Yan, W.-J., Wu, Q., Liang, J., Chen, Y.-H., Fu, X.: How fast are the leaked facial expressions: the duration of micro-expressions. J. Nonverbal Behav. 37, 217–230 (2013). https://doi.org/10.1007/s10919-013-0159-8

    Article  Google Scholar 

  20. Yan, W.J., Chen, Y.H.: Measuring dynamic micro-expressions via feature extraction methods. J. Comput. Sci. 25, 318–326 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Basic and Applied Science for Engineering, University of Rome La Sapienza, Via A. Scarpa 16, 00161, Rome, Italy

    Vittoria Bruni & Domenico Vitulano

  2. Institute for Calculus Applications, CNR, Rome, Italy

    Vittoria Bruni & Domenico Vitulano

Authors
  1. Vittoria Bruni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Domenico Vitulano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vittoria Bruni .

Editor information

Editors and Affiliations

  1. University of Porto, Porto, Portugal

    Aurélio Campilho

  2. University of Waterloo, Waterloo, ON, Canada

    Fakhri Karray

  3. University of Waterloo, Waterloo, ON, Canada

    Zhou Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bruni, V., Vitulano, D. (2020). SSIM Based Signature of Facial Micro-Expressions. In: Campilho, A., Karray, F., Wang, Z. (eds) Image Analysis and Recognition. ICIAR 2020. Lecture Notes in Computer Science(), vol 12131. Springer, Cham. https://doi.org/10.1007/978-3-030-50347-5_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-50347-5_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-50346-8

  • Online ISBN: 978-3-030-50347-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation