Skip to main content
SpringerLink
Log in

Explaining with Attribute-Based and Relational Near Misses: An Interpretable Approach to Distinguishing Facial Expressions of Pain and Disgust

  • Conference paper
  • First Online:
Inductive Logic Programming (ILP 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13779))

Included in the following conference series:

  • 46 Accesses

Abstract

Explaining concepts by contrasting examples is an efficient and convenient way of giving insights into the reasons behind a classification decision. This is of particular interest in decision-critical domains, such as medical diagnostics. One particular challenging use case is to distinguish facial expressions of pain and other states, such as disgust, due to high similarity of manifestation. In this paper, we present an approach for generating contrastive explanations to explain facial expressions of pain and disgust shown in video sequences. We implement and compare two approaches for contrastive explanation generation. The first approach explains a specific pain instance in contrast to the most similar disgust instance(s) based on the occurrence of facial expressions (attributes). The second approach takes into account which temporal relations hold between intervals of facial expressions within a sequence (relations). The input to our explanation generation approach is the output of an interpretable rule-based classifier for pain and disgust. We utilize two different similarity metrics to determine near misses and far misses as contrasting instances. Our results show that near miss explanations are shorter than far miss explanations, independent from the applied similarity metric. The outcome of our evaluation indicates that pain and disgust can be distinguished with the help of temporal relations. We currently plan experiments to evaluate how the explanations help in teaching concepts and how they could be enhanced by further modalities and interaction.

The work presented in this paper was funded partially by grant FKZ 01IS18056 B, BMBF ML-3 (TraMeExCo) and partially by grant DFG (German Research Foundation) 405630557 (PainFaceReader). We would like to thank Mark Gromowski who helped us with preparing the used data set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Achterberg, W.P., Erdal, A., Husebo, B.S., Kunz, M., Lautenbacher, S.: Are chronic pain patients with dementia being undermedicated? J. Pain Res. 14, 431 (2021)

    Article  Google Scholar 

  2. Arrieta et al., A.B.: Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. arXiv preprint arXiv:1910.10045 (2019)

  3. Lucey, P., et al.: Painful data: The UNBC-McMaster shoulder pain expression archive database. In: FG, pp. 57–64. IEEE (2011)

    Google Scholar 

  4. Bach, S., et al.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One 10(7), e0130140 (2015)

    Google Scholar 

  5. Allen, J.F.: Towards a general theory of action and time. Artif. Intell. 23(2), 123–154 (1984)

    Article  Google Scholar 

  6. Barnwell, J.A.: Using near misses to teach concepts to a human intelligence system. Ph.D. thesis, Massachusetts Institute of Technology (2018)

    Google Scholar 

  7. Bradley, E.L.: Overlapping coefficient. Am. Cancer Soc. (2006). https://doi.org/10.1002/0471667196.ess1900.pub2

    Article  Google Scholar 

  8. Chen, Z., Ansari, R., Wilkie, D.: Automated pain detection from facial expressions using facs: a review. arXiv preprint arXiv:1811.07988 (2018)

  9. Cropper, A., Dumancic, S., Muggleton, S.H.: Turning 30: new ideas in inductive logic programming. In: Bessiere, C. (ed.) Proc. of the Twenty-Ninth Int. Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 4833–4839. ijcai.org (2020). https://doi.org/10.24963/ijcai.2020/673

  10. Dhurandhar, A., et al.: Explanations based on the missing: towards contrastive explanations with pertinent negatives. In: Advances in Neural Information Processing Systems, pp. 592–603 (2018)

    Google Scholar 

  11. Ekman, P., Friesen, W.V.: Measuring facial movement. Environ. Psychol. Nonverbal Behav. 1(1), 56–75 (1976). https://doi.org/10.1007/BF01115465

    Article  Google Scholar 

  12. Erickson, B.J., Korfiatis, P., Akkus, Z., Kline, T.L.: Machine learning for medical imaging. Radiographics 37(2), 505–515 (2017)

    Article  Google Scholar 

  13. Fillingim, R.B.: Individual differences in pain: understanding the mosaic that makes pain personal. Pain 158(1), 11–18 (2017)

    Google Scholar 

  14. Finzel, B., Tafler, D.E., Thaler, A.M., Schmid, U.: Multimodal explanations for user-centric medical decision support systems. In: Doyle, T.E., et al. (eds.) Proc. of the AAAI 2021 Fall Symposium on Human Partnership with Medical AI: Design, Operationalization, and Ethics (AAAI-HUMAN 2021), November 4–6, 2021, vol. 3068. CEUR-WS.org (2021), ceur-ws.org/Vol-3068/short2.pdf

    Google Scholar 

  15. Gentner, D., Markman, A.B.: Structure mapping in analogy and similarity. Am. Psychol. 52(1), 45 (1997)

    Article  Google Scholar 

  16. Gurevich, N., Markovitch, S., Rivlin, E.: Active learning with near misses. In: Proc. of the National Conference on Artificial Intelligence, vol. 21, p. 362. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999 (2006)

    Google Scholar 

  17. Hammond, T., Davis, R.: Interactive learning of structural shape descriptions from automatically generated near-miss examples. In: Proc. of the 11th Int. Conference on Intelligent User Interfaces, pp. 210–217 (2006)

    Google Scholar 

  18. Hassan, T., et al.: Automatic detection of pain from facial expressions: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(6), 1815–1831 (2019)

    Article  Google Scholar 

  19. Herchenbach, M., Müller, D., Scheele, S., Schmid, U.: Explaining image classifications with near misses, near hits and prototypes: Supporting domain experts in understanding decision boundaries. In: Proc. of Pattern Recognition and Artificial Intelligence: Third Int. Conference, ICPRAI 2022, Paris, France, June 1–3, 2022, Part II, pp. 419–430. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-09282-4_35

  20. Holzinger, A., Biemann, C., Pattichis, C.S., Kell, D.B.: What do we need to build explainable ai systems for the medical domain? arXiv:1712.09923 (2017)

  21. Jaccard, P.: Distribution de la flore alpine dans le bassin des dranses et dans quelques régions voisines. Bulletin de la Socit Vaudoise des Sciences Naturelles 37, 241–272 (1901)

    Google Scholar 

  22. Karmann, A.J., Maihöfner, C., Lautenbacher, S., Sperling, W., Kornhuber, J., Kunz, M.: The role of prefrontal inhibition in regulating facial expressions of pain: a repetitive transcranial magnetic stimulation study. J. Pain 17(3), 383–391 (2016)

    Article  Google Scholar 

  23. Kira, K., Rendell, L.A.: A practical approach to feature selection. In: Sleeman, D., Edwards, P. (eds.) Machine Learning Proceedings 1992, pp. 249–256. Morgan Kaufmann, San Francisco (CA) (1992). https://doi.org/10.1016/B978-1-55860-247-2.50037-1

  24. Kuhn, S.P.: Identifying near misses for relational concepts with graph matching - explaining classifier decisions of facial expressions (2019)

    Google Scholar 

  25. Kunz, M., Meixner, D., Lautenbacher, S.: Facial muscle movements encoding pain-a systematic review. Pain 160(3), 535–549 (2019). https://doi.org/10.1097/j.pain.0000000000001424

    Article  Google Scholar 

  26. Kunz, M., Peter, J., Huster, S., Lautenbacher, S.: Pain and disgust: the facial signaling of two aversive bodily experiences. PloS One 8(12) (2013)

    Google Scholar 

  27. Lautenbacher, S., et al.: Automatic coding of facial expressions of pain: Are we there yet? Pain Research & Management 2022 (2022)

    Google Scholar 

  28. Miller, T.: Explanation in artificial intelligence: insights from the social sciences. arXiv:1706.07269 [cs], June 2017. arxiv.org/abs/1706.07269

  29. Muggleton, S.: Inductive logic programming. N. Gener. Comput. 8(4), 295–318 (1991). https://doi.org/10.1007/BF03037089

    Article  Google Scholar 

  30. Muggleton, S.H., Schmid, U., Zeller, C., Tamaddoni-Nezhad, A., Besold, T.: Ultra-strong machine learning: comprehensibility of programs learned with ilp. Mach. Learn. 107(7), 1119–1140 (2018)

    Article  MathSciNet  Google Scholar 

  31. Rabold, J., Siebers, M., Schmid, U.: Generating contrastive explanations for inductive logic programming based on a near miss approach. Mach. Learn. 111(5), 1799–1820 (2022)

    Article  MathSciNet  Google Scholar 

  32. Rieger, I., Finzel, B., Seuß, D., Wittenberg, T., Schmid, U.: Make pain estimation transparent: a roadmap to fuse bayesian deep learning and inductive logic programming. In: IEEE EMBS (2019)

    Google Scholar 

  33. Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Mach. Intell. 1(5), 206–215 (2019)

    Google Scholar 

  34. Schmid, U.: Inductive programming as approach to comprehensible machine learning. In: DKB/KIK@ KI, pp. 4–12 (2018)

    Google Scholar 

  35. Schmid, U., Finzel, B.: Mutual explanations for cooperative decision making in medicine. KI - Künstliche Intelligenz (2020)

    Google Scholar 

  36. Schwalbe, G., Finzel, B.: A comprehensive taxonomy for explainable artificial intelligence: A systematic survey of surveys on methods and concepts. CoRR abs/2105.07190 (2021). arxiv.org/abs/2105.07190

  37. Siebers, M., Schmid, U., Seuß, D., Kunz, M., Lautenbacher, S.: Characterizing facial expressions by grammars of action unit sequences-a first investigation using abl. Inf. Sci. 329, 866–875 (2016)

    Google Scholar 

  38. Srinivasan, A.: The Aleph Manual. www.cs.ox.ac.uk/activities/machinelearning/Aleph/

  39. Tsai, I.P., Jeong, S.Y.S., Hunter, S.: Pain assessment and management for older patients with dementia in hospitals: an integrative literature review. Pain Manage. Nursing 19(1), 54–71 (2018). https://www.sciencedirect.com/science/article/pii/S152490421730022X

  40. Wang, Z., Wang, S., Ji, Q.: Capturing complex spatio-temporal relations among facial muscles for facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3422–3429 (2013)

    Google Scholar 

  41. Winston, P.H.: Learning structural descriptions from examples. In: Winston, P. (ed.) The Psychology of Computer Vision, pp. 157–210. McGraw-Hil, New York (1975)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Systems, University of Bamberg, An der Weberei 5, 96047, Bamberg, Germany

    Bettina Finzel, Simon P. Kuhn, David E. Tafler & Ute Schmid

Authors
  1. Bettina Finzel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon P. Kuhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David E. Tafler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ute Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bettina Finzel .

Editor information

Editors and Affiliations

  1. Imperial College London, London, UK

    Stephen H. Muggleton

  2. University of Surrey, Guildford, UK

    Alireza Tamaddoni-Nezhad

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Finzel, B., Kuhn, S.P., Tafler, D.E., Schmid, U. (2024). Explaining with Attribute-Based and Relational Near Misses: An Interpretable Approach to Distinguishing Facial Expressions of Pain and Disgust. In: Muggleton, S.H., Tamaddoni-Nezhad, A. (eds) Inductive Logic Programming. ILP 2022. Lecture Notes in Computer Science(), vol 13779. Springer, Cham. https://doi.org/10.1007/978-3-031-55630-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-55630-2_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-55629-6

  • Online ISBN: 978-3-031-55630-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation