Detecting the Doubt Effect and Subjective Beliefs Using Neural Networks and Observers’ Pupillary Responses

  • Xuanying Zhu
  • Zhenyue Qin
  • Tom GedeonEmail author
  • Richard Jones
  • Md Zakir Hossain
  • Sabrina Caldwell
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11304)


We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.


Neural networks Pupillary responses Information veracity Doubt Trust Subjective belief Semi-unsupervised training 


  1. 1.
    ten Brinke, L., Vohs, K.D., Carney, D.R.: Can ordinary people detect deception after all? Trends Cogn. Sci. 20, 579–588 (2016)CrossRefGoogle Scholar
  2. 2.
    Von Hippel, W., Trivers, R.: The evolution and psychology of self-deception. Behav. Brain Sci. 34, 1–16 (2011)CrossRefGoogle Scholar
  3. 3.
    Bond Jr., C.F., DePaulo, B.M.: Accuracy of deception judgments. Pers. Soc. Psychol. Rev. 10, 214–234 (2006)CrossRefGoogle Scholar
  4. 4.
    DePaulo, B.M., Bond Jr., C.F.: Beyond accuracy: bigger, broader ways to think about deceit. J. Appl. Res. Mem. Cogn. 1, 120–121 (2012)CrossRefGoogle Scholar
  5. 5.
    van’t Veer, A.: Effortless morality: cognitive and affective processes in deception and its detection. Dissertation, Tilburg. University (2016)Google Scholar
  6. 6.
    Albrechtsen, J.S., Meissner, C.A., Susa, K.J.: Can intuition improve deception detection performance? J. Exp. Soc. Psychol. 45, 1052–1055 (2009)CrossRefGoogle Scholar
  7. 7.
    Chow, C., Gedeon, T.: Classifying document categories based on physiological measures of analyst responses. In: 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pp. 421–425 (2015)Google Scholar
  8. 8.
    Kreibig, S.D.: Autonomic nervous system activity in emotion: a review. Biol. Psychol. 84, 14–41 (2010)Google Scholar
  9. 9.
    Tomaka, J., Blascovich, J., Kelsey, R.M., Leitten, C.L.: Subjective, physiological, and behavioral effects of threat and challenge appraisal. J. Pers. Soc. Psychol. 65, 248 (1993)CrossRefGoogle Scholar
  10. 10.
    Sleegers, W., Proulx, T.: The comfort of approach: self-soothing effects of behavioral approach in response to meaning violations. Front. Psychol. 5, 1568 (2015)CrossRefGoogle Scholar
  11. 11.
    Caldwell, S., Gedeon, T., Jones, R., Copeland, L.: Imperfect understandings: a grounded theory and eye gaze investigation of human perceptions of manipulated and unmanipulated digital images. In: Proceedings of the World Congress on Electrical Engineering and Computer Systems and Science (2015)Google Scholar
  12. 12.
    Duran, G., Tapiero, I., Michael, G.A.: Resting heart rate: a physiological predicator of lie detection ability. Physiol. Behav. 186, 10–15 (2018)CrossRefGoogle Scholar
  13. 13.
    DiNicolantonio, J.: The Salt Fix. Harmony (2017)Google Scholar
  14. 14.
    Stanford, S.M., et al.: Diabetes reversal by inhibition of the low-molecular-weight tyrosine phosphatase. Nat. Chem. Biol. 13, 624 (2017)CrossRefGoogle Scholar
  15. 15.
    Laeng, B., Sirois, S., Gredebäck, G.: Pupillometry: a window to the preconscious? Perspect. Psychol. Sci. 7, 18–27 (2012)CrossRefGoogle Scholar
  16. 16.
  17. 17.
    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv Preprint arXiv1412.6980 (2014)Google Scholar
  18. 18.
    Hossain, M.Z., Gedeon, T.: Classifying posed and real smiles from observers’ peripheral physiology. In: 11th International Conference on Pervasive Computing Technologies for Healthcare (2017)Google Scholar
  19. 19.
    Chen, L., Gedeon, T., Hossain, M.Z., Caldwell, S.: Are you really angry?: detecting emotion veracity as a proposed tool for interaction. In: Proceedings of the 29th Australian Conference on Computer-Human Interaction, pp. 412–416 (2017)Google Scholar
  20. 20.
    Bradley, M.M., Miccoli, L., Escrig, M.A., Lang, P.J.: The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology 45, 602–607 (2008)CrossRefGoogle Scholar
  21. 21.
    Beatty, J., Lucero-Wagoner, B., et al.: The pupillary system. In: Handbook Psychophysiology (2rd edn.) (2000)Google Scholar
  22. 22.
    Wiseman, R., Watt, C.: Judging a book by its cover: the unconscious influence of pupil size on consumer choice. Perception 39, 1417–1419 (2010)CrossRefGoogle Scholar
  23. 23.
    Gründl, M., Knoll, S., Eisenmann-Klein, M., Prantl, L.: The blue-eyes stereotype: do eye color, pupil diameter, and scleral color affect attractiveness? Aesthetic Plast. Surg. 36, 234–240 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xuanying Zhu
    • 1
  • Zhenyue Qin
    • 1
  • Tom Gedeon
    • 1
    Email author
  • Richard Jones
    • 1
  • Md Zakir Hossain
    • 1
  • Sabrina Caldwell
    • 1
  1. 1.The Australian National UniversityCanberraAustralia

Personalised recommendations