Embodied Conversational Agent-Based Deception Detection

Concealed Information Test Using Automated Electrodermal, Vocal, and Oculmentric Measurements
  • Aaron C. ElkinsEmail author
  • Jeffrey G. Proudfoot
  • Nathan Twyman
  • Judee K. Burgoon
  • Jay F. NunamakerJr.
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 16)


An Embodied Conversational Agent-based human deception detection system was tested in a laboratory experiment during which some subjects had committed a mock crime and had guilty knowledge of that crime. Sensor signals were analyzed for their ability to discriminate between guilty/deceptive and innocent/truthful examinees and suitability for conducting automated agent-based deception interviews. Electrodermal activity confirmed the presence of arousal during the agent-based interviews.


Agent-based deception interviewing Deception detection Human risk assessment Credibility assessment 


  1. 1.
    Derrick, D.C., Elkins, A.C., Burgoon, J.K., Nunamaker Jr., J.F., Zeng, D.: Border security credibility assessments via heterogeneous sensor fusion. IEEE Intell. Syst. 25(3), 41–49 (2010)CrossRefGoogle Scholar
  2. 2.
    Nunamaker Jr., J.F., Derrick, D.C., Elkins, A.C., Burgoon, J.K., Patton, M.W.: Embodied conversational agent-based kiosk for automated interviewing. J. Manag. Inf. Syst. 28(1), 17–48 (2011)CrossRefGoogle Scholar
  3. 3.
    Twyman, N.W., Lowry, P.B., Burgoon, J.K., Nunamaker Jr., J.F.: Autonomous scientifically controlled screening systems for detecting information purposely concealed by individuals. J. Manag. Inf. Syst. 31(3), 106–137 (2014)CrossRefGoogle Scholar
  4. 4.
    Derrick, D.C., Jenkins, J.L., Nunamaker Jr., J.F.: Design principles for special purpose, embodied, conversational intelligence with environmental sensors (SPECIES). AIS Trans. Hum.-Comput. Interact. 3(2), 62–81 (2011)Google Scholar
  5. 5.
    Sokolov, E.N.: Higher nervous functions: orienting reflex. Annu. Rev. Physiol. 25(1), 545–580 (1963)CrossRefGoogle Scholar
  6. 6.
    Posner, M.I.: Orienting of attention. Q. J. Exp. Psychol. 32(1), 3–25 (1980)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Cannon, W.B.: Bodily changes in pain, hunger, fear and rage: An account of recent research into the function of emotional excitement. Appleton-Century-Crofts, New York (1929)Google Scholar
  8. 8.
    Gardner, R.M., Beltramo, J.S., Krinsky, R.: Pupillary changes during encoding, storing, and retrieval of information. Percept. Mot. Skills 41(3), 951–955 (1975)CrossRefGoogle Scholar
  9. 9.
    Goldwater, B.C.: Psychological significance of pupillary movements. Psychol. Bull. 77(5), 340–355 (1972)CrossRefGoogle Scholar
  10. 10.
    Nieuwenhuis, S., De Geus, E.J., Aston-Jones, G.: The anatomical and functional relationship between the P3 and autonomic components of the orienting response. Psychophysiology 48(2), 162–175 (2011)CrossRefGoogle Scholar
  11. 11.
    Nunnally, J.C., Knott, P.D., Duchowski, A.: Pupillary response as a general measure of activation. Atten. Percept. Psychophys. 2(4), 149–155 (1967)CrossRefGoogle Scholar
  12. 12.
    Bradley, M.T., Janisse, M.P.: Accuracy demonstrations, threat, and the detection of deception: cardiovascular, electrodermal, and pupillary measures. Psychophysiology 18(3), 307–315 (1981)CrossRefGoogle Scholar
  13. 13.
    Janisse, M.P., Bradley, M.T.: Deception, information and the pupillary response. Percept. Mot. Skills 50(3), 748–750 (1980)CrossRefGoogle Scholar
  14. 14.
    Lubow, R.E., Fein, O.: Pupillary size in response to a visual guilty knowledge test: new technique for the detection of deception. J. Exp. Psychol. Appl. 2(2), 164–177 (1996)CrossRefGoogle Scholar
  15. 15.
    Apple, W., Streeter, L.A., Krauss, R.M.: Effects of pitch and speech rate on personal attributions. J. Pers. Soc. Psychol. 37(5), 715–727 (1979)CrossRefGoogle Scholar
  16. 16.
    Bachorowski, J.A., Owren, M.J.: Vocal expression of emotion: Acoustic properties of speech are associated with emotional intensity and context. Psychol. Sci. 6(4), 219–224 (1995)CrossRefGoogle Scholar
  17. 17.
    DePaulo, B.M., Lindsay, J.J., Malone, B.E., Muhlenbruck, L., Charlton, K., Cooper, H.: Cues to deception. Psychol. Bull. 129(1), 74–118 (2003)CrossRefGoogle Scholar
  18. 18.
    Elkins, A.C., Stone, J.: The Effect of Cognitive Dissonance on Argument Language and Vocalics. Kauai, Koloa (2011)Google Scholar
  19. 19.
    Rockwell, P., Buller, D.B., Burgoon, J.K.: Measurement of deceptive voices: Comparing acoustic and perceptual data. Appl. Psycholinguist. 18(4), 471–484 (1997)CrossRefGoogle Scholar
  20. 20.
    Zuckerman, M., DePaulo, B.M., Rosenthal, R.: Verbal and nonverbal communication of deception. In: Berkowitz, L. (ed.) Advances in Experimental Social Psychology, vol. 14, pp. 1–59. Academic Press, New York (1981)Google Scholar
  21. 21.
    Buller, D.B., Burgoon, J.K.: Interpersonal deception theory. Commun. Theory 6(3), 203–242 (1996)CrossRefGoogle Scholar
  22. 22.
    Elkins, A.C., Derrick, D.C., Gariup, M.: The voice and eye gaze behavior of an imposter: automated interviewing and detection for rapid screening at the border. In: Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France (2012)Google Scholar
  23. 23.
    Gamer, M., Rill, H.G., Vossel, G., Gödert, H.W.: Psychophysiological and vocal measures in the detection of guilty knowledge. Int. J. Psychophysiol. 60(1), 76–87 (2006)CrossRefGoogle Scholar
  24. 24.
    Lykken, D.T.: The GSR in the detection of guilt. J. Appl. Psychol. 43, 385–388 (1959)CrossRefGoogle Scholar
  25. 25.
    Krapohl, D.J., McCloughlan, J.B., Senter, S.M.: How to use the concealed information test. Polygraph 35(3), 34–49 (2009)Google Scholar
  26. 26.
    Cronbach, L.J.: Coefficient alpha and the internal structure of tests. Psychometrika 16(3), 297–334 (1951)CrossRefzbMATHGoogle Scholar
  27. 27.
    Rosenthal, R.: Handbook of Methods in Nonverbal Behavior. In: Ekman, P., Scherer, K. (eds.), Cambridge, New York (1982)Google Scholar
  28. 28.
    Nunnally, J.C.: Psychometric Theory. McGraw-Hill, New York (1978)Google Scholar
  29. 29.
    Mossman, D., Somoza, E.: ROC curves, test accuracy, and the description of diagnostic tests. J. Neuropsychiatry Clin. Neurosci. 3, 330–333 (1991)CrossRefGoogle Scholar
  30. 30.
    Somoza, E., Mossman, D.: Neuropsychiatric decision making: Designing nonbinary diagnostic tests. J. Neuropsychiatry 3, 197–200 (1991)CrossRefGoogle Scholar
  31. 31.
    Burgoon, J.K., Floyd, K.: Testing for the motivation impairment effect during deceptive and truthful interaction. West. J. Commun. 64(3), 243–267 (2000)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Aaron C. Elkins
    • 1
    Email author
  • Jeffrey G. Proudfoot
    • 2
  • Nathan Twyman
    • 3
  • Judee K. Burgoon
    • 4
  • Jay F. NunamakerJr.
    • 4
  1. 1.Department of Management Information SystemsSan Diego State UniversitySan DiegoUSA
  2. 2.Information and Process Management DepartmentBentley UniversityWalthamUSA
  3. 3.Department of Business and Information TechnologyMissouri University of Science and TechnologyRollaUSA
  4. 4.Department of Management Information SystemsUniversity of ArizonaTucsonUSA

Personalised recommendations