Comparing Signal-Contingent and Event-Contingent Experience Sampling Ratings of Affect in a Sample of Psychotherapy Outpatients

  • Sindes DawoodEmail author
  • Michael N. Hallquist
  • Aaron L. Pincus
  • Nilam Ram
  • Michelle G. Newman
  • Stephen J. Wilson
  • Kenneth N. Levy


Experience sampling methods are widely used in clinical psychology to study affective dynamics in psychopathology. The present study examined whether affect ratings (valence and arousal) differed as a function of assessment schedule (signal- versus event-contingent) in a clinical sample and considered various approaches to modeling these ratings. A total of 40 community mental health center outpatients completed ratings of their affective experiences over a 21-day period using both signal-contingent schedules (random prompts) and event-contingent schedules (ratings following social interactions). We tested whether assessment schedules impacted 1) the central tendency (mean) and variability (standard deviation) of valence or arousal considered individually, 2) the joint variability in valence and arousal via the entropy metric, and 3) the between-person differences in configuration of valence-arousal landscapes via the Earth Mover’s Distance (EMD) metric. We found that event-contingent schedules, relative to signal-contingent schedules, captured higher average levels of pleasant valence and emotional arousal ratings. Moreover, signal-contingent schedules captured greater variability within and between individuals on arousal-valence landscapes compared to event-contingent schedules. Altogether, findings suggest that the two assessment schedules should not be treated interchangeably in the assessment of affect over time. Researchers must be cautious in generalizing results across studies utilizing different experience sampling assessment schedules.


Signal-contingent schedules Event-contingent schedules Affective variability Entropy Earth Mover’s Distance 



We thank J. Wesley Scala, M.S., and Emily A. Dowgwillo, M.S., for assistance in developing training material and coordinating the collection of data, and Kathleen Bohomey, Colin Carey, Caroline Curran, Wendi Falk, Sarah Forsythe, Laura Frey, Caroline Gooch, Jessica Grom, Brittani Hollern, Lauren Lipner, Kristin McLaughlin, Megan Moyer, Joanna Pantelides, Megan Parker, Jacqueline Proczynski, Carolina Ribo, Silvia Rizkallah, Aimee Sohnleitner, and Alyssa Spaw for their assistance in data collection. We also thank Jennifer Fox and Allison Clark for their assistance in the recruitment of participants. Finally, we thank Drs. William D. Ellison, Peter Molenaar and Joshua Smyth for consultation.

Funding Information

This research was supported by grants from The Pennsylvania State University Social Science Research Institute (principal investigator: Levy), and secondary grants to William D. Ellison from a dissertation grant from The Pennsylvania State University Research and Graduate Studies Office, and grants from American Psychoanalytic Association and International Psychoanalytic Association.

Compliance with Ethical Standards

Conflict of Interest

Sindes Dawood, Michael N. Hallquist, Aaron L. Pincus, Nilam Ram, Michelle G. Newman, Stephen J. Wilson, and Kenneth N. Levy declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.


  1. Barlow, D. H., & Farchione, T. J. (2018). Applications of the unified protocol for transdiagnostic treatment of emotional disorder. New York: Oxford University Press.Google Scholar
  2. Berner, L. A., Crosby, R. D., Cao, L., Engel, S. G., Lavender, J. M., Mitchell, J. E., & Wonderlich, S. A. (2017). Temporal associations between affective instability and dysregulated eating behavior in bulimia nervosa. Journal of Psychiatric Research, 92, 183–190. Scholar
  3. Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616. Scholar
  4. Brown, T. A., Di Nardo, P. A., & Barlow, D. H. (1994). Anxiety disorders interview schedule for DSM-IV (adult version). Albany: Graywind.Google Scholar
  5. Carver, C. S., & Scheier, M. F. (1998). On the self-regulation of behavior. New York: Cambridge University Press.CrossRefGoogle Scholar
  6. Clifton, A., Pilkonis, P. A., & McCarty, C. (2007). Social networks in borderline personality disorder. Journal of Personality Disorders, 21, 434–441. Scholar
  7. Conner, T., & Bliss-Moreau, E. (2006). Sampling human experience in naturalistic settings. In S. Hesse-Biber & P. Leavy (Eds.), Emergent methods in social research (pp. 109–129). Thousand Oaks: Sage.CrossRefGoogle Scholar
  8. Crowe, E., Daly, M., Delaney, L., Carroll, S., & Malone, K.M. (2018). Psychiatry Research. Advanced online publication. Scholar
  9. Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175, 526–536. Scholar
  10. Dawood, S., & Pincus, A. L. (2018). Pathological narcissism and the severity, variability, and instability of depressive symptoms. Personality Disorders: Theory, Research, and Treatment, 9, 144–154. Scholar
  11. Ebner-Priemer, U. W., Kuo, J., Welch, S. S., Thielgen, T., Witte, S., Bohus, M., & Linehan, M. M. (2006). A valence-dependent group specific recall bias of retrospective self-reports: A study of borderline personality disorder in everyday life. Journal of Nervous and Mental Disease, 194, 774–779. Scholar
  12. Ebner-Priemer, U. W., Kuo, J., Kleindienst, N., Welch, S. S., Reisch, T., Reinhard, I., Lieb., Linehan, & Bohus (2007). State affective instability in borderline personality disorder assessed by ambulatory monitoring. Psychological Medicine, 37, 961–970. Scholar
  13. Ebner-Priemer, U. W., Kuo, J., Scholtz, W., Kleindienst, N., Rosenthal, Z., Detterer, L., Linehan, M., & Bohus, M. (2008). Distress and affective dysregulation in patients with borderline personality disorder: A psychophysiological ambulatory monitoring study. Journal of Nervous and Mental Disorders, 196, 314–320. Scholar
  14. Ellison, W.D., Levy, K.N., Newman, M.G., Pincus, A.L., Wilson, S.J., & Molenaar, P.C.M. (2019). Dynamics among borderline personality and anxiety features in psychotherapy outpatients: An exploration of nomothetic and idiographic patterns. Personality Disorders: Theory, Research, and Treatment. Advanced online publication. doi:
  15. Farmer, A. S., & Kashdan, T. B. (2014). Affective and self-esteem instability in the daily lives of people with generalized social anxiety disorder. Clinical Psychological Science, 2, 187–201. Scholar
  16. Himmelstein, P. H., Woods, W. C., & Wright, A. G. C. (2019). A comparison of signal- and event-contingent ambulatory assessment of interpersonal behavior and affect in social situations. Psychological Assessment, 31, 952–960. Scholar
  17. Hollenstein, T. (2007). State space grids: Analyzing dynamics across development. International Journal of Behavioral Development, 31, 384–396. Scholar
  18. Knowles, R., Tai, S., Jones, S. H., Highfield, J., Morriss, R., & Bentall, R. P. (2007). Stability of self-esteem in bipolar disorder: Comparisons among remitted bipolar patients, remitted unipolar patients and healthy controls. Bipolar Disorder, 9, 490–495. Scholar
  19. Kopala-Sibley, D. C., Zuroff, D. C., Russell, J. J., Moskowitz, D. S., & Paris, J. (2012). Understanding heterogeneity in borderline personality disorder: Differences in affective reactivity explained by the traits of dependency and self-criticism. Journal of Abnormal Psychology, 121, 680–691. Scholar
  20. Koval, P., Pe, M. L., Meers, K., & Kuppens, P. (2013). Affective dynamics in relation to depressive symptoms: Variable, unstable, or inert? Emotion, 13, 1132–1141. Scholar
  21. Lawrence, M. A. (2016). ez: Easy analysis and visualization of factorial experiments. URL.
  22. Liu, H., Xie, Q. W., & Lou, V. W. Q. (2019). Everyday social interactions and intra-individual variability in affect: A systematic review and meta-analysis of ecological momentary assessment studies. Motivation and Emotion, 43, 339–353. Scholar
  23. Loranger, A. W. (1999). International Personality Disorder Examination (IPDE). Odessa: Psychological Assessment Resources.Google Scholar
  24. Mason, T., Lavender, J. M., Wonderlich, S. A., Steiger, H., Cao, L., Engel, S. G., Mitchell, J. E., & Crosby, R. D. (2017). Comfortably numb: The role of momentary dissociation in the experience of negative affect around binge eating. The Journal of Nervous and Mental Disease, 205, 335–339. Scholar
  25. Mehl, M. R., & Conner, T. S. (2012). Handbook of research methods for studying daily life. New York: The Guildford Press.Google Scholar
  26. Mohr, C. D., Aprin, S., & McCabe, C. T. (2015). Daily affect variability and context-specific alcohol consumption. Drug and Alcohol Review, 34, 581–587. Scholar
  27. Mohr, C. D., Zhang, M., & Schueller, S. M. (2017). Personal sensing: Understanding mental health using ubiquitous sensors and machine learning. Annual Review of Clinical Psychology, 13, 23–47. Scholar
  28. Moskowitz, D. S., Russell, J. J., Sadikaj, G., & Sutton, R. (2009). Measuring people intensively. Canadian Psychology, 50, 131–140. Scholar
  29. Muehlenkamp, J. J., Engel, S. G., Crosby, R. D., Wonderlich, S. A., Simonich, H., & Mitchell, J. E. (2009). Emotional states preceding and following acts of non-suicidal self-injury in bulimia nervosa patients. Behavior Research and Therapy, 47, 83–87. Scholar
  30. Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespau, P., & van Os, J. (2009). Experience sampling research in psychopathology: Opening the black box of daily life. Psychological Medicine, 39, 1533–1547. Scholar
  31. Palmier-Claus, J. E., Taylor, P. J., Gooding, P., Dunn, G., & Lewis, S. W. (2012). Affective variability predicts suicidal ideation in individuals at ultra-high risk of developing psychosis: An experience sampling study. Journal of Clinical Psychology, 51, 72–83. Scholar
  32. Pincus, A. L., Hopwood, C. J., & Wright, A. G. C. (in press). The interpersonal situation: An integrative framework for the study of personality, psychopathology, and psychotherapy. In D. Funder, J. F. Rauthmann, & R. Sherman (Eds.), Oxford handbook of psychological situations. New York: Oxford University Press.Google Scholar
  33. Pisetsky, E. M., Crosby, R., Cao, L., Fitzsimmons-Craft, E. E., Mitchell, J. M., Engel, S. G., & Peterson, C. B. (2016). An examination of affect prior to and following episodes of getting drunk in women with bulimia nervosa. Psychiatry Research, 240, 202–208. Scholar
  34. Posner, J., Russell, J. A., & Peterson, B. S. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17, 715–734. Scholar
  35. Ram, N., Conroy, D. E., Pincus, A. L., Hyde, A. L., & Molloy, L. (2012). Tethering theory to method: Using measures of intraindividual variability to operationalize individuals’ dynamic characteristics. In G. Hancock & J. Harring (Eds.), Advances in longitudinal modeling in the social and behavioral sciences (pp. 81–110). Charlotte: Information Age Publishing.Google Scholar
  36. Ram, N., Coccia, M., Conroy, D. E., Lorek Dattiko, A., Orland, B., Pincus, A. L., Sliwinski, M., & Gerstorf, D. (2013). Behavioral landscapes and change in behavioral landscapes: A multiple time-scale density distribution approach. Research in Human Development, 10, 88–110. Scholar
  37. Ram, N., Benson, K., Brick, T. R., Conroy, D. E., & Pincus, A. L. (2017). Behavioral landscapes and earth mover’s distance. A new approach for studying individual differences in density distributions. Journal of Research in Personality, 69, 191–205. Scholar
  38. Reis, H. T., & Gable, S. L. (2000). Event-sampling and other methods for studying everyday experience. In H. T. Reis, & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 190–222). New York Cambridge Univ. Press.Google Scholar
  39. Reisch, T., Ebner-Priemer, U. W., Tschacher, W., Bohus, M., & Linehan, M. M. (2008). Sequences of emotions in patients with borderline personality disorder. Acta Psychiatrica Scandinavica, 118, 42–48. Scholar
  40. Roche, M. J., & Pincus, A. L. (2016). Precision assessment. An individualized and temporally dynamic approach to understanding patients in their daily lives. In U. Kumar (Ed.), The Wiley handbook of personality assessment (pp. 192–204). Chichester: Wiley, Ltd.CrossRefGoogle Scholar
  41. Rubner, Y., Tomasi, C., & Guibas, L. (2000). The earth mover’s distance as a metric for image retrieval. International Journal on Computer Vision, 40, 99–121.CrossRefGoogle Scholar
  42. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Scholar
  43. Russell, J. J., Moskowitz, D. S., Zuroff, D. C., Sookman, D., & Paris, J. (2007). Stability and variability of affective experiences and interpersonal behavior in borderline personality disorder. Journal of Abnormal Psychology, 116, 578–588. Scholar
  44. Sadikaj, G., Russell, J. J., Moskowitz, D. S., & Paris, J. (2010). Affect dysregulation in individuals with borderline personality disorder: Persistence and interpersonal triggers. Journal of Personality Assessment, 92, 490–500. Scholar
  45. Sadikaj, G., Moskowitz, D. S., Russell, J. J., Zuroff, D. C., & Paris, J. (2013). Quarrelsome behavior in borderline personality disorder: Influence of behavioral and affective reactivity to perceptions of others. Journal of Abnormal Psychology, 122, 195–207. Scholar
  46. Santangelo, P., Reinhard, I., Mussgay, L., Steil, R., Sawitzki, G., Klein, C., et al. (2014). Specificity of affective instability in patients with borderline personality disorder compared to posttraumatic stress disorder, bulimia nervosa, and healthy controls. Journal of Abnormal Psychology, 123, 258–272. Scholar
  47. Santangelo, P. S., Limberger, M. F., Stiglmayr, C., Houben, M., Coosemans, J., Verleysen, G., Kuppens, P., Tuerlinckx, F., Vanpaemel, W., & Ebner-Priemer, U. W. (2016). Analyzing subcomponents of affective dysregulation in borderline personality disorder in comparison to other clinical groups using multiple e-diary datasets. Borderline Personality Disorder and Emotion Dysregulation, 3, 1–13. Scholar
  48. Scala, J. W., Levy, K. N., Johnson, B. N., Kivity, Y., Ellison, W. D., Pincus, A. L., et al. (2018). The role of negative affect and self concept clarity in predicting self-injurious urges in borderline personality disorders using ecological momentary assessment. Journal of Personality Disorders, 32, 36–57. Scholar
  49. Schimmack, U., & Grob, A. (2000). Dimensional models of core affect: A quantitative by means of structural equation modeling. European Journal of Personality, 14, 325–345.<325::AID-PER380>3.0.CO;2-I.CrossRefGoogle Scholar
  50. Selby, E. A., Doyle, P., Crosby, R. D., Wonderlich, S. A., Engel, S. G., Mitchell, J. D., & Le Grange, D. (2012). Momentary emotion surrounding bulimic behaviors in women with bulimia nervosa and borderline personality disorder. Journal of Psychiatric Research, 46, 1492–1500. Scholar
  51. Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32. Scholar
  52. Smyth, J. M., Wonderlich, S. A., Heron, K. E., Sliwinski, M. J., Crosby, R. D., Mitchell, J. E., & Engel, S. G. (2007). Daily and momentary mood and stress are associated with binge eating and vomiting in bulimia nervosa patients in the natural environment. Journal of Consulting and Clinical Psychology, 75, 629–638. Scholar
  53. Stein, K. F. (1996). Affect instability in adults with a borderline personality disorder. Archives of Psychiatric Nursing, 10, 32–40. Scholar
  54. Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202. Scholar
  55. Thayer, R. E. (1989). The biopsychology of mood. New York: Oxford University Press.Google Scholar
  56. Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory assessment. Annual Review of Clinical Psychology, 9, 151–176. Scholar
  57. Trull, T. J., Solhan, M. B., Tragesser, S. L., Jahng, S., Wood, P. K., Piasecki, T. M., & Watson, D. (2008). Affective instability: Measuring a core feature of borderline personality disorder with ecological momentary assessment. Journal of Abnormal Psychology., 117, 647–661. Scholar
  58. Trull, T. J., Ebner-Priemer, U. W., Brown, W. C., Tomko, R. L., & Scheiderer, E. M. (2012). Clinical psychology. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 620–635). New York: Guilford Press.Google Scholar
  59. Urbanek, S., and Y. Rubner, 2012. emdist: Earth Mover’s Distance. R package version 0.2-1. URL
  60. van Os, J., Delespaul, P., Wigman, J., Myin-Germeys, I., & Wichers, M. (2013). Beyond DSM and ICD: Introducing "precision diagnosis" for psychiatry using momentary assessment technology. World Psychiatry, 12, 113–117. Scholar
  61. Vansteelandt, K., Claes, L., Muehlenkamp, J., Claes, L., Muehlenkamp, K., De Cuyper, K., Lemmens, J., Probst, M., Vanderlinden, J., & Pieters, G. (2013). Variability in affective activation predicts non-suicidal self-injury in eating disorders. European Eating Disorders Review, 21, 143–147. Scholar
  62. Wand, M. (2015). KernSmooth: Functions for kernel smoothing supporting Wand and Jones (1995). R package version 2.23-15, URL
  63. Wang, S., Roche, M. J., Pincus, A. L., Conroy, D. E., Rebar, A., & Ram, N. (2014). Interpersonal dependency and emotion in every day life. Journal of Research in Personality, 53, 5–12. Scholar
  64. Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood. Psychological Bulletin, 98, 219–235. Scholar
  65. Wright, A. G. C., Gates, K. M., Arizmendi, C., Lane, S. T., Woods, W. C., & Edershile, E. A. (2019). Focusing personality assessment on the person: Modeling general, shared, and person specific processes in personality and psychopathology. Psychological Assessment, 31, 502–515. Scholar
  66. Zautra, A. J., Berkhof, J., & Nicolson, N. A. (2002). Changes in affect interrelations as a function of stressful events. Cognition and Emotion, 16, 309–318. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyPennsylvania State UniversityUniversity ParkUSA

Personalised recommendations