The Usability of Multiple Devices for Assessment in Psychological Research: Salience of Reasons Underlying Usability

  • Daniela RaccanelloEmail author
  • Margherita Brondino
  • Margherita Pasini
  • Maria Gabriella Landuzzi
  • Diego Scarpanti
  • Giada Vicentini
  • Mara Massaro
  • Roberto Burro
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 804)


Focusing on an online survey in psychological research, we evaluated the usability of the device chosen to complete the survey and examined underlying reasons through a usability enquiry method. The participants were 149 undergraduate students who completed a questionnaire for assessing achievement emotions and motivation, with open-ended and closed-ended questions. They also evaluated the usability of the device chosen to complete the survey and reported underlying reasons. We analyzed the data with Generalized Linear Mixed Models. The devices chosen to complete the survey were perceived as highly usable, even if usability was lower for smartphones compared to other devices such as personal computers, notebooks, and tablets. The most relevant reasons regarded characteristics of the tools, followed by those of the tasks and then of users and environments. The findings are discussed taking into account their theoretical and applied relevance for monitoring and improving online psychological assessment.


Usability Multiple devices Online assessment Usability enquiry methods Questionnaires 


  1. 1.
    Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67(1), 1–48 (2015). Scholar
  2. 2.
    Bianchi, I., Savardi, U., Burro, R., Martelli, M.F.: Doing the opposite to what another person is doing. Acta Psychologica 151, 117–133 (2014). Scholar
  3. 3.
    Branchini, E., Burro, R., Bianchi, I., Savardi, U.: Contraries as an effective strategy in geometrical problem solving. Think. Reason. 21(4), 397–430 (2015). Scholar
  4. 4.
    Brondino, M., Raccanello, D., Pasini M.: Achievement goals as antecedents of achievement emotions: the 3 X 2 achievement goal model as a framework for learning environments design. In: Advances in Intelligent and Soft Computing, vol. 292, pp. 53–60. Scholar
  5. 5.
    Burro, R., Raccanello, D., Pasini, M., Brondino, M.: An estimation of a nonlinear dynamic process using Latent Class extended Mixed Models. Affect profiles after terrorist attacks. Nonlinear Dyn. Psychol. Life Sci. 22(1), 35–52 (2018)Google Scholar
  6. 6.
    Dombrowsky, U., Wagner, T.: Mental strain as field of action in the 4th industrial revolution. Procedia CIRP 17, 100–105 (2014)CrossRefGoogle Scholar
  7. 7.
    Ham, D.H.: A model-based framework for classifying and diagnosing usability problems. Cogn. Technol. Work 16, 373–388 (2014). Scholar
  8. 8.
    Iwarsson, S., Stahl, A.: Accessibility, usability and universal design – positioning and definition of concepts describing person-environment relationships. Disabil. Rehabil. 25(2), 57–66 (2003)Google Scholar
  9. 9.
    Kenward, M.G., Roger, J.H.: An improved approximation to the precision of fixed effects from restricted maximum likelihood. Comput. Stat. Data Anal. 53, 2583–2595 (2009)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Lenth, R.V.: Least-squares means: the R package lsmeans. J. Stat. Softw. 69(1), 1–33 (2016). Scholar
  11. 11.
    Nolen, S.B.: Young children’s motivation to read and write: development in social contexts. Cogn. Instr. 25(2–3), 219–270 (2007). Scholar
  12. 12.
    Olivier, J., May, W.L., Bell, M.L.: Relative effect sizes for measures of risk. Commun. Stat. Theory Methods, 1–8 (2017). Scholar
  13. 13.
    Pasini, M., Brondino, M., Burro, R., Raccanello, D., Gallo, S. (2016). The use of different multiple devices for an ecological assessment in psychological research: an experience with a daily affect assessment. In: Advances in Intelligent and Soft Computing, vol. 478, pp. 121–129. Scholar
  14. 14.
    Raccanello, D.: Students’ expectations about interviewees’ and interviewers’ achievement emotions in job selection interviews. J. Employ. Couns. 52(2), 50–64 (2015). Scholar
  15. 15.
    Raccanello, D., Brondino, M., Pasini, M.: Achievement emotions in technology enhanced learning: development and validation of self-report instruments in the Italian context. Interact. Des. Archit. J. IxD&A 23, 68–81 (2014)Google Scholar
  16. 16.
    Raccanello, D., Brondino, M., Pasini, M.: On-line assessment of pride and shame: relationships with cognitive dimensions in university students. Adv. Intell. Soft Comput. 374, 17–24 (2015). Scholar
  17. 17.
    Raccanello, D., Brondino, M., Pasini, M.: Two neglected moral emotions in university settings: Some preliminary data on pride and shame. J. Beliefs Values Stud. Relig. Educ. 36(2), 231–238 (2015). Scholar
  18. 18.
    Raccanello, D., Burro, R., Brondino, M., Pasini, M.: Relevance of terrorism for Italian students not directly exposed to it: the affective impact of the 2015 Paris and the 2016 Brussels attacks. Stress and Health 34, 338–343 (2017). Scholar
  19. 19.
    Raccanello, D., Burro, R., Brondino, M., Pasini, M.: Use of internet and wellbeing: a mixed-device survey. In: Advances in Intelligent and Soft Computing, vol. 617, pp. 65–73 (2017). Scholar
  20. 20.
    Raccanello, D., Burro, R., Hall, R.: Children’s emotional experience two years after an earthquake: an exploration of knowledge of earthquakes and associated emotions. PLoS ONE 12(2), 1–21 (2017). Scholar
  21. 21.
    Schmitz, C.: LimeSurvey: an open source survey tool. LimeSurvey Project Hamburg, Germany (2015).
  22. 22.
    Shackel, B.: Usability – Context, framework, definition, design and evaluation. Interact. Comput. 21, 339–346 (2009). Scholar
  23. 23.
    Singmann, H., Bolker, B., Westfall, J., Aust, F.: Afex: analysis of factorial experiments (2016).
  24. 24.
    Zhang, Z.: Overview of usability evaluation method (2011).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Daniela Raccanello
    • 1
    Email author
  • Margherita Brondino
    • 1
  • Margherita Pasini
    • 1
  • Maria Gabriella Landuzzi
    • 1
  • Diego Scarpanti
    • 1
  • Giada Vicentini
    • 1
  • Mara Massaro
    • 1
  • Roberto Burro
    • 1
  1. 1.Department of Human SciencesUniversity of VeronaVeronaItaly

Personalised recommendations