Abstract
The increasing number of interactions with automated systems has sparked the interest of researchers in trust in automation because it predicts not only whether but also how an operator interacts with an automation. In this work, a theoretical model of trust in automation is established and the development and evaluation of a corresponding questionnaire (Trust in Automation, TiA) are described.
Building on the model of organizational trust by Mayer et al. (1995) and the theoretical account by Lee and See (2004), a model for trust in automation containing six underlying dimensions was established. Following a deductive approach, an initial set of 57 items was generated. In a first online study, these items were analyzed and based on the criteria item difficulty, standard deviation, item-total correlation, internal consistency, overlap with other items in content, and response quote, 40 items were eliminated and two scales were merged, leaving six scales (Reliability/Competence, Understandability/Predictability, Propensity to Trust, Intention of Developers, Familiarity, and Trust in Automation) containing a total of 19 items.
The internal structure of the resulting questionnaire was analyzed in a subsequent second online study by means of an exploratory factor analysis. The results show sufficient preliminary evidence for the proposed factor structure and demonstrate that further pursuit of the model is reasonable but certain revisions may be necessary. The calculated omega coefficients indicated good to excellent reliability for all scales. The results also provide evidence for the questionnaire’s criterion validity: Consistent with the expectations, an unreliable automated driving system received lower trust ratings as a reliably functioning system. In a subsequent empirical driving simulator study, trust ratings could predict reliance on an automated driving system and monitoring in form of gaze behavior. Possible steps for revisions are discussed and recommendations for the application of the questionnaire are given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In this literature review, the following work was considered: Barber (1983), Blomqvist (1997), Butler and Cantrell (1984), Butler (1991), Deutsch (1958), Deutsch (1960), Dzindolet et al. (2001), Hoff and Bashir (2015), Hoffman et al. (2013), Jian et al. (2000), Lee and Moray (1992), Lee and See (2004), Madhavan and Wiegmann (2007), Madsen and Gregor (2000), Mayer et al. (1995), McKnight and Chervany (1996), McKnight and Chervany (2001), Muir (1987), Muir (1994), Muir and Moray (1996), Rempel et al. (1985), Rotter (1971).
References
Ajzen I, Fishbein M (1980) Understanding attitudes and predicting social behavior. Prentice Hall, Englewood Cliffs
Annett J (2002) Subjective rating scales: science or art? Ergonomics 45:966–987. https://doi.org/10.1080/00140130210166951
Bagozzi RP (1982) The role of measurement in theory construction and hypothesis testing: toward a holistic model. In: Fornell C (ed) A second generation of multivariate analysis. Praeger, New York, pp 5–23
Barber B (1983) The logic and limits of trust. Rutgers University Press, New Brunswick
Beggiato M, Pereira M, Petzoldt T, Krems JF (2015) Learning and development of trust, acceptance and the mental model of ACC. A longitudinal on-road study. Transp Res Part F Traffic Psychol Behav 35:75–84. https://doi.org/10.1016/j.trf.2015.10.005
Bisantz AM, Seong Y (2001) Assessment of operator trust in and utilization of automated decision-aids under different framing conditions. Int J Ind Ergon 28:85–97. https://doi.org/10.1016/S0169-8141(01)00015-4
Blomqvist K (1997) The many faces of trust. Scand J Manag 13:271–286. https://doi.org/10.1016/S0956-5221(97)84644-1
Brown RD, Galster SM (2004) Effects of reliable and unreliable automation on subjective measures of mental workload, situation awareness, trust and confidence in a dynamic flight task. In: Proceedings of the human factors and ergonomics society annual meeting 2004, pp 147–151
Burisch M (1978) Construction strategies for multiscale personality inventories. Appl Psychol Measur 2:97–111. https://doi.org/10.1177/014662167800200110
Burisch M (1984) Approaches to personality inventory construction: a comparison of merits. Am Psychol 39:214–227. https://doi.org/10.1037/0003-066X.39.3.214
Butler JK, Cantrell RS (1984) A behavioral decision theory approach to modeling dyadic trust in superiors and subordinates. Psychol Rep 55:19–28. https://doi.org/10.2466/pr0.1984.55.1.19
Butler JK (1991) Toward understanding and measuring conditions of trust: evolution of a conditions of trust inventory. J Manag 17:643–663. https://doi.org/10.1177/014920639101700307
Damasio AR (1996) The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos Trans R Soc Lond B Biol Sci 351:1413–1420. https://doi.org/10.1098/rstb.1996.0125
Deutsch M (1958) Trust and suspicion. J Conflict Resolut 2:265–279. https://doi.org/10.1177/002200275800200401
Deutsch M (1960) The effect of motivational orientation upon trust and suspicion. Hum Relations 13:123–139. https://doi.org/10.1177/001872676001300202
Diamantopoulos A, Sarstedt M, Fuchs C, Wilczynski P, Kaiser S (2012) Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective. J Acad Mark Sci 40:434–449. https://doi.org/10.1007/s11747-011-0300-3
Drnec K, Marathe AR, Lukos JR, Metcalfe JS (2016) From trust in automation to decision neuroscience: applying cognitive neuroscience methods to understand and improve interaction decisions involved in human automation interaction. Front Hum Neurosci 10:54. https://doi.org/10.3389/fnhum.2016.00290
Eisinga R, Grotenhuis MT, Pelzer B (2013) The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? Int J Public Health 58:637–642. https://doi.org/10.1007/s00038-012-0416-3
Emons WHM, Sijtsma K, Meijer RR (2007) On the consistency of individual classification using short scales. Psychol Methods 12:105–120. https://doi.org/10.1037/1082-989X.12.1.105
Feuerberg BV, Bahner JE, Manzey D (2005) Interindividuelle unterschiede im umgang mit automation – entwicklung eines fragebogens zur erfassung des complacency-potentials. In: Urbas L, Steffens C (eds) Zustandserkennung und systemgestaltung. 6. Berliner werkstatt mensch-maschine-systeme., Als Ms. gedr. VDI-Verlag, Düsseldorf, pp 199–202
Flake JK, Pek J, Hehman E (2017) Construct validation in social and personality research. Soc Psychol Pers Sci 8:370–378. https://doi.org/10.1177/1948550617693063
Fuchs C, Diamantopoulos A (2009) Using single-item measures for construct measurement in management research: conceptual issues and application guidelines. Die Betriebswirtschaft 69:195
Gigerenzer G, Selten R (eds) (2002) Bounded rationality: the adaptive toolbox. MIT Press, Cambridge
Heene M, Hilbert S, Draxler C, Ziegler M, Bühner M (2011) Masking misfit in confirmatory factor analysis by increasing unique variances: a cautionary note on the usefulness of cutoff values of fit indices. Psychol Methods 16:319–336. https://doi.org/10.1037/a0024917
Hergeth S, Lorenz L, Vilimek R, Krems JF (2016) Keep your scanners peeled: gaze behavior as a measure of automation trust during highly automated driving. Hum Factors 58:509–519. https://doi.org/10.1177/0018720815625744
Hergeth S, Lorenz L, Krems JF (2017) Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Hum Factors 59:457–470. https://doi.org/10.1177/0018720816678714
Hoeppner BB, Kelly JF, Urbanoski KA, Slaymaker V (2011) Comparative utility of a single-item versus multiple-item measure of self-efficacy in predicting relapse among young adults. J Subst Abuse Treat 41:305–312. https://doi.org/10.1016/j.jsat.2011.04.005
Hoff KA, Bashir M (2015) Trust in automation: integrating empirical evidence on factors that influence trust. Hum Factors 57:407–434. https://doi.org/10.1177/0018720814547570
Hoffman RR, Johnson M, Bradshaw JM, Underbrink A (2013) Trust in automation. IEEE Intell Syst 28:84–88. https://doi.org/10.1109/MIS.2013.24
Horn JL (1965) A rationale and test for the number of factors in factor analysis. Psychometrika 30:179–185. https://doi.org/10.1007/BF02289447
Hu L-T, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Eq Model Multidiscip J 6:1–55. https://doi.org/10.1080/10705519909540118
Hubley AM, Zumbo BD (2013) Psychometric characteristics of assessment procedures: An overview. In: Geisinger KF (ed) Test theory and testing and assessment in industrial and organizational psychology, 1st edn. American Psychological Association, Washington, pp 3–20
Jian J-Y, Bisantz AM, Drury CG (2000) Foundations for an empirically determined scale of trust in automated systems. Int J Cogn Ergon 4:53–71. https://doi.org/10.1207/S15327566IJCE0401_04
Kenny DA, Kaniskan B, McCoach DB (2014) The performance of RMSEA in models with small degrees of freedom. Soc Methods Res 44:486–507. https://doi.org/10.1177/0049124114543236
Kirlik A (1993) Modeling strategic behavior in human-automation interaction: why an “aid” can (and should) go unused. Hum Factors 35:221–242. https://doi.org/10.1177/001872089303500203
Körber M (2018) Individual differences in human-automation interaction: a driver-centered perspective on the introduction of automated vehicles. Dissertation, Technical University of Munich
Körber M, Baseler E, Bengler K (2018) Introduction matters: manipulating trust in automation and reliance in automated driving. Appl Ergon 66:18–31. https://doi.org/10.1016/j.apergo.2017.07.006
Krantz DH, Luce RD, Suppes P, Tversky A (2007) Additive and polynomial representations. Foundations of measurement, vol 1. Dover Publisher, Mineola
Lee JD, Moray N (1992) Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35:1243–1270. https://doi.org/10.1080/00140139208967392
Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46:50–80. https://doi.org/10.1518/hfes.46.1.50_30392
Love J, Selker R, Verhagen J, Marsman M, Gronau QF, Jamil T, Šmíra M, Epskamp S, Wild A, Ly A, Matzke D, Wagenmakers E-J, Morey RD, Rouder JN (2015) Software to sharpen your stats. APS Obs 28:27–29
MacCallum RC, Browne MW, Sugawara HM (1996) Power analysis and determination of sample size for covariance structure modeling. Psychol Methods 1:130–149. https://doi.org/10.1037/1082-989X.1.2.130
MacCallum RC, Widaman KF, Zhang S, Hong S (1999) Sample size in factor analysis. Psychol Methods 4:84–99. https://doi.org/10.1037/1082-989X.4.1.84
Madhavan P, Wiegmann DA (2007) Similarities and differences between human–human and human–automation trust: an integrative review. Theor Issues Ergon Sci 8:277–301. https://doi.org/10.1080/14639220500337708
Madsen M, Gregor S (2000) Measuring human-computer trust. In: Proceedings of the 11th Australasian conference on information systems, pp 6–8
Mayer RC, Davis JH, Schoorman FD (1995) An integrative model of organizational trust. Acad Manag Rev 20:709–734. https://doi.org/10.5465/AMR.1995.9508080335
Mayer RC, Davis JH (1999) The effect of the performance appraisal system on trust for management: a field quasi-experiment. J Appl Psychol 84:123–136. https://doi.org/10.1037//0021-9010.84.1.123
McCoach DB (2003) SEM isn’t just the Schoolwide enrichment model anymore: structural equation modeling (SEM) in gifted education. J Educ Gifted 27:36–61. https://doi.org/10.1177/016235320302700104
McCoach DB, Gable RK, Madura JP (2013) Instrument development in the affective domain: School and corporate applications, 3rd edn. Springer, New York
McKnight DH, Chervany NL (2001) Trust and distrust definitions: one bite at a time. In: Falcone R, Singh M, Tan Y-H (eds) Trust in cyber-societies: integrating the human and artificial perspectives. Springer, Heidelberg, pp 27–54
McNeish D (2017) Thanks coefficient alpha, we’ll take it from here. Psychol Methods. https://doi.org/10.1037/met0000144
Merritt SM, Ilgen DR (2008) Not all trust is created equal: dispositional and history-based trust in human-automation interactions. Hum Factors 50:194–210. https://doi.org/10.1518/001872008X288574
Meyer J (2004) Conceptual issues in the study of dynamic hazard warnings. Hum Factors 46:196–204
Michell J (1997) Quantitative science and the definition of measurement in psychology. Br J Psychol 88:355–383. https://doi.org/10.1111/j.2044-8295.1997.tb02641.x
Moosbrugger H, Kelava A (eds) (2012) Testtheorie und fragebogenkonstruktion, 2nd edn. Springer, Heidelberg
Muir BM (1987) Trust between humans and machines, and the design of decision aids. Int J Man Mach Stud 27:527–539. https://doi.org/10.1016/S0020-7373(87)80013-5
Muir BM (1994) Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37:1905–1922. https://doi.org/10.1080/00140139408964957
Muir BM, Moray N (1996) Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39:429–460. https://doi.org/10.1080/00140139608964474
Nagy MS (2002) Using a single-item approach to measure facet job satisfaction. J Occup Organ Psychol 75:77–86. https://doi.org/10.1348/096317902167658
Nunnally JC, Bernstein IH (1994) Psychometric theory, 3rd edn. McGraw-Hill, New York
Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern 30:286–297. https://doi.org/10.1109/3468.844354
Paulhus DL, Vazire S (2007) The self-report method. In: Robins RW, Fraley RC, Krueger RF (eds) Handbook of research methods in personality psychology. The Guilford Press, New York, pp 224–239
Preacher KJ, Zhang G, Kim C, Mels G (2013) Choosing the optimal number of factors in exploratory factor analysis: a model selection perspective. Multivar Behav Res 48:28–56. https://doi.org/10.1080/00273171.2012.710386
Rempel JK, Holmes JG, Zanna MP (1985) Trust in close relationships. J Pers Soc Psychol 49:95–112. https://doi.org/10.1037/0022-3514.49.1.95
Rhemtulla M, Brosseau-Liard PÉ, Savalei V (2012) When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychol Methods 17:354–373. https://doi.org/10.1037/a0029315
Robins RW, Hendin HM, Trzesniewski KH (2001) Measuring global self-esteem: construct validation of a single-item measure and the Rosenberg self-esteem scale. Pers Soc Psychol Bull 27:151–161. https://doi.org/10.1177/0146167201272002
Rotter JB (1971) Generalized expectancies for interpersonal trust. Am Psychol 26:443–452. https://doi.org/10.1037/h0031464
Sakaluk JK, Short SD (2017) A methodological review of exploratory factor analysis in sexuality research: used practices, best practices, and data analysis resources. J Sex Res 54:1–9. https://doi.org/10.1080/00224499.2015.1137538
Scarpello V, Campbell JP (1983) Job satisfaction: are all the parts there? Pers Psychol 36:577–600. https://doi.org/10.1111/j.1744-6570.1983.tb02236.x
Sloan JA, Aaronson N, Cappelleri JC, Fairclough DL, Varricchio C (2002) Assessing the clinical significance of single items relative to summated scores. Mayo Clin Proc 77:479–487. https://doi.org/10.4065/77.5.479
Tal E (2017) Measurement in science. In: Zalta EN (ed) The Stanford encyclopedia of philosophy. Metaphysics Research Lab, Stanford University
Uebersax JS (2006) Likert scales: dispelling the confusion. http://www.john-uebersax.com/stat/likert.htm. Accessed 8 Feb 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Körber, M. (2019). Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds) Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018). IEA 2018. Advances in Intelligent Systems and Computing, vol 823. Springer, Cham. https://doi.org/10.1007/978-3-319-96074-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-96074-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96073-9
Online ISBN: 978-3-319-96074-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)