Methodology for Knowledge Elicitation in Visual Abductive Reasoning Tasks
The potential for bias to affect the results of knowledge elicitation studies is well recognized. Researchers and knowledge engineers attempt to control for bias through careful selection of elicitation and analysis methods. Recently, the development of a wide range of physiological sensors, coupled with fast, portable and inexpensive computing platforms, has added an additional dimension of objective measurement that can reduce bias effects. In the case of an abductive reasoning task, bias can be introduced through design of the stimuli, cues from researchers, or omissions by the experts. We describe a knowledge elicitation methodology robust to various sources of bias, incorporating objective and cross-referenced measurements. The methodology was applied in a study of engineers who use multivariate time series data to diagnose the performance of devices throughout the production lifecycle. For visual reasoning tasks, eye tracking is particularly effective at controlling for biases of omission by providing a record of the subject’s attention allocation.
KeywordsKnowledge elicitation Eye tracking Abductive reasoning
We wish to acknowledge James D. Morrow of Sandia National Laboratories, Albuquerque New Mexico for creating the software used in our study to display the time series stimuli and record subject response times. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.
- 1.Booker, J.M., Meyer, M.A.: Eliciting and Analyzing Expert Judgment: A Practical Guide. ASA, pp. 3–16. SIAM, Philadelphia (2001)Google Scholar
- 2.Thagard, P., Shelley, C.: Abductive reasoning: logic, visual thinking, and coherence. In: DallaChiara, M.L., et al. (eds.) Logic and Scientific Methods: Volume One of the Tenth International Congress of Logic, Methodology and Philosophy of Science, Florence, August 1995, vol. 259, pp. 413–427. Springer, Netherlands (1997)CrossRefGoogle Scholar
- 7.Marshall, S.P.: Identifying cognitive state from eye metrics. Aviat. Space Environ. Med. 78(5), B165–B175 (2007)Google Scholar
- 8.Matzen, L.E., Haass, M.J., McNamara, L.A.: Using eye tracking to asseess cognitve biasses: a position paper. In: DECISIVe Workshop, IEEE Vis 2014 (2014)Google Scholar