Abstract
Because ITSs offer knowledge-based instruction, ITS authoring shells can use knowledge acquisition systems to bring courseware development within the reach of subject-matter experts unskilled in instructional design and development. The Experimental Advanced Instructional Design Advisor (XAIDA), a simple ITS shell, was evaluated in a study of 17 Air Force medical-technician trainers. The study examined XAIDA’s learnability, usability, and acceptability to the participants using a one-week combined training and evaluation program. All participants were able to produce courseware using XAIDA. Measures of cognitive structure indicated that their conception of XAIDA’s features became more similar to that of an expert as the result of training. Performance tests, skill ratings and usability ratings indicated that XAIDA is both usable and learnable. Participant’s attitudes were generally positive except at a point when training required a difficult cognitive shift. Attitudes remained focused througout training on personal and low-level concerns, indicating the participants were not likely to be early adapters of the technology. The results have implications for the deployment of ITS shells in operational environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bailey, D. B., & Palsha, S. A. (1992). Qualities of stages of concern questionnaire and implications for educational innovations. The Journal of Educational Research, 85, 226–232.
Bloom, C., Linton, F., & Bell, B. (1997). Using evaluation in the design of an intelligent tutoring system. Journal of Interactive Learning Research, 8, 235–276.
Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Converse, S. A. (1991). Toward an integration of training theory and technique. Human Factors, 33, 281–292.
Carroll, J. M. (1984). Minimalist Training. Datamation, November, 125–136.
Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal of Human-Computer Studies, 41, 801–849.
Geoghegan, W. H. (1994). Whatever happened to instructional technology? Paper presented at the 22nd Annual Conference of the International Business Schools Computing Association. Baltimore, Maryland.
Glaser, R. (1983). Expertise and learning: How do we think about instructional processes now that we have discovered knowledge structures? In E. Klaher and K. Kotovsky (Eds.) Complex information processing: The impact of Herb Simon. Lawrence Erlbaum: Hillsdale, NJ.
Hall, G. E. (1979). The concerns-based approach to facilitating change. Educational Horizons, 4, 202–208.
Hall, G. E., George, A. A., & Rutherford, W. L. (1986). Measuring stages of concern about the innovation: A manual for the user of SoC Questionnaire. Austin, TX: Southwest Educational Development Laboratory.
Hickey, A. E., Spector, J. M., & Muraida, D. J. (1991). Design specifications for the advanced instructional design advisor (AIDA) (Vols 1 & 2) (AL-TR-1991-0085). Brooks AFB, TX: Technical Report for the Air Force Armstrong Laboratory, Human Resources Directorate.
Hsieh, P. Y, Halff, H. M., & Redfield, C. L. (1996). Four easy pieces: Development systems for knowledge-based generative instruction. (Manuscript submitted for publication).
Kraiger, J., Ford, K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311–328.
Kraiger, J., Salas, E. & Cannon-Bowers, J. (1995). Measuring knowledge organization as a method of assessing learning during training. Human Factors, 37, 804–816.
Merrill, M. D. (1993). An integrated model for automating instructional design and delivery. In J. M. Spector, M. C. Polson, and D. J. Muraida (Eds.) Automating Instructional Design: Concepts and Issues (pp. 147–190). Englewood Cliffs, NJ: Educational Technology Publications.
Palmiter, S., & Elkerton, J. (1993). Animated Demonstrations for learning procedural computer-based tasks. Human-Computer Interaction, 8, 193–216.
Redfield, C. (November, 1997) AAAI Fall Symposium on ITS Authoring Tools (Technical Report FS-97-01). Cambridge, MA, AAAI.
Rowe, A., Cooke, N., Hall, & Halgren, (1996). Toward an on-line knowledge assessment methodology: Building on the relationship between knowing and doing. Journal of Experimental Psychology: Applied, 2, 31–47.
Royer, J. M., Cisero, C. A., & Carlo, M. S. (1993). Techniques and procedures for assessing cognitive skills. Review of Educational Research, 63, 201–243.
Schvaneveldt, R. W. (Ed.). (1990). Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex.
Spector, J. M. (1990). Designing and developing an advanced instructional design advisor. (AFHRL-TP-90-52). Brooks AFB: TX: Technical Paper for the Training Systems Division of the Air Force Human Resources Laboratory.
Spector, J. M., Polson, M. C., & Muraida, D. J., Eds. (1993). Automating Instructional Design: Concepts and Issues. Englewood Cliffs, NJ: Educational Technology Publications.
Wenzel, B. M., Dirnberger, M. T., & Ellis, R. (1997). Evaluation of the effectiveness of distributed interactive multimedia courseware for emergency medical technician training. Manuscript submitted for publication.
Wenzel, B. M., Keeney, C., Licklider, J., Roberts, J. D., & Reyna, H. (1997). Evaluation of the courseware developed and delivered using the Experimental Advanced Instructional Design Advisor. Presentation at the annual meeting of the Texas Association of College Technical Editors. Austin, Texas.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wenzel, B.M., Dirnberger, M.T., Hsieh, P.Y., Chudanov, T.J., Halff, H.M. (1998). Evaluating Subject Matter Experts’ Learning and Use of an ITS Authoring Tool. In: Goettl, B.P., Halff, H.M., Redfield, C.L., Shute, V.J. (eds) Intelligent Tutoring Systems. ITS 1998. Lecture Notes in Computer Science, vol 1452. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-68716-5_21
Download citation
DOI: https://doi.org/10.1007/3-540-68716-5_21
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64770-6
Online ISBN: 978-3-540-68716-0
eBook Packages: Springer Book Archive