Abstract
Goals and plans organize much of complex problem solving behavior and are often inferable from action sequences. This paper addresses the strengths and limitations of inferring goals and plans from information that can be derived from computer traces of software used to solve mathematics problems. We examined mathematics problem solving activity about distance, rate, time relationships in a computer software environment designed to support understanding of functional relationships among these variables (e.g., distance =rate × time; time=distance/rate) using graphical representations of the results of simulations. Ten adolescent-aged students used the software to solve two distance, rate, time problems, and provided think-aloud protocols. To determine the inferability of understanding from the action traces, coders analyzed students' understanding from the computer traces alone (Trace-only raters) and compared these to analyses based on the traces plus the verbal protocols (Traceplus raters). Inferability of understanding from the action traces was related to level of student understanding how they used the graphing tool. When students had a good understanding of distance, rate, time relationships, it could be accurately inferred from the computer traces if they used the simulation tool in conjunction with the graphing tool. When students had a weak understanding, the verbal protocols were necessary to make accurate inferences about what was and was not understood. The computer trace also failed to capture dynamic exploration of the visual environment so students who relied on the graphing tool were inaccurately characterized by the Trace-only coders. Discussion concerns types of scaffolds that would be helpful learning environment for complex problems, the kind of information that is needed to adequately track student understanding in this content domain, and instructional models for integrating learning environments like these into classrooms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, J.R. (1990). Analysis of student performance with the LISP tutor. In N. Frederiksen, R. Glaser, A. Lesgold and M.G. Shafto, eds,Diagnostic Monitoring of Skill and Knowledge Acquisition, pp. 27–50. Hillsdale, NJ: Lawrence Erlbaum Associates.
Anderson, J.R., Corbett, A.T., Fincham, J.M., Hoffman, D. and Pelletier, R. (1992). General principles for an intelligent tutoring architecture. In J.W. Region and V.J. Shute, eds,Cognitive Approaches to Automated Instruction, pp. 81–106. Hillsdale, NJ: Lawrence Erlbaum Associates.
Barron, B., Vye, N.J., Zech, L., Schwartz, D., Bransford, J.D., Goldman, S.R., Pellegrino, J., Morris, J., Garrison, S. and Kantor, R. (1995). Creating contexts for community-based problem solving: The Jasper Challenge Series. In C.N. Hedley, P. Antonacci and M. Rabinowitz, eds,Thinking and Literacy: The Mind at Work, pp. 47–71. Hillsdale, NJ: Lawrence Erlbaum Associates.
Bennett, D., Hartman, J. and Rasmussen, S. (1991).The Geometer's Sketchpad. Berkeley, CA: Key Curriculum Press.
Bereiter, C. and Scardamalia, M. (1989). Intentional learning as a goal of instruction. In L.B. Resnick, ed.,Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser, pp. 361–392. Hillsdale, NJ: Erlbaum.
Carver, S.M., Lehrer, R., Connell, T. and Erickson, J. (1992). Learning by hypermedia design: Issues of assessment and implementation.Educational Psychologist 27: 385–404.
Chi, M.T.H., Glaser, R. and Farr, M. (1991).The Nature of Expertise. Hillsdale, NJ: Lawrence Erlbaum Associates.
Cognition and Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to situated cognition.Educational Researcher 19(6): 2–10.
Cognition and Technology Group at Vanderbilt (1992). The Jasper experiment: An exploration of issues in learning and instructional design.Educational Technology Research and Development 40: 65–80.
Cognition and Technology Group at Vanderbilt (1993). The Jasper series: Theoretical foundations and data on problem solving and transfer. In L.A. Penner, G.M. Batsche, H.M. Knoff and D.L. Nelson, eds,The Challenges in Mathematics and Science Education: Psychology's Response, pp. 113–152. Washington, DC: American Psychological Association.
Cognition and Technology Group at Vanderbilt (1994). From visual word problems to learning communities: Changing conceptions of cognitive research. In K. McGilly, ed.,Classroom Lessons: Integrating Cognitive Theory and Classroom Practice, pp. 157–200. Cambridge, MA: MIT Press.
Cognition and Technology Group at Vanderbilt (1996). Looking at technology in context: A framework for understanding technology and education. In D.C. Berliner and R.C. Calfee, eds,The Handbook of Educational Psychology, pp. 807–840. New York: Macmillan.
Cognition and Technology Group at Vanderbilt (1997).The Jasper Project: Lessons in Curriculum, Instruction, Assessment, and Professional Development. Mahwah, NJ: Erlbaum.
Crews, T.R. (1995). AdventurePlayer: A microworld embedded in a macrocontext. Unpublished PhD dissertation. Vanderbilt University, Nashville, TN.
Crews, T.R. and Biswas, G. (1993). Towards an optimal planner for tutoring systems.Proceedings of the 5th Midwest AI and Cognitive Science Conference, pp. 46–52. Chesterton, IN.
Crews, T.R., Biswas, G., Goldman, S.R. and Bransford, J.D. (in press). Anchored interactive learning environments.Journal of AI in Education.
Goldman, S.R. and Durán, R.P. (1988). Answering questions from oceanography texts: Learner, task and text characteristics.Discourse Processes 11: 373–412.
Goldman, S.R., Vye, N., Barron, L. and Pellegrino, J.W. (1991, April). A problem-space analysis of the Jasper problems and students' attempts to solve them. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
Ericcson, K.A. and Simon, H.A. (1984/1993).Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press.
Lajoie, S.P. and Derry, S.J. (1993). A middle camp for (un)intelligent instructional computing: An introduction. In S.P. Lajoie and S.J. Derry, eds,Computers as Cognitive Tools, pp. 1–14. Hillsdale, NJ: Lawrence Erlbaum Associates.
Means, B., Blando, J., Olson, K., Middleton, T., Morocco, C.C., Remz, A.R. and Zorfass, J. (1993).Using Technology to Support Education Reform. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.
National Council of Teachers of Mathematics (1989).Curriculum and Evaluation Standards for School Mathematics. Reston, VA: Author.
Owens, S., Biswas, G., Nathan, M., Zech, L., Bransford, J.D. and Goldman, S.R. (1995, August). SmartTools: A multi-representational approach to teaching functional relations. Paper presented during the proceedings of the AI in ED 95 conference, Washington, DC.
Schoenfeld, A.H. (1987). What's all the fuss about metacognition? In A.H. Schoenfeld, ed.,Cognitive Science and Mathematics Education, pp. 189–215. Hillsdale, NJ: Erlbaum.
Van Lehn, K. and Brown, J.S. (1980). Planning nets: A representation for formalizing analogies and semantic models of procedural skills. In R.E. Snow, P. Federico and W.M. Montague, eds,Aptitude, Learning and Instruction, Vol. 2, pp. 95–137. Hillsdsale, NJ: Erlbaum.
Vye, N.J., Goldman, S.R., Voss, J.F., Hmelo, C., Williams, S. and the Cognition and Technology Group at Vanderbilt (CTGV) (in press). Complex mathematical problem solving by individuals and dyads.Cognition and Instruction.
White, B.Y. (1993). ThinkerTools; Causal models, conceptual change, and science education.Cognition and Instruction 10: 1–100.
Author information
Authors and Affiliations
Consortia
Additional information
Members of the Cognition and Technology Group at Vanderbilt who have contributed to this project are (in alphabetical order) Helen Bateman, John Bransford, Thaddeus Crews, Allison Moore, Mitchell Nathan, and Stephen Owens. The research was supported, in part, by grants from the National Science Foundation (NSF-MDR-9252990) but no official endorsement of the ideas expressed herein should be inferred.
Rights and permissions
About this article
Cite this article
Goldman, S.R., Zech, L.K., Biswas, G. et al. Computer technology and complex problem solving: Issues in the study of complex cognitive activity. Instr Sci 27, 235–268 (1999). https://doi.org/10.1007/BF00897321
Issue Date:
DOI: https://doi.org/10.1007/BF00897321