CogSketch: Sketch Understanding for Cognitive Science Research and for Education

Abstract

Sketching is a powerful means of working out and communicating ideas. Sketch understanding involves a combination of visual, spatial, and conceptual knowledge and reasoning, which makes it both challenging and potentially illuminating. This talk will describe how a team of AI researchers, cognitive psychologists, learning scientists, and educators is attempting to build the intellectual and software infrastructure needed to achieve more human-like sketch understanding software. We are creating CogSketch, an open-domain sketch understanding system that will serve as both a cognitive science research instrument and as a platform for sketch-based educational software. These missions interact: Our cognitive simulation work leads to improvements which can be exploited in creating educational software, and our prototype efforts to create educational software expose where we need further basic research. CogSketch incorporates a model of visual processing and qualitative spatial representations, facilities for analogical reasoning and learning, and a large common-sense knowledge base. Our vision is that sketch-based intelligent educational software will ultimately be as widely available to students as graphing calculators are today.