Why Connectionist Learning Algorithms Need to be More Creative

Abstract

The topic of creativity is an important one in connectionism. In general, connectionist systems are only as powerful as the learning algorithms they employ and these often need to ‘creatively’ construct internal representations. Of course, some researchers find the notion of connectionist representation hard to deal with. They feel that for something to count as a representation there must be an agent who makes explicit use of some system of symbols for the purposes of representing phenomena in a given domain. They see connectionist mechanisms (i.e. neural networks) as conglomerations of activity-storing units and activity-passing connections. They understand how this sort of mechanism might perform certain types of computation but they cannot see how it could possibly have anything legitimately termed a ‘representation’. Such researchers may therefore be upset by the frequency with which connectionists use the term ‘representation’ in relation to connectionist mechanisms.