Modelling of Novices’ Control Skills With Machine Learning

  • Rafael Morales
  • Helen Pain
Part of the CISM International Centre for Mechanical Sciences book series (CISM, volume 407)


We report an empirical study on the application of machine learning to the modelling of novice controllers’ skills in balancing a pole (inverted pendulum) on top of a cart. Results are presented on the predictive power of the models, and the extent to which they were tailored to each controller. The behaviour of the participants in the study and the behaviour of an interpreter executing their models are compared with respect to the amount of time they were able to keep the pole and cart under control, the degree of stability achieved, and the conditions of failure. We discuss the results of the study, the limitations of the methodology in relation to learner modelling, and we point out future directions of research.


User Action Machine Learning Technique Stability Index Inverted Pendulum Dissimilarity Measure 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bratko, I., Urbančič, T., and Sammut, C. (1997). Behavioural cloning of control skill. In Michalski, R. S., Bratko, I., and Kubat, M., eds., Machine Learning and Data Mining: Methods and Applications. John Wiley & Sons, chapter 14, 335–351.Google Scholar
  2. Bratko, I.(1995). Derivating qualitative control for dynamic systems. In Furukawa, K., Michie, D., and Muggleton, S., eds., Machine Intelligence, volume 14. Oxford: Clarendon Press. 367–386.Google Scholar
  3. Chiu, B. C., Webb, G. I., and Kuzmycz, M. (1997). A comparison of first-order and zeroth-order induction for Input-Output Agent Modelling. In Jameson, A., Paris, C., and Tasso, C., eds., User Modeling: Proceedings of the Sixth International Conference, UM97, 347–358. Chia Laguna, Sardinia, Italy: Springer Wien New York.Google Scholar
  4. Cohen, W. W. (1995). Fast effective rule induction. In Prieditis, A., and Russell, S., eds., Machine Learning: Proceedings of the Twelfth International Conference. Tahoe City, CA: Morgan Kaufmann.Google Scholar
  5. Cotterill, R. (1989). No Ghost in the Machine. London: Heinemann.Google Scholar
  6. Everitt, B. (1993). Cluster Analysis. London: Edward Arnold, 3 edition.Google Scholar
  7. Finton, D. J. (1994). Controller-less driver for the cart-pole problem. Available on the World Wide Web at Scholar
  8. Gilmore, D., and Self, J. (1988). The application of machine learning to intelligent tutoring systems. In Self, J., ed., Artificial Intelligence and Human Learning: Intelligent Computer-Aided Instruction. London: Chapman and Hall Computing, chapter 1, 179–196.Google Scholar
  9. Jonassen, D. H., and Grabowski, B. L. (1993). Handbook of Individual Differences, Learning, and Instruction. Lawrence Erlbaum Associates.Google Scholar
  10. Langley, P., Ohlsson, S., and Sage, S. (1984). A machine learning approach to student modeling. Technical Report CMU-RI-TR-84–7, The Robotics Institute, Carnegie-Mellon University, Pittsburgh, Pennsylvania, USA.Google Scholar
  11. Michie, D., and Camacho, R. (1994). Building symbolic representations of intuitive real-time skills from performance data. In Furukawa, K., Michie, D., and Muggleton, S., eds., Machine Intelligence, volume 13. Oxford: Clarendon Press. 385–418.Google Scholar
  12. Michie, D., Bain, M., and Hayes-Michie, J. (1990). Cognitive models from subcognitive skills. In McGhee, J., Grimble, M. J., and Mowforth, P., eds., Knowledge-Based Systems for Industrial Control. London: Peter Peregrinus. chapter 5, 71–99.CrossRefGoogle Scholar
  13. Morales, R., Ramscar, M., and Pain, H. (1998). Cognitive effects of participative learner modelling. In Ayala, G., ed., Proceedings of the Current Trends and Applications of Artificial Intelligence in Education Workshop, 49–56. Mexico City, Mexico: ITESM.Google Scholar
  14. Quinlan, R. J. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.Google Scholar
  15. Sison, R., and Shimura, M. (1988). Student modeling and machine learning. International Journal of Artificial Intelligence in Education 9:128–158.Google Scholar
  16. Sleeman, D. H. (1982). Inferring (mal) rules from pupil’s protocols. In ECAI-82:1982 European Conference on Artificial Intelligence, 160–164.Google Scholar
  17. Urbančič, T., and Bratko, I. (1994). Reconstructing human skill with machine learning. In Conn, A. G., ed., ECAI 94:11th European Conference on Artificial Intelligence, 498–502. Amsterdam, The Netherlands: John Wiley & Sons.Google Scholar
  18. Webb, G. I., and Kuzmycz, M. (1996). Feature based modelling: A methodology for producing coherent, consistent, dynamically changing models of agents’ competencies. User Modeling and User-Adapted Interaction 5:117–150.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Rafael Morales
    • 1
  • Helen Pain
    • 1
  1. 1.School of Artificial IntelligenceUniversity of EdinburghUK

Personalised recommendations