Progress Appraisal as a Challenging Element of Coordination in Human and Machine Joint Activity

  • Paul J. Feltovich
  • Jeffrey M. Bradshaw
  • William J. Clancey
  • Matthew Johnson
  • Larry Bunch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4995)


Joint activity, as we define it, is a mutually interdependent social endeavor that requires sufficient predictability among participating parties to enable coordination. Coordination, in turn, sometimes requires the parties to appraise the state of progress of their activities so that, if necessary, they can adjust their actions to meet coordination needs and communicate their status to others as appropriate. A significant impediment as yet precluding the full participation of automation in joint activity with people is its inability to sense and communicate aspects of its state that would allow other participants to meaningfully assess progress toward (or anticipate failure with respect to) mutual objectives. In the current article, we address various issues associated with “progress appraisal” and the challenges it poses for human-machine systems. We point to promising directions for future work.


Coordination culture human-agent-robotic systems joint activity ontology policy predictability regulation teamwork progress appraisal common ground 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allen, J.F., Ferguson, G.: Human-machine collaborative planning. In: Proceedings of the NASA Planning and Scheduling Workshop, Houston, TX (2002)Google Scholar
  2. 2.
    Arkin, R.C.: Homeostatic control for a mobile robot: Dynamic replanning in hazardous environments. Journal of Robotic Systems 9(2), 197–214 (1992)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Biever, C.: Bots as peers: It’s all a matter of teamwork. San Francisco Chronicle, San Francisco, CA, May 6 (2007) (accessed September 15, 2007),
  4. 4.
    Biever, C.: If you’re happy, the robot knows it. The New Scientist 193(2596), 30–31 (March 24, 2007) (accessed September 15 (2007),
  5. 5.
    Bradshaw, J.M., Sierhuis, M., Acquisti, A., Feltovich, P., Hoffman, R., Jeffers, R., Prescott, D., Suri, N., Uszok, A., Van Hoof, R.: Adjustable autonomy and human-agent teamwork in practice: An interim report on space applications. In: Hexmoor, H., Falcone, R., Castelfranchi, C. (eds.) Agent Autonomy, pp. 243–280. Kluwer Academic Publishers, Dordrecht (2003)Google Scholar
  6. 6.
    Bradshaw, J.M., Acquisti, A., Allen, J., Breedy, M.R., Bunch, L., Chambers, N., Feltovich, P., Galescu, L., Goodrich, M.A., Jeffers, R., Johnson, M., Jung, H., Lott, J., Olsen Jr., D.R., Sierhuis, M., Suri, N., Taysom, W., Tonti, G., Uszok, A.: Teamwork-centered autonomy for extended human-agent interaction in space applications. In: AAAI 2004 Spring Symposium, 22-24 March, 2004. Stanford University, CA (2004)Google Scholar
  7. 7.
    Bradshaw, J.M., Feltovich, P., Jung, H., Kulkarni, S., Taysom, W., Uszok, A.: Dimensions of adjustable autonomy and mixed-initiative interaction. In: Nickles, M., Rovatsos, M., Weiss, G. (eds.) AUTONOMY 2003. LNCS (LNAI), vol. 2969. Springer, Heidelberg (2004)Google Scholar
  8. 8.
    Bradshaw, J.M., Feltovich, P.J., Jung, H., Kulkarni, S., Allen, J., Bunch, L., Chambers, N., Galescu, L., Jeffers, R., Johnson, M., Sierhuis, M., Taysom, W., Uszok, A., Van Hoof, R.: Policy-based coordination in joint human-agent activity. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, The Hague, Netherlands, October 10-13 (2004)Google Scholar
  9. 9.
    Bradshaw, J.M., Jung, H., Kulkarni, S., Johnson, M., Feltovich, P., Allen, J., Bunch, L., Chambers, N., Galescu, L., Jeffers, R., Suri, N., Taysom, W., Uszok, A.: Toward trustworthy adjustable autonomy in KAoS. In: Falcone, R. (ed.) Trusting Agents for Trustworthy Electronic Societies. Springer, Berlin (2005)Google Scholar
  10. 10.
    Bradshaw, J.M., Feltovich, P., Johnson, M., Bunch, L., Breedy, M.R., Jung, H., Lott, J., Uszok, A.: Coordination in Human-Agent Teamwork. Invited Paper and Presentation. In: AAAI Fall Symposium, November 8-10 (2007)Google Scholar
  11. 11.
    Bruemmer, D.J., Marble, J.L., Dudenhoeffer, D.D.: Mutual initiative in human-machine teams. In: Proceedings of the 2002 IEEE 7th Conference on Human Factors and Power Plants, vol. 7, pp. 22–30 (2002)Google Scholar
  12. 12.
    Burke, J.J., Murphy, R.R., Coovert, M., Riddle, D.L.: Moonlight in Miami: A field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise. Human-Computer Interaction 19(1-2), 85–116 (2004)CrossRefGoogle Scholar
  13. 13.
    Christofferson, K., Woods, D.D.: How to make automated systems team players. In: Salas, E. (ed.) Advances in Human Performance and Cognitive Engineering Research, vol. 2. JAI Press, Elsevier (2002)Google Scholar
  14. 14.
    Clancey, W.J.: Situated Cognition: On Human Knowledge and Computer Representations. Cambridge University Press, Cambridge (1997)Google Scholar
  15. 15.
    Clancey, W.J.: Conceptual Coordination: How the Mind Orders Experience in Time. Lawrence Erlbaum, Hillsdale (1999)Google Scholar
  16. 16.
    Clancey, W.J.: Simulating activities: Relating motives, deliberation, and attentive coordination. Cognitive Systems Review (2002);Special issue on Situated and Embodied CognitionGoogle Scholar
  17. 17.
    Clancey, W.J.: Automating Capcom: Pragmatic operations and technology research for human exploration of Mars. In: Cockell, C. (ed.) Martian Expedition Planning. AAS Science and Technology Series, vol. 107, pp. 411–430 (2004)Google Scholar
  18. 18.
    Clancey, W.J.: Roles for agent assistants in field science: Understanding personal projects and collaboration. IEEE Transactions on Systems, Man, and Cybernetics—Part C: Applications and Reviews 32(2) (2004)Google Scholar
  19. 19.
    Clancey, W.J., Sierhuis, M., Damer, B., Brodsky, B.: Cognitive modeling of social behavior. In: Sun, R. (ed.) Cognition and Multi-Agent Interaction: From Cognitive Modeling to Social Simulation. Cambridge University Press, New York City (2005)Google Scholar
  20. 20.
    Clark, H.H.: Using Language. Cambridge University Press, Cambridge (1996)Google Scholar
  21. 21.
    Cohen, P.R., Levesque, H.J.: Teamwork. SRI International, Menlo Park (1991)Google Scholar
  22. 22.
    Davis, L.E., Rough, J., Ceccine, G., Gareban-Schaefer, A., Zeman, L.L.: Hurricane Katrina: Lessons for Army Planning and Operations. Rand Corporation, Santa Monica (2007)Google Scholar
  23. 23.
    Feltovich, P., Bradshaw, J.M., Jeffers, R., Suri, N., Uszok, A.: Social order and adaptability in animal and human cultures as an analogue for agent communities: Toward a policy-based approach. In: Omacini, A., Petta, P., Pitt, J. (eds.) Engineering Societies for the Agents World IV. LNCS, vol. 3071, pp. 21–48. Springer, Berlin (2004)Google Scholar
  24. 24.
    Feltovich, P., Bradshaw, J.M., Clancey, W.J., Johnson, M.: Toward and Ontology of Regulation: Socially-based Support for Coordination in Human and Machine Joint Activity. In: O’Hare, G.M.P., Ricci, A., O’Grady, M.J., Dikenelli, O. (eds.) ESAW 2007. LNCS (LNAI), vol. 4457, pp. 175–192. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  25. 25.
    Fong, T.W., Thorpe, C., Baur, C.: Robot as partner: Vehicle teleoperation with collaborative control. In: Workshop on Multi-Robot Systems, Naval Research Laboratory, Washington, DC (March 2002)Google Scholar
  26. 26.
    Fong, T.W., Nourbaksh, I., Ambrose, R., Simmon, R., Scholtz, J.: The peer-to-peer human-robot interaction project. In: AIAA Space 2005 (September 2005)Google Scholar
  27. 27.
    Goodrich, M.A., Olsen Jr., D.R., Crandall, J.W., Palmer, T.J.: Experiments in adjustable autonomy. In: Proceedings of the IJCAI_01 Workshop on Autonomy, Delegation, and Control: Interacting with Autonomous Agents, Seattle, WA (August 2001)Google Scholar
  28. 28.
    Johnson, M., Bradshaw, J.M., Feltovich, P., Jeffers, R., Uszok, A.: A semantically-rich policy-based approach to robot control. In: Proceedings of the International Conference on Informatics in Control, Automation, and Robotics, Lisbon, Portugal (2006)Google Scholar
  29. 29.
    Johnson, M., Feltovich, P.J., Bradshaw, J.M., Bunch, L.: Human-robot coordination through dynamic regulation. In: Proceedings of the International Conference on Robotics and Automation (ICRA), Pasadena, CA (in press, 2008)Google Scholar
  30. 30.
    Johnson, M., Intlekofer Jr., K., Jung, H., Bradshaw, J.M., Allen, J., Suri, N., Carvalho, M.: Coordinated operations in mixed teams of humans and robots. In: Marik, V., Bradshaw, J.M., Meyer, J. (eds.) Proceedings of the First IEEE Conference on Distributed Human-Machine Systems (DHMS 2008), Athens, Greece (in press, 2008)Google Scholar
  31. 31.
    Kelly, G.A.: The Psychology of Personal Constructs. Two vols. Norton, New York (1955)Google Scholar
  32. 32.
    Kelly, G.A.: A Theory of Personality. W. W. Norton & Co, New York City (1963)Google Scholar
  33. 33.
    King, I., Mason, W.: Peace at Any Price: How the World Failed Kosovo. Cornell University Press, Ithaca (2006)Google Scholar
  34. 34.
    Klein, G., Feltovich, P.J., Bradshaw, J.M., Woods, D.D.: Common ground and coordination in joint activity. In: Rouse, W.B., Boff, K.R. (eds.) Organizational Simulation, pp. 139–184. John Wiley, New York City (2004)Google Scholar
  35. 35.
    Klein, G., Woods, D.D., Bradshaw, J.M., Hoffman, R., Feltovich, P.: Ten challenges for making automation a team player in joint human-agent activity. IEEE Intelligent Systems 19(6), 91–95 (2004)CrossRefGoogle Scholar
  36. 36.
    Leont’ev, A.N.: The problem of activity in psychology. In: Wertsch, J.V. (ed.) The Concept of Activity in Soviet Psychology. M. E. Sharpe, Armonk (1979)Google Scholar
  37. 37.
    Newtson, D.: Attribution and the unit of perception of ongoing behavior. Journal of Personality and Social Psychology 28, 28–38 (1973)CrossRefGoogle Scholar
  38. 38.
    Norman, D.A.: Categorization of action slips. Psychological Review 88, 1–15 (1981)CrossRefMathSciNetGoogle Scholar
  39. 39.
    Norman, D.A.: The ‘problem’ with automation: Inappropriate feedback and interaction, not ‘over-automation’. In: Broadbend, D.E., Reason, J., Baddeley, A. (eds.) Human Factors in Hazardous Situations, pp. 137–145. Clarendon Press, Oxford (1990)Google Scholar
  40. 40.
    Patterson, E.S., Watts-Perotti, J., Woods, D.D.: Voice loops as coordination aids in Space Shuttle Mission Control. Computer Supported Cooperative Work 8, 353–371 (1999)CrossRefGoogle Scholar
  41. 41.
    Sarter, N., Woods, D.D., Billings, C.E.: Automation surprises. In: Salvendy, G. (ed.) Handbook of Human factors/Ergonomics, 2nd edn. John Wiley, New York (1997)Google Scholar
  42. 42.
    Sierhuis, M., Bradshaw, J.M., Acquisti, A., Van Hoof, R., Jeffers, R., Uszok, A.: Human-agent teamwork and adjustable autonomy in practice. In: Proceedings of the Seventh International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS), Nara, Japan, May 19-23 (2003)Google Scholar
  43. 43.
    Smith, W.J.: The biological bases of social attunement. Journal of Contemporary Legal Issues 6, 361–371 (1995)Google Scholar
  44. 44.
    Tambe, M., Shen, W., Mataric, M., Pynadath, D.V., Goldberg, D., Modi, P.J., Qiu, Z., Salemi, B.: Teamwork in cyberspace: Using TEAMCORE to make agents team-ready. In: Proceedings of the AAAI Spring Symposium on Agents in Cyberspace, Menlo Park, CA (1999)Google Scholar
  45. 45.
    Uszok, A., Bradshaw, J.M., Johnson, M., Jeffers, R., Tate, A., Dalton, J., Aitken, S.: KAoS policy management for semantic web services. IEEE Intelligent Systems 19(4), 32–41 (2004)CrossRefGoogle Scholar
  46. 46.
    Winograd, T.: A language/action perspective on the design of cooperative work. Human-Computer Interaction 3(1), 3–30 (1987-1988)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Paul J. Feltovich
    • 1
  • Jeffrey M. Bradshaw
    • 1
  • William J. Clancey
    • 1
  • Matthew Johnson
    • 1
  • Larry Bunch
    • 1
  1. 1.Florida Institute for Human & Machine Cognition (IHMC)PensacolaUSA

Personalised recommendations