Advertisement

Human-Machine Teaming and Cyberspace

  • Fernando J. Maymí
  • Robert Thomson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10915)

Abstract

Artificial Intelligence is becoming the key enabler of solutions to a variety of problems including those associated with cyberspace operations. Based on our analysis of cyber threats and opportunities in the coming years, we assess it as very likely that teams consisting of humans and synthetic agents will routinely work together in many if not most organizations. To fully leverage the potential of these teams, we must continue to develop new paradigms in human-machine teaming. Specifically, we must address three areas that are currently in their infancy. Firstly, we need interfaces that allow all teammates to communicate effectively with each other and seamlessly transfer tasks among them. This must be true regardless of whether the endpoints are human or not. Secondly, we will need cybersecurity operators with broad knowledge and skills. They must know how their synthetic teammates “think,” when to task them and when to question their reports. Thirdly, our AI systems must be able to explain their decision-making processes to their human teammates. This paper provides an overview of cyberspace threats and opportunities in the next ten years and how these will impact human-machine teaming. We then apply the key lessons we have learned while working a multitude of advanced research projects at the intersection of human and AI agents to cyberspace operations. Finally, we propose areas of research that will allow humans and machines to better collaborate in the future.

Keywords

Human-machine teaming Artificial intelligence Cyberspace 

References

  1. Abbas, H., Petraki, E., Kasmarik, K., Harvey, J.: Trusted autonomy and cognitive cyber symbiois: open challenges. Cogn. Comput. 8(3), 1–24 (2015)Google Scholar
  2. Aiken, M.: The Cyber Effect: A Pioneering Cyberpsychologist Explains How Human Behavior Changes Online. Spiegel & Grau, New York (2017)Google Scholar
  3. Asgharpour, F., Liu, D., Camp, L.J.: Mental models of computer security risks. In: Dietrich, S., Dhamija, R. (eds.) International Conference on Financial Cryptography and Data Security. Lecture Notes in Computer Science, vol. 47886, pp. 367–377. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-77366-5_34CrossRefGoogle Scholar
  4. Baumann, M.R., Bonner, B.L.: Member awareness of expertise, information sharing, information weighting, and group decision making. Small Group Res. 44, 532–562 (2013)CrossRefGoogle Scholar
  5. Baker, M.: Striving for Effective Cyber Workforce Development. Software Engineering Institute, Carnegie Mellon University, Pittsburgh (2016)Google Scholar
  6. Ben-Asher, N., Gonzalez, C.: Effects of cyber security knowledge on attack detection. Comput. Hum. Behav. 48, 51–61 (2015)CrossRefGoogle Scholar
  7. Bearman, C.R., Paletz, S.B.F., Orasanu, J., Thomas, M.J.W.: The breakdown of coordinated decision making in distributed systems. Hum. Factors 52, 173–188 (2010)CrossRefGoogle Scholar
  8. Brooks, T.N.: Survey of Automated Vulnerability Detection and Exploit Generation Techniques in Cyber Reasoning Systems (2017). arXiv preprint arXiv:1702.06162
  9. Buchanan, A., Goodall, L., Walczak, D’Amico, P.: Mission impact of cyber events: Scenarios and ontology to express the relationship between cyber assets (2009). http://www.dtic.mil/cgibin/GetTRDoc?AD=ADA517410
  10. Buchler, N., Rajivan, P., Marusich, L., Lightner, L., Gonzalez, C.: Sociometrics and observational assessment of teaming and leadership in a cyber security defense competition. J. Comput. Secur. 73, 114–136 (2017)CrossRefGoogle Scholar
  11. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57. IEEE, May 2017Google Scholar
  12. Caulkins, B.D., Badillo-Urquiola, K., Bockelman, P., Leis, R.: Cyber workforce development using a behavioral cybersecurity paradigm. In: Connelly, C., Brantly, A., Thomson, R., Vanatta, N., Maxwell, P., Thomson, D. (eds.) International Conference for Cyber Conflict US. Army Cyber Institute, West Point (2016)Google Scholar
  13. Champion, M.A., Rajivan, P., Cooke, N.J., Jariwala, S.: Team-based cyber defense analysis. In: 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 218–221 (2012)Google Scholar
  14. Champion, M., Jariwala, S., Ward, P., Cooke, N.J.: Using cognitive task analysis to investigate the contribution of informational education to developing cyber security expertise. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 58(1), 310–314 (2014)CrossRefGoogle Scholar
  15. Chen, J.Y., Barnes, M.J.: Human-agent teaming for multirobot control: a review of human factors issues. IEEE Trans. Hum. Mach. Inter. 44(1), 13–29 (2014)CrossRefGoogle Scholar
  16. Cohen, W.W.: Learning rules that classify e-mail. In: AAAI Spring Symposium on Machine Learning in Information Access, vol. 18, p. 25, March 1996Google Scholar
  17. Cook, M.: Cyber Acquisition Professionals Need Expertise (But They Don’t Necessarily Need to be Experts). Defense Acquisition University, Fort Belvoir (2014)Google Scholar
  18. Cooke, N.J., Gorman, J.C., Myers, C.W., Duran, J.L.: Interactive team cognition. Cogn. Sci. 37, 255–285 (2013).  https://doi.org/10.1111/cogs.12009CrossRefGoogle Scholar
  19. Cooke, N.J., Shope, S.M.: Designing a synthetic task environment. In: Scaled Worlds: Development, Validation, and Application, pp. 263–278 (2004)Google Scholar
  20. Dvorsky, G.: Hackers Have Already Started to Weaponize Artificial Intelligence 11 September 2017. https://gizmodo.com/hackers-have-already-started-to-weaponize-artificial-in-1797688425. Accessed 22 Feb 2018
  21. Emmanuel, Z.: Security experts air concerns over hackers using AI and machine learning for phishing attacks, 5 October 2017. http://www.computerweekly.com/news/450427653/Security-experts-air-concerns-over-hackers-using-AI-and-machine-learning-for-phishing-atttacks. Accessed 22 Feb 2018
  22. Franke, U., Brynielsson, J.: Cyber situational awareness: a systematic review of the literature. Comput. Secur. 46, 18–31 (2014)CrossRefGoogle Scholar
  23. Gonzalez, C., Ben-Asher, N., Oltramari, A., Lebiere, C.: Cognitive models of cyber situation awareness and decision making. In: Wang, C., Kott, A., Erbacher, R. (eds.) Cyber Defense and Situation Awareness. Springer (in press)Google Scholar
  24. Granåsen, M., Andersson, D.: Measuring team effectiveness in cyber-defense exercises: a cross-disciplinary case study. Cogn. Technol. Work 18(1), 121–143 (2016)CrossRefGoogle Scholar
  25. Hackman, J.R., Katz, N.: Group Behavior and Performance, pp. 1208–1251. Wiley, New York (2010)Google Scholar
  26. Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.D.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58. ACM, October 2011Google Scholar
  27. Internet Live Stats - Internet Usage & Social Media Statistics. Accessed 19 Feb 2018. http://www.internetlivestats.com/
  28. Jajodia, S., Liu, P., Swarup, V., Wang, C.: Cyber Situational Awareness. Springer Publishing, New York (2010)CrossRefGoogle Scholar
  29. Jariwala, S., Champion, M., Rajivan, P., Cooke, N.J.: Influence of team communication and coordination on the performance of teams at the iCTF competition. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 56(1), pp. 458–462. SAGE Publications, September 2012CrossRefGoogle Scholar
  30. Joint Publication 3–12: Cyberspace Operations, Washington, DC: Joint Chiefs of Staff, U.S. Department of Defense (2013)Google Scholar
  31. Knowles, W., Prince, D., Hutchison, D., Disso, J.F., Jones, K.: A survey of cyber security management in industrial control systems. Int. J. Crit. Infrastruct. Protections 9, 52–80 (2015)CrossRefGoogle Scholar
  32. Kott, A., Alberts, D., Zalman, A., Shakarian, P., Maymi, F., Wang, C., Qu, G.: Visualizing the tactical ground battlefield in the year 2050: Workshop report (No. ARL-SR-0327). Army Research Lab Adelphi Maryland (2015)Google Scholar
  33. Metz, R.: Why Microsofts teen chatbot, Tay, said lots of awful things online, 24 March 2016. https://www.technologyreview.com/s/601111/why-microsoft-accidentally-unleashed-a-neo-nazi-sexbot/. Accessed 23 Feb 2018
  34. Newhouse, B., Keith, S.S., Witte, G.: NICE Cybersecurity Workforce Framework. National Institute of Standards and Technology, Gathersburg (2016)Google Scholar
  35. Onwubiko, C., Owens, T.J.: Situational Awareness in Computer Network Defense: Principles, Methods and Applications. Information Science Reference, Hershey (2011)Google Scholar
  36. Paletz, S.B.F.: Multidisciplinary teamwork and big data. In: Human-Centered Big Data Workshop, At Raleigh, NC (2014).  https://doi.org/10.1145/2609876.2609884
  37. Papernot, N., McDaniel, P., Sinha, A., Wellman, M.: Towards the science of security and privacy in machine learning (2016). arXiv preprint arXiv:1611.03814
  38. Pellerin, C.: Work: Human-Machine Teaming Represents Defense Technology Future (2015). https://www.defense.gov/News/Article/Article/628154/work-human-machine-teaming-represents-defense-technology-future/. Accessed 1 Feb 2018
  39. Pfleeger, S.L., Caputo, D.D.: Leveraging behavioral science to mitigate cyber security risk. Comput. Secur. 31(4), 597–611 (2012)CrossRefGoogle Scholar
  40. Proctor, R.W., Chen, J.: The role of human factors/ergonomics in the science of security decision making and action selection in cyberspace. Hum. Factors J. Hum. Factors Ergon. Soc. (2015).  https://doi.org/10.1177/0018720815585906
  41. Rajivan, P., Champion, M., Cooke, Nancy J., Jariwala, S., Dube, G., Buchanan, V.: Effects of teamwork versus group work on signal detection in cyber defense teams. In: Schmorrow, Dylan D., Fidopiastis, Cali M. (eds.) AC 2013. LNCS (LNAI), vol. 8027, pp. 172–180. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-39454-6_18CrossRefGoogle Scholar
  42. Rajivan, P.: Information Pooling Bias in Collaborative Cyber Forensics (Doctoral dissertation, Arizona State University) (2014)Google Scholar
  43. Rajivan, P., Cooke, N.: Information Pooling Bias in Collaborative Security Incident Analysis. Human Factors (in press)Google Scholar
  44. Rajivan, P., Moriano, P., Kelley, T., Camp, J.: Factors in an end user security expertise instrument. Inf. Comput. Secur. 25(2), 190–205 (2017)CrossRefGoogle Scholar
  45. Salas, E., Dickinson, T.L., Converse, S.A., Tannenbaum, S.I.: Toward an understanding of team performance and training. In: Teams their Training and Performance, pp. 3–29 (1992)Google Scholar
  46. Seymour, J., Tully, P.: Weaponizing data science for social engineering: Automated E2E Spear Phishing on Twitter. Black Hat USA, 37 (2016)Google Scholar
  47. Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18. IEEE, May 2017Google Scholar
  48. Shortliffe, E.: Computer-Based Medical Consultations: MYCIN. Elsevier, New York (2012)CrossRefGoogle Scholar
  49. Simsarian Webber, S.: Leadership and trust facilitating cross-functional team success. J. Manage. Dev. 21(3), 201–214 (2002)CrossRefGoogle Scholar
  50. Spiro, R.J.: Cognitive Flexibility Theory: Advanced Knowledge Acquisition in Ill-Structured Domains. Technical Report No. 441 (1988). http://eric.ed.gov/?id=ED302821. Accessed 5 Oct 2017
  51. Srinidhi, B., Yan, J., Tayi, G.K.: Allocation of resources to cyber-security: the effect of misalignment of interest between managers and investors. Decis. Support Syst. 75(1), 49–62 (2015). http://doi.org/10.1016/j.dss.2015.04.011CrossRefGoogle Scholar
  52. Stasser, G., Titus, W.: Pooling of unshared information in group decision making: biased information sampling during discussion. J. Pers. Soc. Psychol. 48(6), 1467 (1985)CrossRefGoogle Scholar
  53. Trexler, E.: Why Human-Machine teaming is the future of cybersecurity (2017). https://federalnewsradio.com/commentary/2017/11/why-human-machine-teaming-is-the-future-of-cybersecurity/. Accessed 1 Feb 2018
  54. Tung, L.: IoT devices will outnumber the world’s population this year for the first time 13 February 2017. http://www.zdnet.com/article/iot-devices-will-outnumber-the-worlds-population-this-year-for-the-first-time/. Accessed 19 Feb 2018
  55. Veksler, B.Z.: Visual search strategies and the layout of the display. In: Salvucci, D.D., Gunzelmann, G. (eds.) Proceedings of the 10th International Conference on Cognitive Modeling, pp. 323–324. Drexel University, Philadelphia (2010)Google Scholar
  56. Vieane, A., Funke, G., Mancuso, V., Greenlee, E., Dye, G., Borghetti, B., Brown, R.: Coordinated displays to assist cyber defenders. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60(1), pp. 344–348. SAGE Publications, September 2016CrossRefGoogle Scholar
  57. Whitley, J., D’Amico, K.: The real work of computer network defense analysts. In: Goodall, J.R., Conti, G., Ma, K.L. (eds.) Workshop on Visualization for Computer Security, pp. 19–37 (2008)Google Scholar

Copyright information

© This is a U.S. government work and its text is not subject to copyright protection in the United States; however, its text may be subject to foreign copyright protection  2018

Authors and Affiliations

  1. 1.Soar TechnologyAnn ArborUSA
  2. 2.Army Cyber InstituteWest PointUSA

Personalised recommendations