Improving IT Support by Enhancing Incident Management Process with Multi-modal Analysis

  • Atri MandalEmail author
  • Shivali Agarwal
  • Nikhil Malhotra
  • Giriprasad Sridhara
  • Anupama Ray
  • Daivik Swarup
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11895)


IT support services industry is going through a major transformation with AI becoming commonplace. There has been a lot of effort in the direction of automation at every human touchpoint in the IT support processes. Incident management is one such process which has been a beacon process for AI based automation. The vision is to automate the process from the time an incident/ticket arrives till it is resolved and closed. While text is the primary mode of communicating the incidents, there has been a growing trend of using alternate modalities like image to communicate the problem. A large fraction of IT support tickets today contain attached image data in the form of screenshots, log messages, invoices and so on. These attachments help in better explanation of the problem which aids in faster resolution. Anybody who aspires to provide AI based IT support, it is essential to build systems which can handle multi-modal content.

In this paper we present how incident management in IT support domain can be made much more effective using multi-modal analysis. The information extracted from different modalities are correlated to enrich the information in the ticket and used for better ticket routing and resolution. We evaluate our system using about 25000 real tickets containing attachments from selected problem areas. Our results demonstrate significant improvements in both routing and resolution with the use of multi-modal ticket analysis compared to only text based analysis.


Service delivery Incident management Multimodal analysis Image understanding Automated routing and resolution 


  1. 1.
    Agarwal, S., Aggarwal, V., Akula, A.R., Dasgupta, G.B., Sridhara, G.: Automatic problem extraction and analysis from unstructured text in IT tickets. IBM J. Res. Dev. 61(1), 4:41–4:52 (2017)CrossRefGoogle Scholar
  2. 2.
    Agarwal, S., Sindhgatta, R., Sengupta, B.: SmartDispatch: enabling efficient ticket dispatch in an IT service environment. In: 18th ACM SIGKDD (2012)Google Scholar
  3. 3.
    Aggarwal, V., Agarwal, S., Dasgupta, G.B., Sridhara, G., Vijay, E.: ReAct: a system for recommending actions for rapid resolution of IT service incidents. In: IEEE International Conference on Services Computing, SCC 2016 (2016)Google Scholar
  4. 4.
    Botezatu, M.M., Bogojeska, J., Giurgiu, I., Voelzer, H., Wiesmann, D.: Multi-view incident ticket clustering for optimal ticket dispatching. In: 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2015, pp. 1711–1720 (2015)Google Scholar
  5. 5.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRefGoogle Scholar
  6. 6.
    Dasgupta, G.B., Nayak, T.K., Akula, A.R., Agarwal, S., Nadgowda, S.J.: Towards auto-remediation in services delivery: context-based classification of noisy and unstructured tickets. In: Franch, X., Ghose, A.K., Lewis, G.A., Bhiri, S. (eds.) ICSOC 2014. LNCS, vol. 8831, pp. 478–485. Springer, Heidelberg (2014). Scholar
  7. 7.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR09 (2009)Google Scholar
  8. 8.
    Duda, R.O., Hart, P.E.: Use of the hough transformation to detect lines and curves in pictures. Commun. ACM 15(1), 11–15 (1972)CrossRefGoogle Scholar
  9. 9.
    Gupta, A., Ray, A., Dasgupta, G., Singh, G., Aggarwal, P., Mohapatra, P.: Semantic parsing for technical support questions. In: COLING, Santa Fe, New Mexico, USA, August 2018Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  11. 11.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar
  12. 12.
    Maire, M.R.: Contour detection and image segmentation. Ph.D. thesis (2009)Google Scholar
  13. 13.
    Mandal, A., Malhotra, N., Agarwal, S., Ray, A., Sridhara, G.: Cognitive system to achieve human-level accuracy in automated assignment of helpdesk email tickets. ArXiv e-prints, August 2018Google Scholar
  14. 14.
    Mandal, A., Malhotra, N., Agarwal, S., Ray, A., Sridhara, G.: Cognitive system to achieve human-level accuracy in automated assignment of helpdesk email tickets. In: Pahl, C., Vukovic, M., Yin, J., Yu, Q. (eds.) ICSOC 2018. LNCS, vol. 11236, pp. 332–341. Springer, Cham (2018). Scholar
  15. 15.
    Mani, S., et al.: Hi, how can I help you? automating enterprise IT support help desks. CoRR abs/1711.02012 (2017).
  16. 16.
    Mori, S., Nishida, H., Yamada, H.: Optical Character Recognition. Wiley, New York (1999)Google Scholar
  17. 17.
    Sampat, A., Haskell, A.: CNN for task classification using computer screenshots for integration into dynamic calendar/task management systems.
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)Google Scholar
  19. 19.
    Smith, L.: Cyclical Learning Rates for Training Neural Networks, pp. 464–472, March 2017Google Scholar
  20. 20.
    Xu, J., Callan, J.: Effective retrieval with distributed collections. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1998, pp. 112–120. ACM (1998)Google Scholar
  21. 21.
    Zhou, W., Tang, L., Zeng, C., Li, T., Shwartz, L., Ya. Grabarnik, G.: Resolution recommendation for event tickets in service management. IEEE Trans. Netw. Serv. Manage. 13(4), 954–967 (2016)CrossRefGoogle Scholar
  22. 22.
    Zhou, W., et al.: Star: a system for ticket analysis and resolution. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2017, pp. 2181–2190 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Atri Mandal
    • 1
    Email author
  • Shivali Agarwal
    • 1
  • Nikhil Malhotra
    • 2
  • Giriprasad Sridhara
    • 1
  • Anupama Ray
    • 1
  • Daivik Swarup
    • 1
  1. 1.IBM Research AIBengaluruIndia
  2. 2.IBM Global Technology ServicesBengaluruIndia

Personalised recommendations