Skip to main content

Advertisement

SpringerLink
Log in

Towards building an affect-aware dialogue agent with deep neural networks

  • S.I. : Visvesvaraya
  • Published:
CSI Transactions on ICT Aims and scope Submit manuscript

Abstract

In this paper, some of the recent developments, aiming at building human-like Conversational Artificial Intelligence (AI) agents, have been presented very briefly. These robust dialogue systems are capable in dealing with the various affect attributes, such as sentiment, emotion, and courteousness. Firstly, the motivation, background and impact of these new frontiers of Artificial Intelligence (AI), Machine Learning (ML) and Natural Language Processing (NLP) have been described. Thereafter, two of our very recent research have been presented, where the first one attempts at incorporating courteousness in a dialogue agent, and the second one addresses natural language generation in a multi-modal setup involving text and images both.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://chatbotsmagazine.com/chatbot-report-2019-global-trends-and-an alysis-a487afec05b

References

  1. Vinyals O, Le Q (2015) A neural conversational model. arXiv:1506.05869

  2. Shen X, Su H, Niu S, and Demberg V (2018) Improving variational encoder-decoders in dialogue generation. In: McIlraith SA, Weinberger KQ (eds.) Proceedings of the Thirty-Second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), AAAI Press, New Orleans, Louisiana, USA, February 2–7, pp 5456–5463

  3. Wu X, Martinez A, Klyen M (2018) Dialog generation using multi-turn reasoning neural networks. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, Volume 1 (Long Papers), vol 1, pp 2049–2059

  4. Serban IV, Klinger T, Tesauro G, Talamadupula K, Zhou B, Bengio Y, Courville AC (2017) Multiresolution recurrent neural networks: an application to dialogue response generation. In: AAAI, pp 3288–3294

  5. Raghu D, Gupta N, et al (2018) Hierarchical pointer memory network for task oriented dialogue. arXiv:1805.01216

  6. Zhang H, Lan Y, Guo J, Xu J, Cheng X (2018) Reinforcing coherence for sequence to sequence model in dialogue generation. In: IJCAI, pp 4567–4573

  7. Li J, Monroe W, Ritter A, Jurafsky D, Galley M, Gao J (2016) Deep reinforcement learning for dialogue generation. In: Proceedings of the 2016 conference on empirical methods in natural language processing, 1–5 Nov 2016. Austin, Texas, pp 1192–1202

  8. Reddy S, Raghu D, Khapra MM, Joshi S (2017) Generating natural language question-answer pairs from a knowledge graph using a rnn based question generation model. In: Proceedings of the 15th conference of the European chapter of the association for computational linguistics, Long Papers, vol 1, pp 376–385

  9. Duan N, Tang D, Chen P, Zhou M (2017) Question generation for question answering. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 866–874

  10. Zhou H, Huang M, Zhang T, Zhu X, Liu B (2018) Emotional chatting machine: emotional conversation generation with internal and external memory. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the Thirty-Second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2–7, 2018, pp 730–739

  11. Wang K, Wan X (2018) Sentigan: generating sentimental texts via mixture adversarial networks. In: IJCAI, pp 4446–4452

  12. Serban IV, Sordoni A, Bengio Y, Courville A, Pineau J (2015) Hierarchical neural network generative models for movie dialogues. vol 7, no 8. arXiv:1507.04808

  13. Lin Z, Xu P, Winata GI, Siddique FB, Liu Z, Shin J, Fung P (2020) Caire: an end-to-end empathetic chatbot. In: The Thirty-Fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, The tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, AAAI Press, New York, February 7–12, 2020, pp 13622–13623

  14. Rashkin H, Smith EM, Li M, Boureau Y-L (2019) Towards empathetic open-domain conversation models: a new benchmark and dataset. In: Proceedings of the 57th conference of the association for computational linguistics, pp 5370–5381

  15. Song Z, Zheng X, Liu L, Xu M, Huang X-J (2019) Generating responses with a specific emotion in dialog. In: Proceedings of the 57th conference of the association for computational linguistics, pp 3685–3695

  16. Das A, Kottur S, Gupta K, Singh A, Yadav D, Moura JM, Parikh D, Batra D (2017) Visual dialog. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 326–335

  17. Shi W, Yu Z (2018) Sentiment adaptive end-to-end dialog systems. In: Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 1: Long Papers), (Melbourne, Australia), Association for Computational Linguistics, pp 1509–1519

  18. Poria S, Hazarika D, Majumder N, Naik G, Cambria E, Mihalcea R (2019) MELD: a multimodal multi-party dataset for emotion recognition in conversations. In: Korhonen A, Traum DR, Màrquez L (eds) Proceedings of the 57th conference of the association for computational linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp 527–536

  19. Majumder N, Poria S, Hazarika D, Mihalcea R, Gelbukh AF, Cambria E (2019) Dialoguernn: an attentive RNN for emotion detection in conversations. In: The thirty-third AAAI conference on artificial intelligence, AAAI 2019, The Thirty-First innovative applications of artificial intelligence conference, IAAI 2019, The ninth AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27–February 1, 2019, pp 6818–6825

  20. Akhtar MS, Chauhan DS, Ekbal A (2020) A deep multi-task contextual attention framework for multi-modal affect analysis. ACM Trans. Knowl. Discov. Data, vol 14, no 3, pp 32:1–32:27

  21. Chauhan DS, Akhtar MS, Ekbal A, Bhattacharyya P (2019) Context-aware interactive attention for multi-modal sentiment and emotion analysis. In: Inui K, iang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Association for Computational Linguistics, Hong Kong, China, November 3–7, 2019, pp 5646–5656

  22. Ghosal D, Akhtar MS, Chauhan DS, Poria S, Ekbal A, Bhattacharyya P (2018) Contextual inter-modal attention for multi-modal sentiment analysis. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds) Proceedings of the 2018 conference on empirical methods in natural language processing, Association for Computational Linguistics, Brussels, Belgium, October 31–November 4, 2018, pp 3454–3466

  23. Golchha H, Firdaus M, Ekbal A, Bhattacharyya P (2019) Courteously yours: inducing courteous behavior in customer care responses using reinforced pointer generator network. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp 851–860

  24. Chauhan H, Firdaus M, Ekbal A, Bhattacharyya P (2019) Ordinal and attribute aware response generation in a multimodal dialogue system. In: Proceedings of the 57th conference of the association for computational linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, Volume 1: Long Papers, pp 5437–5447

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  26. Yu Z, Yu J, Fan J, Tao D (2017) Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In: Proceedings of the IEEE international conference on computer vision, pp 1821–1830

  27. Saha A, Khapra MM, Sankaranarayanan K (2018) Towards building large scale multimodal domain-aware conversation systems. In: Thirty-second AAAI conference on artificial intelligence, pp 696–704

  28. Agarwal S, Dušek O, Konstas I, Rieser V (2018) Improving context modelling in multimodal dialogue generation. In: Proceedings of the 11th international conference on natural language generation, (Tilburg University, The Netherlands), pp 129–134

  29. Agarwal S, Dusek O, Konstas I, Rieser V (2018) A knowledge-grounded multimodal search-based conversational agent. arXiv:1810.11954

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

Authors
  1. Asif Ekbal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Asif Ekbal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ekbal, A. Towards building an affect-aware dialogue agent with deep neural networks. CSIT 8, 249–255 (2020). https://doi.org/10.1007/s40012-020-00304-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40012-020-00304-5

Keywords

Search

Navigation