World Wide Web

, Volume 22, Issue 4, pp 1427–1446 | Cite as

Neural personalized response generation as domain adaptation

  • Wei-Nan ZhangEmail author
  • Qingfu Zhu
  • Yifa Wang
  • Yanyan Zhao
  • Ting Liu


One of the most crucial problem on training personalized response generation models for conversational robots is the lack of large scale personal conversation data. To address the problem, we propose a two-phase approach, namely initialization then adaptation, to first pre-train an optimized RNN encoder-decoder model (LTS model) in a large scale conversational data for general response generation and then fine-tune the model in a small scale personal conversation data to generate personalized responses. For evaluation, we propose a novel human aided method, which can be seen as a quasi-Turing test, to evaluate the performance of the personalized response generation models. Experimental results show that the proposed personalized response generation model outperforms the state-of-the-art approaches to language model personalization and persona-based neural conversation generation on the automatic evaluation, offline human judgment and the quasi-Turing test.


Personalized response generation Conversation generation Sequence to sequence learning Domain adaptation 



This paper is supported by NSFC (No. 61502120, 61472105, 61772153) and Heilongjiang philosophy and social science research project (No. 16TQD03).


  1. 1.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)Google Scholar
  2. 2.
    Banchs, R.E.: Movie-dic: A movie dialogue corpus for research and development. In: ACL (2012)Google Scholar
  3. 3.
    Banchs, RE, Li, H: Iris: A chat-oriented dialogue system based on the vector space model. In: ACL, pp. 37–42 (2012)Google Scholar
  4. 4.
    Bang, J, Noh, H, Kim, Y, Lee, GG: Example-based chat-oriented dialogue system with personalized long-term memory. In: ICBDSC, pp. 238–243 (2015)Google Scholar
  5. 5.
    Bergstra, J., Bastien, F., Breuleux, O., Lamblin, P., Pascanu, R., Delalleau, O., Desjardins, G., Warde-Farley, D., Goodfellow, I.J., Bergeron, A., Bengio, Y.: Theano: Deep learning on gpus with python. In: NIPS (2011)Google Scholar
  6. 6.
    Berry, P.M., Gervasio, M., Peintner, B., Yorke-Smith, N.: Ptime: Personalized assistance for calendaring. ACM Trans. Intell. Syst. Technol. (TIST) 2(4), 40 (2011)Google Scholar
  7. 7.
    Bin, Y, Yang, Y, Shen, F, Xie, N, Shen, HT, Li, X: Describing video with attention-based bidirectional LSTM. IEEE Transactions on Cybernetics (2018)Google Scholar
  8. 8.
    Cao, J., Wu, Z., Mao, B., Zhang, Y.: Shilling attack detection utilizing semi-supervised learning method for collaborative recommender system. World Wide Web-internet Web Inf. Syst. 16(5–6), 729–748 (2013)CrossRefGoogle Scholar
  9. 9.
    Casanueva, I, Hain, T, Christensen, H, Marxer, R, Green, P: Knowledge transfer between speakers for personalised dialogue management. In: SIGDD, pp. 12–21 (2015)Google Scholar
  10. 10.
    Cho, K., Merrienboer, B.V., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H, Bengio, Y: Learning phrase representations using rnn encoder-decoder for statistical machine translation (2014)Google Scholar
  11. 11.
    Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76(5), 378 (1971)CrossRefGoogle Scholar
  12. 12.
    Genevay, A, Laroche, R: Transfer learning for user adaptation in spoken dialogue systems. In: ICAAMS, pp. 975–983 (2016)Google Scholar
  13. 13.
    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. In: ICML, pp. 1319–1327 (2013)Google Scholar
  14. 14.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  15. 15.
    Kim, Y., Bang, J., Choi, J., Ryu, S., Koo, S., Lee, G.G.: Acquisition and use of long-term memory for personalized dialog systems. In: MAAAHMI, pp. 78–87 (2014)Google Scholar
  16. 16.
    Li, J., Galley, M., Brockett, C., Gao, J, Dolan, B: A diversity-promoting objective function for neural conversation models. NAACL (2015)Google Scholar
  17. 17.
    Li, J, Galley, M, Brockett, C, Spithourakis, G, Gao, J, Dolan, B: A persona-based neural conversation model. In: ACL, pp. 994–1003 (2016)Google Scholar
  18. 18.
    Li, J, Monroe, W, Ritter, A, Dan, J: Deep reinforcement learning for dialogue generation, pp. 1192–1202 (2016)Google Scholar
  19. 19.
    Luan, Y., Ji, Y, Ostendorf, M: Lstm based conversation models (2016)Google Scholar
  20. 20.
    Luan, Y., Brockett, C., Dolan, B., Gao, J., Galley, M.: Multi-task learning for speaker-role adaptation in neural conversation models. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1, Long Papers). Asian Federation of Natural Language Processing, pp. 605–614. Taipei (2017)Google Scholar
  21. 21.
    Mairesse, F., Young, S.: Stochastic language generation in dialogue using factored language models. Comput Linguis. 40(4), 763–799 (2014)CrossRefGoogle Scholar
  22. 22.
    Mairesse, F, Jurcicek, M, Ek, F, Keizer, S, Thomson, B, Yu, K, Young, S: Phrase-based statistical language generation using graphical models and active learning. In: ACL, pp. 1552–1561 (2010)Google Scholar
  23. 23.
    Marjan, G., Chris, B., Ming-Wei, C., Bill, D., Jianfeng, G., Wen-tau, Y, Michel, G: A knowledge-grounded neural conversation model (2018)Google Scholar
  24. 24.
    Mei, H., Bansal, M, Walter, M: Coherent dialogue with attention-based language models (2017)Google Scholar
  25. 25.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. NIPS 26, 3111–3119 (2013)Google Scholar
  26. 26.
    Mo, K., Li, S., Zhang, Y., Li, J, Yang, Q: Personalizing a dialogue system with transfer learning (2016)Google Scholar
  27. 27.
    Papineni, K, Roukos, S, Ward, T, Zhu, WJ: Bleu: A method for automatic evaluation of machine translation. In: ACL, pp. 311–318 (2002)Google Scholar
  28. 28.
    Peng, M., Zeng, G., Sun, Z., Huang, J., Wang, H., Tian, G.: Personalized app recommendation based on app permissions. World Wide Web-internet Web Inf. Syst., 1–16 (2017)Google Scholar
  29. 29.
    Rasch, K., Li, F., Sehic, S., Ayani, R., Dustdar, S.: Context-driven personalized service discovery in pervasive environments. World Wide Web-internet Web Inf. Syst. 14(4), 295–319 (2011)CrossRefGoogle Scholar
  30. 30.
    Ritter, A, Cherry, C, Dolan, B: Unsupervised modeling of twitter conversations. In: NAACL, pp. 172–180 (2010)Google Scholar
  31. 31.
    Ritter, A, Cherry, C, Dolan, WB: Data-driven response generation in social media. In: EMNLP, pp. 583–593 (2011)Google Scholar
  32. 32.
    Serban, I.V., Sordoni, A., Bengio, Y., Courville, A, Pineau, J: Hierarchical neural network generative models for movie dialogues (2015)Google Scholar
  33. 33.
    Serban, I.V., Sordoni, A., Bengio, Y., Courville, A, Pineau, J: Building end-to-end dialogue systems using generative hierarchical neural network models. Computer Science (2016)Google Scholar
  34. 34.
    Shang, L, Lu, Z, Li, H: Neural responding machine for short-text conversation. In: ACL, pp. 1577–1586 (2015)Google Scholar
  35. 35.
    Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., Nie, JY: A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In: CIKM (2015)Google Scholar
  36. 36.
    Sordoni, A, Galley, M, Auli, M, Brockett, C, Ji, Y, Mitchell, M, Nie, JY, Gao, J, Dolan, B: A neural network approach to context-sensitive generation of conversational responses. In: NAACL, pp. 196–205 (2015)Google Scholar
  37. 37.
    Sutskever, I., Vinyals, O., Le, Q.V., Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. NIPS 4, 3104–3112 (2014)Google Scholar
  38. 38.
    Vinyals, O, Le, Q: A neural conversational model. Computer Science (2015)Google Scholar
  39. 39.
    Wang, C, Zhang, M, Ma, S, Ru, L: Automatic online news issue construction in web environment. In: WWW, pp. 457–466 (2008)Google Scholar
  40. 40.
    Wang, D., Jojic, N., Brockett, C., Nyberg, E.: Steering output style and topic in neural response generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 2140-2150. Copenhagen (2017)Google Scholar
  41. 41.
    Wen, TH, Heidel, A, Lee, H.Y., Tsao, Y, Lee, LS: Recurrent neural network based language model personalization by social network crowdsourcing. In: INTERSPEECH, pp. 2703–2707 (2013)Google Scholar
  42. 42.
    Wen, T.H., Gasic, M., Kim, D., Mrksic, N., Su, P.H., Vandyke, D, Young, S: Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking. SIGDial (2015)Google Scholar
  43. 43.
    Wen, T.H., Gasic, M., Mrksic, N., Su, P.H., Vandyke, D, Young, S: Semantically conditioned lstm-based natural language generation for spoken dialogue systems. EMNLP (2015)Google Scholar
  44. 44.
    Wen, TH, Gašic, M, Mrkšic, N, Rojas-Barahona, LM, Su, PH, Vandyke, D, Young, S: Multi-domain neural network language generation for spoken dialogue systems. In: NAACL, pp. 120–129 (2016)Google Scholar
  45. 45.
    Xing, C, Wu, W, Wu, Y, Liu, J, Huang, Y, Zhou, M, Ma, WY: Topic augmented neural response generation with a joint attention mechanism. arXiv:1606.08340 (2016)
  46. 46.
    Yang, M, Zhao, Z, Zhao, W, Chen, X, Zhu, J, Zhou, L, Cao, Z: Personalized response generation via domain adaptation. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1021–1024. ACM (2017)Google Scholar
  47. 47.
    Yao, W., He, J., Huang, G., Cao, J., Zhang, Y.: Personalized recommendation on multi-layer context graph. Lect. Notes Comput. Sci. 8180, 135–148 (2013)CrossRefGoogle Scholar
  48. 48.
    Young, S., Gasic, M., Thomson, B., Williams, J.D.: Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE, 1160–1179 (2013)Google Scholar
  49. 49.
    Zhang, X, LeCun, Y: Text Understanding from Scratch. Computer Science (2015)Google Scholar
  50. 50.
    Zhou, X, He, J, Huang, G, Zhang, Y: A personalized recommendation algorithm based on approximating the singular value decomposition (approsvd). In: Ieee/wic/acm International Conferences on Web Intelligence and Intelligent Agent Technology, pp. 458–464 (2012)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Wei-Nan Zhang
    • 1
    Email author
  • Qingfu Zhu
    • 1
  • Yifa Wang
    • 1
  • Yanyan Zhao
    • 1
  • Ting Liu
    • 1
  1. 1.Research Center for Social Computing and Information RetrievalHarbin Institute of TechnologyHarbin CityChina

Personalised recommendations